>> 2. Tests can only ever answer close-ended questions: "does this work?", but not "how does this work?", "what does this look like?" etc.
> I fail to understand this point. I mean you can obviously write tests that just run stuff and not throw an exception or error. Furthermore you can share how you set stuff up with other developers.... and again you can just delete it if its obnoxious.
Yes, but the point is that a test mostly gives you a Yes/No answer, not a visualization. What's more, sometimes you need to set up a fair amount of state as you explore (think of the examples in the video, where you call an external API based on a previous result of that external API etc.) - not something that is convenient to do in a test.
> > 3. Tests typically won't run in real-world conditions: they'll use simple, artificial data and mocks of services such as databases or API clients. As a result, they don't typically help you understand a problem that only happens on real-life data, nor do they give you confidence that the real-life implementations of the services they emulate do work.
> This is exactly what I do not like about REPLs. You setup a custom environment and its hard to keep track of what you have done. I don't like the "not repeatable" nature of it. I do think you make excellent points about how immutability helps that problem as stuff basically becomes a log but for other languages this is not the case.
It's not that the environment is custom, is that it's real. Do you call your payment service or your mail sending service from your tests? This is exactly the kind of thing you want to experiment with in a supervised, non-repeatable way.
You can put stuff in the REPL, or in your test suite, or both, and there are pitfalls in each case, but at least you have a choice; indeed, you have to use your best judgement to decide what will be persisted, repeated and shared with your team and what will be forgotten after your REPL session end, but I wouldn't call making the wrong choice a deficiency of the REPL, rather an error on the programmer's side.
> It's not that the environment is custom, is that it's real. Do you call your payment service or your mail sending service from your tests? This is exactly the kind of thing you want to experiment with in a supervised, non-repeatable way.
The irony you mentioning that is I have actually written "tests" that call braintree and stripe...
Just to clarify when I say "test" I don't mean some precise definition of "unit test" or "integration test". I mean in a runner that runs your code. Unit tests basically allow you to make a whole bunch of entry points (aka static main(args)).
I have many times setup a particularly environment and then repeatedly ran a unit test against that environment (I then commented or deleted that code later).
I think in large part of what your saying is good about REPLs is that they allow hot code swapping but that is only IMO really one part of the REPL. The key to really good REPL should be human input (think Excel) and human output (think images and graphs). There was a company recently shown on HN called Luna [1] and I think that is what a REPL should be.
And I particularly pick on this because you mention Bret Victor who is (err I guess was) actually working on environments like this.
Otherwise call it a unit test... call it a REPL... call it hot code swapping but in current JVM environments the difference can be pretty nebulous.
> I think in large part of what your saying is good about REPLs is that they allow hot code swapping but that is only IMO really one part of the REPL. The key to really good REPL should be human input (think Excel) and human output (think images and graphs).
I definitely could have spent more time on this part in the article. You may want to have a look at Proto-REPL (https://atom.io/packages/proto-repl), which is one of Clojure's REPLs (for the Atom editor)
>> 2. Tests can only ever answer close-ended questions: "does this work?", but not "how does this work?", "what does this look like?" etc.
> I fail to understand this point. I mean you can obviously write tests that just run stuff and not throw an exception or error. Furthermore you can share how you set stuff up with other developers.... and again you can just delete it if its obnoxious.
Yes, but the point is that a test mostly gives you a Yes/No answer, not a visualization. What's more, sometimes you need to set up a fair amount of state as you explore (think of the examples in the video, where you call an external API based on a previous result of that external API etc.) - not something that is convenient to do in a test.
> > 3. Tests typically won't run in real-world conditions: they'll use simple, artificial data and mocks of services such as databases or API clients. As a result, they don't typically help you understand a problem that only happens on real-life data, nor do they give you confidence that the real-life implementations of the services they emulate do work.
> This is exactly what I do not like about REPLs. You setup a custom environment and its hard to keep track of what you have done. I don't like the "not repeatable" nature of it. I do think you make excellent points about how immutability helps that problem as stuff basically becomes a log but for other languages this is not the case.
It's not that the environment is custom, is that it's real. Do you call your payment service or your mail sending service from your tests? This is exactly the kind of thing you want to experiment with in a supervised, non-repeatable way.
You can put stuff in the REPL, or in your test suite, or both, and there are pitfalls in each case, but at least you have a choice; indeed, you have to use your best judgement to decide what will be persisted, repeated and shared with your team and what will be forgotten after your REPL session end, but I wouldn't call making the wrong choice a deficiency of the REPL, rather an error on the programmer's side.