Lots of people in the comments here stuck on the boring semantic definition debate of mocks vs stubs vs nulls vs whatever.
This is not really by what the article is about IMHO. It’s a great overview of how to grow and organize your application with some opinionated patterns that give some favorable properties to your system. The nullable stuff just helps achieve those properties. If you want to use a mocking library to write nullables, and call them mocks, fine. Not important.
I think this might be a problem of expectation management that could stem from the title. A headline along the lines of “Patterns to organize your code for testability” might reflect the general character of the article more clearly. But as it specifically says “testing without mocks”, I was intuitively expecting something different, and even was a bit confused when the author suggested techniques that I would refer to as “mocking”.
Yes, exactly. “Bake your mocks into your third-party library implementations” is an interesting idea and might be a useful thing to do, but it’s certainly not “testing without mocks.”
Agreed and I find this happening a lot in discussion of testing. It seems everyone/every company has "their way" of doing testing, which at times even includes new words for certain concepts that they find useful (e.g. stubs) and then the conversation about testing gets muddled by people doing things/naming things differently.
I only recently got into testing my code and while I love writing tests, getting started was kind of difficult because there was just so much noise. I eventually had to find one person to follow in my space (Kent C. Dodds) and just kind of go all in on his methodology.
Hi everyone, author here. This article tends to raise a lot of questions. I'm happy to answer them.
(The most common question: isn't it the same as a mock? The answer is no, mocks are used for isolated, interaction-based testing, and this is for sociable, state-based testing. These are polar opposite approaches to testing with distinct tradeoffs. If you're not familiar with those terms, they're defined in the article in the "Foundational Patterns" section.)
This seems close to what I do, except I dependency inject the alternate implementations rather than modifying production ones to do the job, which seems weird. Plus I may want more than one implementation.
At least at the scale of code I've been working with, it has been excellent. People keep threatening me with dire promises of how this or that testing practice will result in fragile, highly-coupled tests, and all I can say in response is that in 10+ years of continuously writing tests you'd think it would have happened to me by now. Instead I tend to make very reasonable changes to my tests as I go, and the only times I've ever ended up rewriting them top to bottom is when the underlying code under test changed so much that it was inevitable anyhow.
I tend to have a test module that sets up an entire test environment with all the stub implementations, which works fairly well in that middle ground between full integration and unit tests.
My introduction to mocking was through C++ and googlemock, where you have to dependency inject the mock object to do any mocking. This always made sense to me, and could often make APIs nicer and more generic with this approach. The API for dependency injection just ends up an other public interface that you need to test, and one natural way to test it is with mock objects.
Then I learned about pytest, magicmock, and that the norm for other testing frameworks in other (mostly dynamic) languages is that you can replace whatever member or free function with a mock without this being part of the public interface of the original class or function. This feels insane to me, as this just tests implementation details.
Is there terminology to distinguish between these two kinds of mocking?
Alright, so you get numbers from a json file in a ftp servers, you use scipy.newton to perform root finding on it, but needs the number of steps to be outputted as well (which newton() doesn't return), and you will save the result in a postgres table.
All this should be send to a task queue, triggered by a call to your REST API.
"Railway Oriented Programming"[1] is a (functional) technique to achieve purity despite steps that could result in error states. The error states are hidden/pushed to the very edges/end of your code. So you can focus on the "happy state" for the main business logic:
> Many examples in functional programming assume that you are always on the “happy path”. But to create a robust real world application you must deal with validation, logging, network and service errors, and other annoyances.
> So, how do you handle all this in a clean functional way?
> This talk will provide a brief introduction to this topic, using a fun and easy-to-understand railway analogy.
Most of your counter example needs integration tests to test properly anyhow - not really a good point.
If you can give up encapsulation (which IMO isn't actually that useful in practice) and in return, your program becomes 95% functional, that seems like a really good tradeoff.
What's your definition of "pure function". Apparently not "pure function" in the sense of "pure functional programming", because if that were so, your claim would just be wrong. Counterexample: "printToConsole: IO[null]": is a pure function but still not easy to test.
The definition of "pure function" I'm familiar with from functional programming has no side effects including I/O. Assuming 'printToConsole' is performing I/O, you're describing a impure function.
Then you have not understand what (pure) functional programming is and how it works.
I recommend you to have a look at Haskell, probably the most famous language for enforcing only pure functions in your program. And I took this example from Haskell.
While there is an important sense in which even IO values in Haskell are still pure values, it is not helpful in the case of testing.
$ ghci
GHCi, version 8.8.4: https://www.haskell.org/ghc/ :? for help
Prelude> putStrLn "hello" == putStrLn "hello"
<interactive>:1:1: error:
• No instance for (Eq (IO ())) arising from a use of ‘==’
• In the expression: putStrLn "hello" == putStrLn "hello"
In an equation for ‘it’: it = putStrLn "hello" == putStrLn "hello"
If you want to test your IO code, and you should, you still have to test them from the perspective of them actually executing in the real world. There's no way to test them as pure values. The snippet above is only the beginning of your problems if you wanted to do that, but it a plenty sufficient problem on its own. In addition to the obvious problem that it points out that IO values are already not comparable to each other in the naive sense, there's also the more philosophical problem that the IO values are your programs, so even if you could run the test I show above, on more complicated expressions, your tests would be asserting that you wrote the program you thought you wrote, not that the program you wrote does what you think it should do.
Obviously he's keeping the interesting bit about how this works for the actual course, but I already only mock bits right on the edge, e.g. the point where the system would connect to the outside world, and that seems to work.
What I'd really like, is some way of acknowledging redundancy - lots of tests ultimately go through the same code underneath, it would be good if that could be tested only once, and then the data replayed from those points but done in such a way that everything is still nice and readable.
> Obviously he's keeping the interesting bit about how this works for the actual course
No.
Absolutely everything in the course is in the article. The benefit of the course is that you get to interact with me directly. You can ask for clarification, conduct exercises involving real-world code, and bring your own code for my feedback. It's a guided tour, but you don't need it. Everything is in the article.
I get that in other software stacks you might want to avoid mocks and stubs due to friction, but in JavaScript what's the big deal? Mocks are easy– just do it.
Mocks, especially spies, couple your tests to the implementation of the system under test. This leads to a very common code smell in tests: Fragile Test. The problem is that instead of being concerned only with behaviour, the test are knowing something about the implementation details of the SUT (dependencies are an implementation detail that consumers ought to be unaware of).
One of the major benefits of automated tests is that they can provide a safety net that allows you to refactor with less risk of breaking behaviour. If tests get in your way of refactoring, because changing the implementation details of your SUT cause your tests to start failing, then the tests have failed at being a tool to facilitate refactoring.
Mocks and stubs provide lower test fidelity than the real thing. Also makes your tests more brittle as you need to update them whenever the real implementation changes.
Seems like a lot of work just to avoid mocks. Don’t know if this is language specific but I find it’s really easy to create mocks and/or in-memory substitutions in .NET using something like Moq.
A "fake" (as opposed to a mock) is a pretty well understood concept.
Is there a difference between a fake and a "nullable"? As far as I can tell this is a pattern for merging your fakes and reals which seems cumbersome and error prone
> The factory should create a “Nulled” instance that disables all external communication, but behaves normally in every other respect....
>...For example, calling LoginClient.createNull().getUserInfo(...) should return a default response without actually talking to the third-party login service.
They use the word mock several times to explain stubbing. I think they mean stubbing is a type of mocking not an independent approach. They even use a mocking library to create stubs in the Google article.
"Mock" can be both a noun and a verb. As a noun, it's an alternative to stubs, fakes, and other such things. As a verb, it refers to the "monkey patching" method of putting those into place, in a way that undoing it is automatic and won't leak into other tests. The title seems to be referring to this second version, showing us ways to avoid patching with things like dependency injection.
Indeed, like people stating something is a deferred or a future instead of promise, and then pointing out a minor difference in their approach that justifies a whole new taxonomy.
Yeah no, that’s a mock. At least, that’s been a mock ever since I started prgramming 15 years ago. Only later did libraries that automatically mock dependencies appear (and I presume people suddenly felt a need for a new name for not-automatically mocked dependencies?
I think we call those "fakes", and reserve mocks for those tests that count the number of times a function was called, yielding "change detector tests".
Personally, I find it best to not be too concerned about the terminology, and make sure your test doubles get the data that the code under test needs to the place that needs it. The more real code you can involve in integration tests, the better; only look at replacing real code with test doubles if there is a performance or reliability downside. Databases have never hit that performance/reliability downside for me; network services do.
> Personally, I find it best to not be too concerned about the terminology
I second this. My experience is that some people know this terminology if they're actively learning or teaching the techniques, but most of the people I've worked with who know the techniques and apply them successfully don't know the terminology and use the word "mock" universally, so it usually creates more confusion than clarity to try to distinguish the different kinds of test doubles by name.
No, fakes are specifically designed to emulate the underlying dependency somehow so that you can assert on it’s state later. Stubs don’t try to do that at all and are mostly for returning a default response.
The nulled instance continues to be available in production, which has different pros and cons than a conventional mock class which is only available when running tests. Seems worth a different name so we can compare and contrast the two.
I’m not sure I manage to follow the author correctly, but to me it seems that by “mock” they strictly refer to external mocking libraries, in the sense of JS’s `testdouble` [1] or Java’s `Mockito` [2]. The static fake object as returned by `LoginClient.createNull().getUserInfo(...)` appears to be a “Nullable” for them.
I think the issue is "a mock" is a generalized term that implies different things depending on the usage. More junior developers think that it only means when you have an external entity that you have control over that "mocks" the external service that you don't (shameless plug, I built https://mockadillo.com for this). But yeah, these are obviously mocks and the article kind of doesn't make sense.
This is not really by what the article is about IMHO. It’s a great overview of how to grow and organize your application with some opinionated patterns that give some favorable properties to your system. The nullable stuff just helps achieve those properties. If you want to use a mocking library to write nullables, and call them mocks, fine. Not important.