I don't really understand the obsession around snapshot testing. Seems like a lazy way of doing testing that doesn't add any confidence that you're doing the right thing. It just adds confidence that the initial value you got the snapshot from, is the same going forward, which is not super valuable.
Snapshot testing is basically what used to be referred to "master-knows testing" (don't remember if this is the exact term) where only the one who initially created the test knows if it's correct, because the intent is not exposed at all. Instead, a properly written unit tests exposes what you want that unit to do, and focuses on only that part, so you can change things in the unit without breaking those specific parts. Snapshot testing would ruin this.
> You want to add tests, but you don’t have time to!
This also seems like a weird thing. The idea behind testing is to save you time. If you're taking longer time because you're adding tests, you're doing something wrong! The whole idea is to make the feedback if something is working or not faster, not slower.
Snapshot testing is useful for catching regressions and nothing else. It’s better than nothing, and in some cases is the only thing you can do if you don’t properly understand what all the inputs and outputs of a system mean.
For example if working with a large legacy codebase, it’s important to maintain previous functionality and only change it if you can rationalize why. People and other systems might be relying on something that’s a bug, or a side effect that’s not obvious.
If you don’t do this, then you can either choose to have no tests, or have to tear down the whole system to understand it.
>Snapshot testing is useful for catching regressions and nothing else.
Visualizing and eyeballing snapshots in behavioral tests is a highly effective way of catching bugs and defining behavior (edit code - run test - verify output is correct, if it is, lock it down).
I find them to be vastly more effective and cheaper to build than unit tests even on greenfield code bases.
The painful part is "verify output is correct" which only the person who initially did the test, can actually say it's correct or not.
If a new person joins the codebase and sees that a snapshot is now different, how they know what's correct or not? What I've seen in the wild is 1) talk with the person who authored the test or 2) just say yes and move on.
I mean you should have the context of your change right. If you're submitting a change to reduce the margins on everything, you should expect to need a new snapshot.
If you're are doing a pure refactor of your css, you shouldn't see a change. Unless your css rules are order dependent and now you caught that.
This also seems like a weird thing. The idea behind testing is to save you time. If you're taking longer time because you're adding tests, you're doing something wrong! The whole idea is to make the feedback if something is working or not faster, not slower.
This is true when you're writing the code initially, but the fact is some people don't write tests, and then you might inherit a huge pile of untested code. Adding tests to would take a lot of time, and sometimes you need to start making changes fast.
When there are no tests it's very likely you won't be able to add tests without changing the code to be something that you can test. This is where snapshot testing shines. Snapshot tests are the only type of test you can add without needing to modify any of the code. You can add it in easily, and it gives you some confidence that you're not breaking things.
> When there are no tests it's very likely you won't be able to add tests without changing the code to be something that you can test
Indeed, it will most likely require refactoring the code to support the testing. But the initial idea of _why_ you want to add testing there in the first place, is so you can refactor with confidence and not having to manually test all the cases. Compare "refactor to test -> add tests -> refactor behaviour" with "refactor behaviour -> manual tests" and the first one will most likely be faster.
A few years ago our team rewrote a critical part of our system in a more resource efficient language to cut a large chunk out of our AWS bill.
The system handled around 500K requests per minute. The inputs/outputs of the system looked like this.
1. Receive HTTP request
2. Make a bunch of HTTP requests
- Call external web services for business logic
- Call reporting services for analytics
3. Render result
On a single production box of the legacy system, we instrumented the http clients to dump the requests/replies to disk for 15 minutes.
Using the snapshots, we wrote some scripts to allow us to replay the data through the new system. It was a totally ad hoc manual process that took about a week. It found a TON of bugs and greatly increased our confidence in the new system. After we were done we threw it away.
I am convinced that using standard unit testing alone would not have found many of these bugs and caused them to surface in production.
The headline of the article should be a hint. "3 steps to add tests on existing code when you have short deadlines"
So, if you're on a short deadline and maybe you didn't write the code the first place, so you might not be familiar with the internals, snapshot testing is the quickest way to get confidence that you aren't causing further regressions.
These aren't meant to be used forever more. Just to stop things from sliding backwards.
After all, the title of the site is understandlegacycode.com. It's all about getting to a point where you can refactor and do things properly when you do get time.
If that doesn't make sense, what would you suggest to do?
I made a similar objection the last time this came up (might have been on another site.) Someone pointed out that they can still be useful for confirming that refactoring for readability/testability has not changed behavior, at least for the paths the snapshots exercise. That could be valuable when dealing with a hairy code base. Still, you wouldn't want to keep such tests around for long.
Snapshot testing is basically what used to be referred to "master-knows testing" (don't remember if this is the exact term) where only the one who initially created the test knows if it's correct, because the intent is not exposed at all. Instead, a properly written unit tests exposes what you want that unit to do, and focuses on only that part, so you can change things in the unit without breaking those specific parts. Snapshot testing would ruin this.
> You want to add tests, but you don’t have time to!
This also seems like a weird thing. The idea behind testing is to save you time. If you're taking longer time because you're adding tests, you're doing something wrong! The whole idea is to make the feedback if something is working or not faster, not slower.