>Now, suppose that the product manager comes to us one day and says, "We have a new design, we'd now like it to say "Greetings Brandon, welcome to our website!"
I have been writing YAML based tests which rewrite themselves based upon program output in these cases.
Change the code -> run one script and now the test is updated. There is near zero test maintenance cost for surface level changes like this message or a tweaked JSON output from an API.
I call it snapshot test driven development.
I agree with the overall sentiment - testing ought to considered an investment that provides a recognized ROI.
There's a similar concept of "snapshots" in UI testing, which I maybe should have included in the post
I think snapshots definitely lower (but don't eliminate) the cost of a certain class of tests, and can sometimes tip the scales from "not worth it" to "worth it"
It's almost the same thing, I think. I do the same thing with text artefacts and screenshots.
There is a hidden cost to snapshot testing - theyre highly susceptible to flakiness. The tests have to be hermetic and the all the code has to be deterministic or it doesnt work.
I have been writing YAML based tests which rewrite themselves based upon program output in these cases.
Change the code -> run one script and now the test is updated. There is near zero test maintenance cost for surface level changes like this message or a tweaked JSON output from an API.
I call it snapshot test driven development.
I agree with the overall sentiment - testing ought to considered an investment that provides a recognized ROI.