Hacker News new | past | comments | ask | show | jobs | submit login

Additionally, but on a slightly different note, I can think of two other common situations where these equivalence validation techniques are useful.

When refactoring or doing performance work, its very helpful to create two variant implementations, for instance the simple, obviously correct version and the new, optimized version. One can then send the output from a property based tester or a fuzzer into both implementations and test: assert forall x . referenceVersion(x) == optimizedVersion(x). A good property based tester or fuzzer will give one high confidence that behavior has been both understood and reproduced. Since changes to legacy code and bug fixes are among the primary causes of defect introduction, these testing techniques usually quickly bring to ones attention how fallible one is.

For non code artifacts like configurations and build systems where output is assembled or generated. The Unix diff utility can be useful. One assembles or generates the artifacts in two directories. One directory is representative of the project before the change and the second directory after the change. Anything that shows up in the directory diff should correspond to exactly to what one expected to see as a change. Since the full actions of package assembly and generation are often opaque, this technique provides assurance in parts of the system development process where there are often few other safety nets.




> One can then send the output from a property based tester or a fuzzer into both implementations and test: assert forall x . referenceVersion(x) == optimizedVersion(x).

You can even do that in a production environment, assuming the "optimised version" is properly protected so it can't take down the entire system if there's an error in it.


Additionally, for performance work you can put your performance expectations in a test: run both the old and new version, assert the perf. difference you want in addition to the result being the same.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: