Hacker News new | past | comments | ask | show | jobs | submit login

Although I wouldn't argue against unit testing(I use it too), I'm a little more hesitant about incentivizing test quantity. Many times when this comes up having been tried it gets attached to a anecdote that goes: "And so we wrote very bad tests and atomized the codebase into unmaintainable pasta code that could be trivially tested to make the number go up."

Which leads me to a hypothesis that there's some other quantifiable out there that would work better than unit test coverage: a "diversity test" metric. The software artifacts produced by writing code and after a build are not "the software" in the sense of the whole project. Documentation, support, and so on are also "the software." Logging and profiling tools enable debugging methods other than "stare at the code very intensely". If you start from a holistic standpoint and assume quality requires multiple angles of inspection then your choices of technique broaden substantially.

If you changed a mandate of "must test all code" to a mandate of "must either test, log, profile, or document all code" you would add leeway for the corners that are test-averse to be documented or logged instead, and vice-versa. The combination gives you a palette of feedback: if it's hard to do any of those things, you have bad code. Ideally you can get all of it, but that is unlikely to be the norm for most organizations most of the time. But more bugs will be caught by this web of feedback than if you mandate any one of those methods alone.




I agree with this, and I think software quality is like security. Multiple layers are better, and orthogonal layers are even better still.

A good logging system is a great example. To be effective, your logging should be going into something like Kibana or Grafana or both. Logging levels should be correct. Errors should be easily visible and have useful stack traces; all other noise should be easily filtered out. Alerting should be in place for any crashes, ideally going directly into a bug tracking system. (Sentry.io has a nice story for this on the front end).

This ties in well with the "let it crash" philosophy. You just cannot anticipate all the crazy inputs that will go into the typical software system of today. You can't. Most systems I work with have multiple external dependencies, each with their own schemas, data workflows, deployment processes and so on. Unit testing won't be as valuable with that stuff. Full end-to-end testing will be more useful.

Look at all your options as a whole and allocate developer resources according to the value of each.

And personally, I don't think the OkCupid model is as bad as it sounds... ;)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: