We talk about something like this now and then but the thing is, if fizzbuzz is still managing to eliminate more than half the candidates I don't think it's useless.
I’ve been having trouble finding the scientific papers behind this theory, but one I’ve latched onto is that “filtering against known strong signals is helpful” but also that “additional filtering beyond that is more harmful than random choice of the remaining pool”.
Basically, it can probably be shown that if you hire a group of completely random sample of resumes vs. hiring a random sample of people who can produce a working fizz-buzz program, that the candidates from the latter group will perform better. But if you then filter the fizz-buzz group by “can they solve this negative base math problem?” you’ll be removing a disproportionate amount of candidates who would have turned out to be excellent for your organization, and your final “super-candidate” pool will actually be weaker (for your org / that position) than the average of all those who could solve fizz-buzz.
I think the research shows that you should filter based on what you know for sure provides a true signal, then select randomly from the pool which passed your known filters. If anyone can help me find anything relevant about this, I’d appreciate it.
Too many orgs treat hiring like the “secretary problem” but that requires the assumption that you can grade everyone accurately on a continuous scale. We can’t yet do that with software engineers - theres no way to say someone is “80% awesome” vs. “93% awesome”.
I don't agree. One part of coding tests is getting the right output, the other part is how they went about it - which is relevent even if they didn't solve the problem. You can tell a lot from the second part.
I'm not a huge fan of coding tests, and I have a lot of sympathy for those who refuse to do them, but it's a case of "It's not you, it's everyone else"
I believe it to be much better to see a candidate write some code quickly, have them write unit tests for it, discover that some cases were not handled properly and then they were.
This is how 99% of developers work, I don't believe anyone who says they write their code perfectly and under stress every single time.
That's my impression. Anything more complicated and you're starting to filter based on your appreciation of the candidate's choices, style, etc. I.e. things that can be worked on or fitted to the company's work style.
