Don't discard good anecdotal evidence because there's no proper randomly controlled trial. If you're experienced/senior in the industry you probably worked with dozens of developers closely, with hundreds less closely, you've seen many projects and designs, you've interviewed and was involved in the hiring of dozens of people.
This random controlled trial you're talking about is so hard to do because there are infinite confounders. How do you even measure success? After how long? How do you separate politics from technical ability?
In reality people above a certain bar are able to make some contribution to most projects. You can't and don't need to staff every project with superstars. The superstars are rare but are not hard to identify. They'll have the track record, reputation and the skills.
Another thing to consider is that the best outcome of interviews depends on the quality of the pipeline into those interviews. If you can't get good employees to apply you won't get good employees hired. Pay well and build a company that people want to work for, and then your hiring is going to be easier.
I guess we could make some reasonable evidence about hiring approaches based on HN hirers: What percentage of hires using your favourite technique/interview question do you think worked out as a success?
My rate is around 50% good, 50% I wish I hadn't hired.
I've had a much better rate than that. What does "wish you hadn't hired" mean exactly. Did you completely misjudge them or did you have an idea of what you're getting and still hired them. To be way off on 50% of interviews feels like way too high failure rate.
This random controlled trial you're talking about is so hard to do because there are infinite confounders. How do you even measure success? After how long? How do you separate politics from technical ability?
In reality people above a certain bar are able to make some contribution to most projects. You can't and don't need to staff every project with superstars. The superstars are rare but are not hard to identify. They'll have the track record, reputation and the skills.
Another thing to consider is that the best outcome of interviews depends on the quality of the pipeline into those interviews. If you can't get good employees to apply you won't get good employees hired. Pay well and build a company that people want to work for, and then your hiring is going to be easier.