Bar-raiser program didn't have a feedback loop to track its effectiveness. Bar raisers periodically met to discuss topics related to hiring and increasing interview bandwidth but we didn't evaluate the process as such.
That said, the program (like any such program I guess) didn't scale well with the immense pressure on increasing the head-count which in itself was a result of delivering more. Hiring managers and recruiters (who are incentivised to hire more) would rig up interview loops in a way to get their candidates through.
The program itself came under increasing scrutiny (I guess for the right reasons) as being too restrictive. I distinctly remember period between 2008-2012 they hired at a blistering pace, all over the world. In fact in Seattle when they moved into SLU campus the buildings went from empty to nearly 100% occupancy in a matter of year or so. You could even see it based on the number of product launches from 2012 onwards (AWS, the hardware products, new country launches etc.,).
And now Amazon is such a behemoth that it doesn't even make sense to speak about it or analyse it as a single entity anymore. My prediction is in 5-8 years it'll be broken up into at least three companies under Amazon as a holding corporation --- cloud, retail, and consumer devices.
My judgment of Amazon is tainted badly by a bad apple we couldn't fire soon enough - I had the unsavory role to collect evidence for failing the PIP that would inevitably happen. Week after person was let go, he was at Amazon in a pretty good division.
I keep wondering how they hired this person - never in a million years would they have passed a regular hiring chain even with fizzbuzz type questions. Wonder if they had someone stand in for the interviews.
It's often been mentioned that interviewing skills is different from actual work skills. Could be that he's an interviewing wiz but a poor day to day engineer?
Netflix is well known for firing people that don't pull their weight with far more ease than most companies. But a friend who works there tells me there's still people there that perform poorly but somehow managed to get both hired and stay on.
Companies tend to keep track of the onsite success to failure ratio and then if it's too low examine if the bar raising is too aggressive.
Which ends up going in the direction of meetings like "Should we standardize on what questions we are asking in interviews?" or "Should we standardize what pass means or fail means for a particular problem?"
