I think hiring is broken for a different reason: workers, whether in software or any other industry, are viewed as less-than-equal ("second-class citizens"). Except in rare cases the entire hiring process in software especially, from start to finish, just about everywhere I've ever interviewed, seems to me designed to find reasons to reject candidates, and as a side-effect is (perhaps unintentionally) also designed to demean and diminish human beings.
I agree. But why are they so choosy, and why do they complain about a lack of qualified candidates when their rejection rates are so high? I think it's the zero-sum game I mentioned. Firms are trying to outhire each other, like talent is some magic wand that can replace compelling products. There's a lot of snake oil in the industry, like SEO (now called "data science"), meant to substitute for things like vision, leadership, and customer service.
firms are trying to outhire each other, like talent is some magic wand that can replace compelling products.
Where do you think the compelling products come from?
There are plenty of companies that take the view that you seem to be advocating - that there are employees who "create" and other employees who just turn the wheels and implement those ideas. Your average fortune 500 company is just like that. The engineers working there pretty consistently complain that the software dev team isn't respected, and isn't sees as producing things of value, or providing a competitive advantage.
Those companies believe that thevakue isn't in the oroduct/idea and the execution is easy and ought to be cheap. Which is fine, as long as you never need to do anything that requires significant technical skills. Because such environments force out anyone who thinks that they gave more to offer than simply churning out cookie-cutter, mediocre code.
Their are ways to build successful teams that don't require that everyone is top 10%, but the notion that you just need compelling product ideas and a bunch of average developers to implement them is pretty common place, and rarely produces exceptional products.
I don't know if it's exactly a zero-sum game, but the rate at which talent is created doesn't seem to be amazingly high given that there are over 7 billion people on Earth. Certainly the talent pool seems to be a small fraction of the population.
Top universities, which for the most part feed the tech industry, face a similar problem at an earlier stage in the pipeline. It's been noted that less prestigious universities tend to draw their faculty from a very narrow selection of elite universities.
The education and labor markets are linked, and inefficient in ways which perhaps reflect the same underlying cause: scarcity of people with apparent ability, and self-interest wanting to profit from oligopolistic control of this incalculably valuable limited resource.
I think part of the problem is that hiring bars are unrealistically high. Pulling numbers out of my ass for illustration, it's like companies that control 10% of the software engineering jobs all only want to hire the top 5% of software engineers. They can't all get their way, so the top 5%[1] bounces between these companies whilst the companies whine about "talent shortages".
[1] Should be noted that it isn't the entire group, since some will choose to work for organizations outside this group because the work is more interesting or fulfilling.
I don't know that there is a "right" level for the bar. There might be a game theoretic way of looking at the problem which explains how we wind up with the levels that we do.
In relative terms, the bar for, say, the NBA is much higher than for top universities and tech companies. They set the bar high enough so that, for all practical purposes, minor league teams cannot come close to matching their combined level of play. Also, if you don't compete regularly against NBA caliber players, with NBA quality coaches and trainers, it would be hard to reach the same level of skill even with comparable native ability. I don't think the league is that concerned with missing out on marginal players, as long as on balance it gets those correct. The disaster would be to miss out on the Michael Jordans or Lebrons, Bird and Magic. So they scout and pay through the nose even for the slim chance of that kind of greatness, with the tolerable outcome being a good role player.
At some level, we're all looking for greatness. Nobody wants to see a league entirely made up of role players doing somewhat above average things.
The analogy breaks down in that it's a lot easier to accurately measure (though perhaps not predict) athletic ability. I do think that investors in tech tend to heavily favor obviously stellar performers over "good but not great" ones.
That is because people do not nurture and train and help their talent enough, they want them to turn up exactly like some mental model they have for a mental model of a role they have. There are huge numbers of people once you stop trying to find exactly the thing in your head.
>just about everywhere I've ever interviewed, seems to me designed to find reasons to reject candidates
I'm sure that's intentional. Joel Spolsky wrote an influential guide to interviewing in 2000 that lays out this philosophy; the latest version (updated in 2006) is here:
The gist of a large part of the article is that interviewers should have a preference to saying "no hire", because bad hires are toxic and hard to get rid of.
But then he writes this:
----
Of course, it’s important to seek out good candidates. But once you’re actually interviewing someone, pretend that you’ve got 900 more people lined up outside the door. Don’t lower your standards no matter how hard it seems to find those great candidates.
----
Which is terrible advice if your standards are, in fact, unreasonable.
It's just a toxic environment.