I've found in my experience that (many, not all) self-proclaimed "experts" do tend to grossly overestimate the capabilities and competence, and will impede those who are more capable than them for fear that they will diminish the value of their "expert" status. I feel this article is correct in stating that the appropriate way to approach a problem is from the perspective of an amateur, as amateurs typically have little fear of abiding by rules that have congealed within the minds of the "experts", and are more willing to take the chance of being wrong when success is imminent.
As they say, "an expert is someone who has made the greatest possible number of mistakes in their field."
I remember reading about a study where people who are below average at a task tended to rate themselves as competent (or better than average) for that task, while more competent people wold under-rate themselves.
There are all sorts of possible explanations (don't know enough to know you suck, pride, etc) and I don't think the study really looked at causality at all, but yes, I think the phenomenon is well known.
The study you're thinking of is probably "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own
Incompetence Lead to Inflated Self-Assessments" - it's completely fascinating and slightly worrying.
I'd say that's true of some, perhaps a few self-proclaimed experts. Most people who claim to know something in a field actually do have useful knowledge to impart.
Sure, there are quack doctors and flim-flam car repair men and consultants full of bafflegab. But most doctors really do know something about medicine, most mechanics know how to fix cars, and most consultants have highly useful experience to share.
That's my experience with experts. Maybe I'm not smart enough to know how little experts really know?
Anyone can talk about software, some people can even give useful advice, but there are patterns which define a true expert:
Experts know how to find a useful solution in known time. This does not mean they can instantly come up with the best solution just that they already know a solution and how long that solution will take.
Most experts are willing to try new things up until the point where they will not get it down in time and then fall back to the known solution.
It might not seem like much but it's amazing how much talent and experience it takes to get to that point. What messes with most people is training does not create experts you need to do it without a safety net to really know. And experts don't necessarily know how to teach they can just get things done.
This anti-expert bias seems strangely familiar. Almost as if... The last eight years... Were a bad dream...
He goes on to say he really has an anti-self-proclaimed-expert bias. A safe target, nobody likes a smug know-it-all. But sometimes the self-proclaimed expert is an expert. How do we know if we admit to being amateurs? We don't know, of course, but the odds are we should listen to people who demonstrate expertise and not dismiss them because they come off as smug or claim to have expertise.
Imagine, for a moment, Jeff was talking about Linus Torvalds. Or perhaps Paul Graham. How would you take this post?
This comes off as being worthy of the last administration: Pandering to people's prejudices and distrust of anyone who comes across as smart.
p.s. Does anybody else think this is just a continuation of his bashing people like Kent Beck and Robert Martin for daring to try to systematize agile software development with anything m,ore complex than Jeff's four rules?
Continuation because noprocrast locked me out awhile: As pointed out in Jeff's article, McQueen's character knows what questions to ask. And per The Unthinkable [ISBN 0307352897], Rescorla knew what mistakes people were likely to make. An expert can anticipate problems and raise good questions.
in my organization, anyone who advertises their own credentials, especially a PhD, had better be very very good or that person will be ridiculed mercilessly.
As they say, "an expert is someone who has made the greatest possible number of mistakes in their field."