Forgive potential ignorance here, but people often quote Joel Spolsky as saying things like: "...the trouble with using a lot of mediocre programmers instead of a couple of good ones is that no matter how long they strive, they will still produce something mediocre."
What really genius-level things has Fog Creek done? I remember demoing FogBugz, and it was marginally better than its competitors, but it was far from a complete game-changer. Trello is ok, but it's only marginally better than a whiteboard with post-it notes. None of these reinvent an industry, or provide something so outstanding that people rush toward it.
Don't get me wrong; I think Joel has a lot of good ideas in general, and Stack Exchange is indeed a game-changer. But as someone who continually talks about hiring "rock star programmers", I simply don't see a result that I'd consider "rock star" equivalent. But, maybe I'm missing something?
Edit: More than that, I don't even consider myself a rock star developer, just a normal developer who loves learning new things and getting a lot of stuff done. I'd be thrilled to work for a Google or Microsoft, but what would attract "rock star developers" to Fog Creek? Basically, are fancy chairs enough? Isn't the most attractive thing a super interesting problem domain?
I'm constantly fighting that same urge. The problem is one of domain. UI coding for straightforward data models (which is 90% of all "web" development) just ... isn't very hard or impressive. Great programmers in that regime produce software that is smaller, cleaner, tighter and higher quality. But it's still just a bug database or social thing or web store, etc... Anyone (even mediocre programmers) can look at it and say "I know how that works."
There's plenty of rock star coding out there, of course. But it tends to be either hidden inside companies due to secrecy concerns (Google's scaling architecture or the NVIDIA driver stack, say) or exposed in the open source community instead of the startup world (Fabrice Bellard is a great example here).
Startups all say they want rock stars, but then they put them to work doing "better versions" of the same boring stuff we all recognize. Which, if you think about it, is sort of what startups are supposed to do.
Although the article never used the word "super star", it's something I see around HN and the startup world quite a bit....
I sort of snicker when I see startups post jobs looking for "rock star" coders [1]. What self respecting person would call him or herself a "rock star", let alone apply for such a position? Maybe it's just a gimmick to get mediocre developers to get to work on boilerplate web work..
[1] I also saw a job posting recently where they were looking for a coder "to work on hard problems and blow a hole through the universe"... Why not have them blow bullshit of their face while they're at it? Is this just HR trying to sound too cool, or what?
It's just a meme. Some people are into it, some aren't. There are lots of ways to motivate people, and this works for some. I try not to get to wound up over language usage.
I'm just guessing here, but I bet its more about communicating a shared perspective on software development than people thinking they're actually "rock stars". When a job ad says "looking for rock stars to blow a hole in the universe", you would be a fool to take that literally. But its along the lines of a "secret handshake" that lets both parties know something about the other in as few words as possible. A job posting like this is basically saying "hey, we read HN, reddit, etc, we know there's a ton of crap jobs out there full of walking-dead programmers. But this isn't one of them". Of course, the whole "rock star" meme has been hammered to death so that you're more likely to be a company pretending to be something you're not. The point is there is validity in this method of job posting.
It's nonsense.
I recoil every time I see a job description like that. Anyone who believes in those mythical rock star programmers has a serious problem and I certainly don't want to be part of it.
When a job listing says ninja/rockstar/badass coder, its really just a codeword for someone they can force to do anything at all costs. They act like they want a 10x'er but really just want someone who they can get to do 10x much work.
The term "rock star programmer" is by now a symbol that people intuitively understand the meaning of. Essentially equivalent to the "creme de la creme" of programmers, but if we switched to using that term few would balk at the comparison to dairy production. I've described support staff coworkers as "total rock stars" before. Everybody gets what I mean when I say that.
So it's hilarious when companies like Geico or whatever post ads on Stack Overflow Careers 2.0 saying they only accept "rock stars"... Because we all know that's where all the hot shots flock to...
Rock stars generally don't get a salary, but royalties or profit sharing. The better the star/band does, the more money they make right now at the gig, and longer term in increased sales over time.
If someone wanted me as a 'rock star' developer, I'd want some serious profit/revenue sharing plan in place - something more than "10% of profits are held back and divvied up amongst 90 other people based on seniority".
Not sure about that—a lot of famous guitarists were originally pickers, and while the electric guitar may be a sexier instrument, most of the drummers I’ve known have actually been very attractive.
The majority of amazing software that has not been created yet is "CRUD". Why developers keep thinking the low level stuff is the good stuff.
A huge class of such systems that are waiting to be created are crowd labor systems. How do you create leaderless organizations that produce what google or boeing or walmart produces and that compensates individuals for performance.
The answer today is a messy unorganized market system. Proper information systems that better organize that would create a lot of value. The analogy would be this site. Thousands of people communicating, without the organizing software it would be much worse (thousands of people standing around talking to each other, the good ideas spoken would not easily reach the top as it does in this organized software system).
tldr; the biggest value opportunities lie in CRUD apps, not the low level code
The answer, of course, is that the "good stuff" is subjective (and almost never aligned with "the biggest value opportunities", ick).
Yes, CRUD pays. Being able to make more CRUD in less time (and/or CRUD that is better performing or more maintainable, etc...) is a valuable skill.
But it's boring. And more, it's just not impressive in the same way that, say, qemu/kvm is (or Mesa, or Linux, or llvm, etc...) That stuff is hard, and fun, and interesting in a way that CRUD will never be. So it has value too, and IMHO it's entirely reasonable to celebrate its practitioners even if they don't make as much money as CRUD jockeys like Zuckerberg do.
I agree, the classic engineers like to work problems with an easily definable end result (so that business types don't tell them what to write).
It's a pity though, there's a huge space of products that are needed but the talent is going into slugging it out in societally useless and relatively low paying niches (such as many of the most talented programmers entering the game industry or social networking or linux kernels). Meanwhile dumbasses are raking in millions with shit like SAP and the like.
While I agree on your remark about talented programmers in game and linux kernels. I don't necessarily agree with lumping social networking with the two fields aforementioned. I also disagree of your remark about SAP.
I don't necessarily agree with SAP or SAP consultants but there are cases where people would prefer to use SAP to implement some of their core: http://www.infoq.com/presentations/Building-a-Hybrid-Cloud-a.... SAP is a complex, specialized tools that requires specific knowledge on how to tame it.
(I'm not arguing if complexity is good, bad, or anything like that. I'm focusing on what SAP is good at).
You're missing the point that the term rock-star programmer doesn't mean they will produce world changing programs. It means they'll do an order of magnitude better/faster than their non-rockstar peers.
That might be a bit contradicting to what Joel is explaining.
From what I've experienced first hand, there's programmers that can produce similar code/software in a week that takes 5 mediocre programmers to produce in a month. Those are extreme cases though.
There's not enough "rock star programmers" to go around, so not everyone can have a piece of that cake. Much of this is also very domain specific, it's not a given that a "rock star" game programmer is going to be a orders-of-magnitude better at developing business support systems. Finally, this is not that specific to programming. Not everyone's Einstein, Feynman or Michelangelo.
Can we please stop using the term "rock star?" It's getting old.
This idea that a single programmer is somehow so much better or superior may have its merits, but it's harmful to your overall business if you put all your weight on that single support.
You have to accept that above a certain number, your employees will fall on a bell-curve of ability, and you should support the efficiency and productiveness of the entire system, not depend on a few "rock stars" to drag it up with you.
Here are words I associate with Rock Star: driven (sure), but myopic, single-minded, selfish, unbalanced, and unfaithful. I say we strike it from the vocab.
I agree completely with that. If you're betting your future on having all your employees being an order of magnitude better than the average, well - you're gambling high.
Well, in the linked article he used a Mozart/Salieri analogy, implying that rock star programmers write masterpieces, not simple bug trackers. If all Fog Creek programmers were truly rock stars, wouldn't they be beating Google instead of writing menial money-farm stuff? In my experience, those of us who move at a faster pace than our peers tend to get the most interesting projects. There's a reason that ERP software is so bad, after all.
In the rock star analogy, it'd be like getting the Beatles to write music for Justin Bieber fans, or getting Metallica to switch to the "easy listening" genre. That's not what you do with rock stars.
The point is that if John Lennon was borne today he (or if Jean, she..) would make great music. You might never hear it because of "the industry" but John, or Jean, would be out there somewhere doing ingenious things with melodies that no one else would have.
You can't stop people like that, the same is true for programming, if it's your thing you'll do it. If not.. you won't, you'll be a session musician, and every tune you hear (or function you see) you will be able to say "I could of written that" but you didn't.
> What really genius-level things has Fog Creek done?
Made himself independently wealthy. Repeatedly (IE: Not luck).
Not that I disagree with you. But that is worthy of respect in itself. Also, there is an art to the very difficult task of making the difficult look simple and easy.
There have been many people that have won the lottery more than once. Does that mean they are skillful at winning the lottery? Simply succeeding more than once is not sufficient evidence that the success wasn't because of luck.
All success is based on luck to some extent. It's silly to argue otherwise. Of course there is skill involved as well.
To quote The Spolsk himself:
"This paper is the ultimate rebuttal to those grumpy people who email me, barely able to conceal their disgust, saying, 'why do you need to hire such smart people to work on bug tracking software?'"
"Don't get me wrong; I think Joel has a lot of good ideas in general"
The saying that comes to mind is perhaps the following. "Those who can, do, those who can't teach." I'm guessing that Joel, by his writings, is a combination of the two and as a result his output is not "rockstar" quality.
Another example might be Steve Blank. Who no doubt achieved more fame as a teacher then he ever did creating anything.
There is obviously quite a value in people who can do both in varying degrees.
Indeed; and my post was never intended to belittle Joel himself. He has provided great value to the industry in both capacities. I'm mainly challenging the notion that if he truly hires only developers he considers to be rockstars, then why doesn't the company have a more compelling domain than bug tracking and PM?
Choosing a good domain is a challenge in itself, quite separate from executing well once the choice is made. Programmers who don't explicitly tackle that challenge will tend to drift into writing stuff they would use themselves, which obviously means going where the low-hanging fruit has been scoured terribly clean; looked at that way, it is a testimony to Fog Creek's execution skills that they can stay in business at New York living costs at all.
Well that certainly has to do with both business sense as well as the risk taking ability and what he feels comfortable with.
To do something compelling you have to want to take that risk (which could result in spectacular success or failure). Joel might not want to bite off more than he can chew or he simply might not be creative enough to come up with anything "more compelling domain than bug tracking and PM".
That is why a good programmer needs to be around good business people (add: as well as people with ideas and a feel for the marketplace). They are two different skills. Same with medicine. Doctors specialize. You don't find that someone is the best thoracic surgeon and world famous also does urology. Maybe he also dables in photography but he's not going to be as good as Ansel Adams who does it full time. Business is a skill and takes full time attention. So does programming.
Look at PG. Running YC is a full time job. While he can certainly do programming on the side now and maybe write a book you're not going to tell me he can do all three better than a specialist who can devote full attention.
That is why a good programmer needs to be around good business people
Partnering with a business person for my next venture is something I'd like to try, but it's not exactly clear where you find a good business person who wants to partner up. The business people I am in contact with are already busy with their own things.
Unlike a programer who might be presently doing the 9-5 corporate job, it seems like a good business person will always be busy doing their own businesses, as that is what business people do.
I have been mulling over something in particular as of late, but I presently don't have much interest in building it if I'm also going to be responsible for selling it. A partner seems like it would be a good fit.
"where you find a good business person who wants to partner up"
My first question would be to ask you where you are located? Not sure if you want to reveal a specific city or not. Where you are matters.
"The business people I am in contact with are already busy with their own things."
True. Which is why you need to be somewhere where there is a chance of meeting people who are in between, recently graduated etc. There is also the possibility of someone who is employed full time starting to help and give advice on the side. When/if the venture takes off they would feel more secure in leaving their permanent job.
"it seems like a good business person will always be busy doing their own businesses, as that is what business people do."
You only need to find one person (it's like getting married). There are of course compatibility issues as well to consider.
"have much interest in building it if I'm also going to be responsible for selling it."
If you build it and it makes sense from a business perspective and have a prototype you can attract that one or two people you need. With just an idea of course you can't. Care to elaborate on the idea at all?
My first question would be to ask you where you are located?
Rural Canada, which I assume does not bode well. There's certainly no startup scene, so to speak.
Where you are matters.
Which is unfortunate because one thing that has always excited me about software in particular is that it democratizes location. I've had the opportunity to work with some amazing organizations from around the world without having to be physically present for any of them.
If you build it and it makes sense from a business perspective and have a prototype you can attract that one or two people you need. With just an idea of course you can't. Care to elaborate on the idea at all?
That's fair. I've built "hundreds" of prototypes before. Some I've come to realize are simply bad ideas, others I've decided were still good and were taken to market with some mediocre success. In those cases I have felt were limited due to my lack of business acumen. What is not clear to me is where to go even after a prototype has been made and still shows merit.
I don't mind sharing the idea, but after some further research I discovered some companies who are already working on pretty exactly what I had envisioned, so I'll spare you the details. Though I do appreciate your advice in general as there are millions of other problems that need to be solved so it will no doubt come up again.
I think this gets to the heart of your bias. "Bug tracking and PM" is not compelling to you, but I suspect Joel and his hires find it a lot more fascinating than you do.
Not to answer someone else but I think what the parent means is that it's not original, unique, or greatly improving on whatever is already out there. (Is it?)
I mean it's kind of meat and potatoes addressing a problem that has been addressed and maybe doing it better.
Let me state that while I can program I am certainly not a programmer. But from my perspective it seems that Fog Creek, while a great company is not the place that a world class programmer would go to work on world class software. I mean this as no disrespect to Joel (who I admire greatly) or what he has done.
Using an analogy if you want to be in law enforcement are you going to work for the local big city police department or try to work for the secret service or the FBI?
What would you say about people working on search algorithms at Google? Is that prestigious enough for you to call it world class? Because that seems like an addressed problem and maybe doing it only a little better.
> Using an analogy if you want to be in law enforcement are you going to work for the local big city police department or try to work for the secret service or the FBI?
Do tell me what makes the FBI or the Secret Service more "compelling", "original", or "unique" than a "local big city police department". Because I see law enforcement, full stop. Not Swanky Law Enforcement versus Ghetto Law Enforcement.
I think it is always important to consider time-lines on stuff like this. In this field (many fields actually, but it seems especially this one), the first version of something may be revolutionary or particularly good, and then everyone copies that as obvious, making the first one seem only OK or even bad. TV-Tropes covers this effect nicely in the world of fictional television at: http://tvtropes.org/pmwiki/pmwiki.php/Main/SeinfeldIsUnfunny
I'm not sure the timeline on FogBugz, but it wouldn't surprise me if this was a case of it.
FogBugz arrived when the main player on the scene was BugZilla, and capitalized on the fledgling "AJAX" craze. It was definitely a Bugzilla zapper, and it still remains a quality product to this day, gaining features and maintaining simplicity. It's a quality piece of software, but it's still "only a bug tracker". It wasn't the first of its kind, and it's definitely not the only one these days.
When I think of a rock star, I think of someone who not only produces stuff of quality, but actually changes the landscape of the industry. If FogBugs hires 40 rock star developers, I'd expect to see a lot more come out of them than products all in crowded spaces. While they are all quality products, they aren't changing the landscape of the industries involved.
For a lot of developers, the how of making software can be as important as the what of making software. I've worked on casual games, bug trackers, retail signage databases, public outreach platforms, streaming media servers, and lolcat captioners.
And you know what? It's all about the same. It comes down to things like being respected, challenged, engaged, and working with people you like being around.
If Spolsky were to go slumming it and open an office in my town, I would be first in line to apply to work on FogBugz, and I don't care about the fancy chairs.
You can achieve faster reading rates with this sort of stuff by only reading one sentence from the middle of each paragraph, or reading the second and last sentence in the rare case that the middle sentence wasn't enough. The content is of such low density that it takes little more than that to decode the meaning and intent of the article.
I subscribe to the Economist and listen to it during my commute. Many of their articles are verbose, but it's still better, IMO, than many alternatives.
Oh, no... Not that thing again.I don't want to go too much off topic but to quote xkcd "More harm has come from people decrying societal decline, than societal decline ever has."
I like how Carr's own sentiment mirror the sentiment of people who thought books will stop people from remembering things. And he wants us to return to books?
Ugh, no. I suppose you've extrapolated quite a bit from the title. Carr thinks the benefits of the internet are massive, undeniable, and (probably) inevitable.
He's simply pointing out evidence that our tools change the way we think. He makes the same point about clocks, maps, books, etc.
The book has zilch to do with societal decline in the sense you mean it.
Hey, you're exactly right. Most articles have really terrible information density and this is one of them.
For example Paul Graham's essays are often very insightful, but they are frustratingly long. The core idea, which is the only reason why I am reading the essay, can be summarized into 1 - 3 sentences. The rest is just a boring extrapolation and repetition of that.
I stopped reading newspapers years ago when I realized they used a full page article to verbosely repeat the article title. Sadly, 140 characters seems to be the correct length for headline news in a society with no attention span.
It’s interesting that in the research from Peopleware mentioned in the article, the factor of 10 difference was mostly between different organisations, while individuals within an organisation were typically quite close in performance.
This suggests to me that the biggest influences on productivity for the organisation as a whole are probably to do with collaboration, culture and environment rather than extraordinary individual skill.
Maybe the very best programmers aren’t extraordinary just because they can solve the most difficult problems, but also because they are better at presenting their solutions in ways that are more accessible to other developers. That would have the twin benefits of improving productivity in its own right and of implicitly teaching by example so less experienced developers in the organisation would tend to develop the same skills themselves over time.
If you had a seed of programmers on that level and a culture that promoted mentoring, pair programming, code reviews or similar collaborative exercises, it’s not hard to imagine that such an organisation would develop significantly higher collective performance over time than another organisation without those benefits, even if most people joining each organisation were of a similar level of skill and experience to start with.
I find this is typically due to a difference between the level of abstraction people are modeling in their head. A great coder can flip back and forth between several different levels (the 50m lines of OS code, the 100,000 of app code, the 1,000 of local code, etc) and have a reasonable idea of how those parts fit together.
Not a complete understanding, just a basic idea.
This is why mediocre programmers are what they are---they lack this ability and end up "flailing around", looking at somewhat random issues, bouncing back and forth, unable to see the pattern, and so end up spending more time writing code that eventually has to get rewritten.
A great programmer will systematically take a problem apart so that even if they don't solve a problem in one minute, they've at least narrowed down the window of options for the future.
This is also why great programmers are always great debuggers.
Good points. Just to elaborate a bit, the mediocre programmers tend to end up writing five functions that do almost the same thing because they don't have the vision to see how it all fits together and come up with a single, elegant solution. The code may very well work to accomplish the immediate task, but the lack of clarity and good planning makes it rather fragile if modification/extension is ever necessary. They tend to produce a lot of unnecessary lines of code that become a maintenance liability as the project grows.
I think every previous study of this topic has determined that the best programmers are those that can keep more layers of abstractions in their heads (I'll add a citation when/if I find it).
For me, there are days of brilliance and days of complete distraction. I won't call myself a 10xer but I think much of my success has come because I truly love what I do.
EDIT - I couldn't find a reference to the study I was remembering, but Steve McConnell of "Code Complete" fame has written a blog post describing how flawed the studies he's reviewed have been - http://forums.construx.com/blogs/stevemcc/archive/2011/01/09.... There are links to a lot of interesting sources and he clearly isn't arguing against the idea of dramatic differences in programmer ability ... just sayin'.
I find that programming well, as with much in life, is often a matter of balancing different perspectives or competing priorities.
Can I pay attention to the details without losing sight of the big picture?
Can I do a good job technically, but keep within any other project constraints like budgets and timescales?
Can I write code that is good enough to ship tomorrow, but maintainable enough to work on again next year?
My best work tends to get done when I figure out the right balances. If I let one aspect become too dominant and neglect something else as a result, that’s usually when the problems start.
Agree -- also one of the most under-rated skills is someone who keeps up to date on all the latest frameworks / libraries, etc. More often than not, you're building something that solves a unique problem and you do that by stitching together pieces of other people's engineering.
I would argue that this presents several sub-skills:
Being able to do this quickly b/c you don't have to spend the time learning
Being able to do this effectively -- by not painting yourself into a corner b/c you well versed in best practices and anti-patterns with using said 3rd party technology
Being able to do this smartly -- because you are well-versed in a lot of the different options, you can effectively way their pros/cons, and make smart decisions on which 3rd party stuff to use, and when to build vs. buy
I think these skills are more effective (towards your end goal) than simply being able to quickly code some brilliant labyrinth of an algorithm in less SLOC than a 1 or 2-xer. I feel like the emphasis is always put on the latter, and these skills are overlooked.
Combine these with effective communication, and then you are talking about someone who can get things done.
I think you came close to the biggest differentiator: adaptability. Great programmers can dive into a new technology and use their past experience to quickly gain competency in it. If I hand node.js to a great programmer, I expect that he/she will be able to start being productive with it in just a day or two. Lesser programmers struggle to apply their past experience to that new paradigm. It takes months for them to truly get comfortable.
That type of agility is hugely helpful in an organization.
Communication/Collaboration & ability to write elegant code are pretty symbiotic but I'd generally say it's harder to find someone who can elegantly design/code a system than it is to find people to talk about it(Reqs, specs etc).
Smaller teams means more of the communication takes place in the 10x programmer's head rather than in multiple meetings between 1x programmers. A team of 3 10X programmers should be more efficient than 30 1x programmers simply because communication will be more efficient.
Hah, my own anecdotal evidence has been in favor of smaller teams for the communications reason you cite. Probably a bit of a bad analogy but it made me think of 3 Unix processes vs 30. Way harder to manage IPC between 30 processes as opposed to 3.
Here's what McConnell writes on the topic, in chapter 30 of "Making Software." (The rest of this comment comes from McConnell.)
Some people have objected to the "10x" label, making an argument like "We had a super programmer on our team once. He was so obnoxious he alienated the whole team, and overall productivity was actually better without him.
In general, any practical definition of a real 10x programmer has to include the effect the programmer has on the rest of the team. I have known super programmers who were obnoxious. More often, a supposed super programmer with an obnoxious personality was really an average programmer, or worse than average, using an obnoxious personality to hide a lack of performance. The true super programmers I've worked with have generally also been cooperative team players, although of course there are always exceptions."
I can think of one real, actual, verifiable superstar: Steve Yegge. There are more, but I don't know who they are.
Taking his blogs at face value, and assuming that my sample size of 1 is enough, here's what I've got:
"Superstars" get things done because they get things done. They may be ADD in their multitude of projects or in their personal lives, but while they're producing, they produce. They don't fuck around on reddit while they're coding. They might screw around interminably at other times - they might work only a few hours a day - but while they work, they're working, and that's it.
They also get things done because they understand what the hell's going on, at least one or two layers of abstraction below the one they're officially working on. Stevey claims that his understanding stops at transistors - transistors are black boxes, but he gets what has to happen at the layers of abstraction above that, from the machine code to the guts of the interpreter.
Haha Steve Yegge, hilarious! I'd agree he's a good programmer. A guy I knew at Google interviewed him and he was rejected after the hiring committee meeting. He had to try again 6 months later to get in.
I "worked" on Wyvern in high school (in that I played a lot of Wyvern and made a few terribly shitty game areas and ended up being an administrator, or "Wizard" in classical MUD parlance).
You know how Dwarf Fortress, that ASCII game, simulates a whole lot of ridiculous detail? Over-the-top, "the goblin's spear strikes the baby dwarf in the back upper left tooth and it really hurts so he cries for mommy"? Wyvern didn't quite have that, but the API implied most of the functionality for it.
You didn't have inventory slots, you had body parts, which brought with them the ability to wear or wield certain other items.
You could've written Wyvern in a much simpler fashion. A bright high schooler could make a game that had all of Wyvern's functionality at the end of its life. It's the mountain of "do a lot of other shit later" that got in the way.
And that's what he's writing about in that blog post - not that he's perfect and Java sucks, but that he was trying to employ industry best practices and make everything perfect without compromises. It's not very hackerish, and sure, it was stupid. But the guy wasn't doing this for his day job and didn't have defined deadlines. He was a programmer, in other words, not a project manager.
And that's where he fell down. Project management, not programming. And if you read back through his massive wall-of-text blog archives, you can see him mature as a programmer and project management.
(Please note that I am an avowed fanboi, if you didn't catch that already, and as such everything I say must be taken with a grain of salt.)
I'm disappointed to see the 10x programmer meme hit a publication like the Economist. There's no scientific evidence of 10x differences in programmer productivity. To quote Laurent Bossavit [1], who's investigated the studies people like McConnell use:
"How strong is the support conferred to the 10x claim by the best-reputed list of references, for a reader persistent enough to follow the chain of citations back to primary sources?
"Based on our close reading of the “10x files”, we can now answer: quite weak.
"Not a single one of the references is to a replication, in the scientific sense of the term, of the original exploratory experiment.
"The empirical data is in general quite old, most if not all of it predating widespread use of the Internet - which we can safely expect to have wrought major changes in programming practice.
"None of the studies address the question of construct validity, that is, how meaningful it is to speak of an individual programmer’s productivity, and if it is meaningful, whether the experimental measurements line up adequately with that meaning."
Don't get me wrong--I've certainly seen 10x (and greater) variations in productivity. But they've been due to factors other than inherent individual capability. They're things like
* technical debt
* familiarity with the codebase and problem domain
* interruptions, distractions, multi-tasking and other office environment issues
* choice of language - but this is a distant fourth compared to other the other three
On top of all this, the Economist article is a thinly-veiled press release for tenXer, which is itself nothing more than an exercise in testerone-fueled egotism. tenXer measures activity, not productivity [2], and I expect the primary result of increasing your tenXer metrics will be less time thinking, more time hacking, and insufferable smugness.
[1] The Leprechauns of Software Engineering explores what science is and how we distinguish between fact and folklore in software engineering. It specifically explores the 10x claim, and determines that it's folklore. http://leanpub.com/leprechauns
[2] Productivity is defined as output over input, and the output of programming is software capability. No one's come up with a good measure of that yet. So any time someone claims they can prove increased productivity, ask them how they measure it. Chances are good you'll get a bogus value that measures activity instead, such as "lines of code." http://martinfowler.com/bliki/CannotMeasureProductivity.html
One of the better sources for these differences and their magnitude is Peopleware. Instead of citing vague research "somewhere", they did their own primary research. More specifically they set up coding wars where different programmers at different companies assigned to the same task independently.
They found 10x differences in time spent on the average coding war (which was typically a pretty small sample) with those finishing faster typically producing programs that worked better. So by any reasonable measure, at least 10x productivity.
There is your 10x, right? Wrong. They found that the best predictor of programmer performance was the performance of another programmer at the same company. Furthermore they managed to correlate a lot of that performance factor to specific factors, such as having a phone that turned off, a private room, adequate desk space, and so on. There was still something like an unexplained factor of 3 left over, but they didn't know whether it was environment variables they had not looked at (eg training), individual ability (which might be correlated across institutions), etc.
It is true that this research happened in the 80s, before the Internet was widespread. I would love to see it replicated again.
(Note, I'm aware of the existence of many other poorly controlled studies, see http://www.flownet.com/gat/papers/lisp-java.pdf for an example, but what is critical in the Peopleware study is that they acquired information both about productivity and about factors that might cause productivity differences.)
In Peopleware, they also observed that some programmers did not themselves appear to produce much, but their presence on a team made others more productive. They were not managers, they were not leads, they just helped the team work better together, in some fashion, "managing" out or across rather than up or down.
I will jump the shark and speculate that these gel-members tend to be female more often than male, and the gelling-like effect comes from how their presence and patterns of behavior influence the group dynamic. I will further speculate that the dearth of women in development incurs a hidden cost to productivity, since statistically, fewer teams will have women, and consequently, the incidence of gel-members will be lower, and teams more fragmented.
I think there is a business opportunity here somewhere, or maybe it has already been realized in places, but the players have not been eager to trumpet their results.
The thing about these studies, which I do appreciate, is that they are measuring a programmer's ability to work on something:
1) Small.
2) By themselves.
3) Well understood and documented.
4) Doesn't have any threat of changing out from under them.
In the real world, the bottleneck usually isn't writing code, it's writing code that does the right thing and making the right people happy in the right way.
In some ways, trying to apply the lessons from Peopleware in the real world is like using your knowledge of unicorn anatomy to predict the Kentucky Derby.
I've seen developers who create, rather than reduce technical debt; who take longer to get familiar with codebases; have a hard time focusing in the presence of distractions; and who seem to have difficulty ramping up solution domain knowledge (i.e. programming / software engineering techniques). And all of these things seem to be down to individual ability or motivation, i.e. something about the person, rather than external factors.
Most developers suck at programming (and that's actually OK). 90% of the code they write is copy and pasted from elsewhere in the codebase or from the web, and then tweaked until it does something approximately correct. They usually need help when solving a new pattern of problem. What they do often have is domain knowledge not related to programming, and produce just about acceptable results out of the machine in a labour-intensive way that doesn't really scale over time or problem size, but they muddle through with determination.
I don't know about a 10x productivity differences though. I'm not really arguing in favour of that hypothesis. Rather, if anything, I've seen step differences in potential; that is, there are developers that can do things other developers can never do, not in 10x the time, not in 1000x the time.
> I've seen developers who create, rather than reduce technical debt; who take longer to get familiar with codebases; have a hard time focusing in the presence of distractions; and who seem to have difficulty ramping up solution domain knowledge (i.e. programming / software engineering techniques). And all of these things seem to be down to individual ability or motivation, i.e. something about the person, rather than external factors.
My worthless opinion is based on about 20 years of newsgroup / forum membership, but the few years I spent working in companies whose software product was a cost centre for its customers was particularly enlightening. We (the guys working on framework / architecture stuff) tried to grow some of the people from the domain-specific side of things into roles on our side of things. It was an uphill struggle; these people weren't interested in it. Code didn't excite them, at all. They might as well have been shovelling fast food for all the love of craft they had. They came in at 8:30, got their work done, and went home at 5:30.
But the work they did? It would have bored me to tears. I would have quit had I had to do it; or perhaps, I would have surreptitiously built a framework or macro system or something to remove the redundancy and repetition from the tasks and indirectly reducing the tedium of the workload, rather than doing it more explicitly in what I was actually doing (developing the v.next framework for the company). But there's only so much redundancy you can squeeze out of many of these things (turning business rules from specifications into code) before you create too much complexity elsewhere. We still need lots of these people doing this boring work (most software development does not occur in software companies), and it's good that they're expert in the business rules they're translating. At the end of the day, it's OK that they're not great at producing code.
Of course, my opinions are completely invalid because they are generalized from limited experience, have no scientific value, etc. But it's the world I've lived, so I'd have to see decent studies and censuses to convince me otherwise.
That's the tricky thing about productivity, isn't it? You said that those domain-specific programmers were "not great at producing code," but I would argue (with tongue firmly in cheek) that in fact they were the productive ones, and you were not.
Let me explain. Productivity is defined as output over input. Programming input is activity--usually, time working--and output is software capability (or, more generally, business value). As architects, I'll bet that you didn't produce much that the company used. You probably produced designs, frameworks, or other tools for domain programmers to use. But the real software capability was produced by those domain experts you're criticizing, which means that they were the important actors and you were assistants at best. At worst, you could have been an impediment; I've seen "core" teams whose architecture-astronaut frameworks actually have to be subverted in order for the domain teams to get their work done.
I'm playing devil's advocate here, and I'm not entirely serious, but it's with a point: skill in coding isn't important unless it leads to tangible business results. In the short term, in my experience, business savvy outweighs coding skill.
(In the long term, technical debt becomes an overwhelming cost, so programmer skill does matter. But keeping technical debt low is more about maintainability, tests, and boringly understandable design than the kinds of things I see "rock star programmers" emphasize.)
I tried to be specific in what I said; I said that it was OK, and good, that these people existed. I was not making an argument that these people are necessarily less productive; but rather, that there was a qualitative difference in how they approached code that made a difference in what they could accomplish with it.
(Though I think in the medium to long term, they will end up less productive, because of a lack of abstraction and leverage of existing code beyond copy and paste. Probably, we don't disagree that much.)
At this point, the terms grow a bit wonkey though, and I am struggling to communicate this properly in upcoming interviews. The big, underlying question is: How productive is a programmer that enables other programmers to be productive. Especially if the enabling programmer works slower and only outputs things no customer will ever see and especially if the work done with the framework outweighs the time spent creating the framework by magnitudes?
You pointed out that productivity can't be measured (though that hasn't stopped you from invoking the concept). Something else that can't be measured? "Business value".
Let alone the "business value" of an individual programmer's commits.
Well, statements like "skill in coding isn't important unless it leads to tangible business results" seem to suggest that there is a way of linking the two. But there isn't. So it's misleading to talk this way. In practice, people just use it to confirm their prejudices (specifically about who is and isn't productive), which they've arrived at for other reasons.
I think "business value" is a particularly bad phrase because it sounds like something that can be quantified and has a clear implication about which class of people will do the quantifying (namely, business people as opposed to technical people). In reality, it can't, and the term is really about power dynamics within companies.
Hmm... I can't agree. "Business value" is a useful term for describing sources of value beyond revenue. It's extremely fluffy and usually unmeasurable, sure, but it's still a useful way of talking about things that are worth doing that don't produce revenue, such as competitive differentiation, market research, brand promotion, philanthropic efforts, and relationship building. For software that doesn't produce direct revenue (the majority, in my experience), what better term is there?
I don't know why you say business value is actually about power dynamics, unless you mean that people use the term to promote their pet projects. If so, I think that has more to do with human nature than the term "business value."
I don't think we understand the term "business value" the same way. It seems to me a better term for what you're talking about would just be "value".
The power dynamic aspect is this: who gets to decide what has "business value"? I would like the answer to be: anyone who has good insight and can convince their team. But when the answer instead is "a class of people called 'managers' or 'businessperson'", that is what I object to. Membership in that class is a matter of organizational status, not insight or merit.
But in a message board discussion it's hard to tell where we're talking about the same things and where we're not. Wish there were an easy way to get semantic diffs.
Well of course, but that's just a trivial restatement of the problem. Anyone can measure dollar amounts once they have them; money is a measure. The question is how to assign such amounts.
For business value to be measurable means you have a function M(x) that maps x to money. The trouble isn't with the range of that function, it's establishing what the domain is and how to compute M.
People often talk about business value on software projects as if they have such an M when they don't. What they have is an imaginary M whose domain is "what I care about" and whose output is "what I want to believe". Such discourse is dysfunctional.
The truth is that you have M for the business as a whole and maybe for specific products (though I can cite cases where even that was dubious) and it typically breaks as you try to get finer-grained.
Wait, so you have seen 10x variations in productivity but assume because 'bob' is say better able to avoid interruptions that does not mean he is a 10x better programmer? Programming is not just a question of time spent in front of a computer soft skills are important. If someone looks at inconsistent requirements and get's clarification rather than jump into a quagmire without end they are being more productive even if there not writing code at the time.
I think the 10-50x observed increase in productivity is task specific. But, people also deal with a finite list of specific problems so what if bob is 2x as productive on random tasks if there is 10 years worth of work where he can be 10x as productive he is worth a lot more than the average programmer.
From what I heard the 10x meme came from a study back in the 60's where they timed something like ten programmers programming on punch cards for a couple of hours. Now, I didn't check the facts behind this myself, but if thats the case its hardly scientifically rigorous and hardly relevant at all today.
It's misleading to single out the 10x meme for the "that's not science" treatment. What beliefs about software development do hold up to the standard of hard (well-designed, controlled, and replicated) experiment? The published literature that I've read is all embarrassingly weak - understandably, given the complexity of what it's trying to study and the minuscule investment in such research.
I haven't read the ebook you're advertising, but a book with an identical pitch, Robert Glass' Facts and Fallacies of Software Engineering, is an example. The difference between his facts and fallacies is that the "facts" tend to have one poorly controlled, never replicated study (usually from 1978) to their name. Better than nothing (maybe), but close to nothing.
My point is that at this stage in history, it's all still folklore. Even you, immediately after playing the science card, followed with mushy folklore of your own ("I've certainly seen...").
Edit: come to think of it, your quotes make it sound like there is empirical evidence for the 10x claim, just that it's old and wasn't replicated. The "old" criticism is weird; it's not at all obvious that the internet invalidated the data. So the critique seems to boil down to: there was a study, but it wasn't bullet-proof and it wasn't properly replicated. What studies of software development isn't that true of?
I'm hardly "singling out" the 10x claim. If you knew me, you would know that I readily criticize programming studies [1]. At any rate, the prevalence of poor science in software development is no reason to be silent about this particular case of poor science.
I'll admit to being guilty of using anecdotes to describe my experiences with software development. (That's not quite the same as folklore, but whatever.) You won't find me claiming that my anecdotes prove anything, though, and I try to be clear that my perspective is based on experiences, not objective truth.
Microsoft (Research?) published some stuff in the last year or two that looked really interesting. I don't have a link at the moment and I'm on my way out the door, unfortunately. But I remember thinking that it had promise because it drew conclusions from a huge base of data from actual Microsoft developers working on long-term projects.
Another paper I like is Little's report on data from projects at Landmark Graphics [1]. It's not a controlled study, and shouldn't be treated like one, but it also looks at a large base of data from real projects.
My biggest complaint about academic papers is that they're often conducted with students, using practices out of context, working on projects of trivial size and depth. I like both of the above not because they're perfect, but because they get beyond those problems.
I haven't read Bossavit's book. I have read McConnell's writings. Chapter 30 of "Making Software: What Really Works, and Why We Believe It" is by him, and edited by Andy Oram and Greg Wilson. The point of the book was to collect scientific evidence. I know Greg, and his high standards, so I strongly doubt that McConnell's evidence of weak.
I'm immediately suspicious of the statement you quoted: "Not a single one of the references is to a replication, in the scientific sense of the term." Strict replication is not and never has been a requirement to good science. As a trivial example, how do you replicate a supernova observation? You can't. But you can make models and test the models against the observed data and against future supernovas, and you estimate the reasonableness of the model effectiveness.
If that quote is indicative of Bossavit's views, then he is a poor judge of what's good evidence.
Going on with more quotes you gave, "which we can safely expect to have wrought major changes in programming practice." That is a conjecture, not a statement of fact. It seems like he's saying that once upon a time there was 10x difference, but the Internet makes everything different now there isn't. However, without evidence to back it up, I don't know why I should believe it.
What tests has Bossavit carried out to show that this is the case?
To the contrary, L. Prechelt "An empirical comparison of C, C++, Java ..." was both carried out during the internet era, was done with good rigor, and shows a large spread in the overall time to complete the project. That's a small project, granted, but it's another data point in the overall trends.
(If you don't like the small project size, another example, given by McConnell, is the large productivity difference between the Excel and Lotus 1-2-3 teams, which produced roughly comparable spreadsheet programs but with about an order of magnitude productivity difference across several different metrics.)
Multiple independently determined data points which all trend in the same direction strengthen the likelihood that the ~10x hypothesis is a useful model. That's what scientific evidence looks like.
McConnell even has a section in his chapter on "how meaningful it is to speak of an individual programmer's productivity", along with all of the caveats and cautions that everyone here is bringing up (some programmers produce negative lines of code, the best programmers tend to get the hardest problems, the effect of teams, etc.), and concludes "So while I see the value in measuring individual performance in research settings, I think it's difficult to find cases in which the measurement effort is justified on real projects."
Bossavit's book comes after McConnell's reply, and goes into more depth than the essay McConnell is replying to. Personally, I found the book more convincing. And having read what Bossavit had to go through to verify McConnell's references, I'm not convinced Greg Wilson did the same.
Rather than try to defend someone else's words, though, I'll just direct you to the book. You're going into a lot of detail critiquing quotes from an end-of-chapter summary, which isn't fair to Bossavit's work. I hope you'll read the book, because I'd like to hear your critique of the full work.
I would rather not. As it turns out, I haven't even finished reading "Making Software", not for lack of the quality of the essays but because my interest in this area isn't as high as I thought it would be.
I can be convinced otherwise for this one case. Could you elaborate for me on why my critique on these points is unjustified? Why is sufficient scientific evidence for to Bossavit? Why does the Internet necessarily discard older studies, and how does Bossavit establish that that's the case?
Given that the idea of the 10x programmer seems to be considered bunk by most of HN (as per the above post), I'm curious about the idea of a Free Electron described here: http://www.randsinrepose.com/archives/2005/03/20/free_electr... . Would this just be another name for the same idea, or something separate that's more realistic?
Looks at Spolskys article "Hitting the high notes" which is based on Yale prof. Stanley Eistenstat's meticolously gathered data of time spent among students for programming assignments.
I think the author glossed over an important point:
"Within individual firms, the difference in performance was only 20% or so."
Is this because good developers congregate together, as suggested, or because having at least one good developer raises the productivity of those around him or her? They can certainly influence an organization through their choice of platforms and tools, aiding the productivity of everyone on the team.
Side issue with this article/tenXer press release: the thing about constantly reworking code being a sign of a bad programmer was wrong -- it actually takes more skill to be able to do routine refactoring. Not to say that that is always practical.
Also, taking advantage of existing software makes a huge difference in 'productivity' at least as far as business people can measure. And someone who does that without acknowledging that they are doing it may appear to be a 'superstar' to management.
For example, someone may just consider all of the source code they have ever written, regardless of the circumstances they wrote it in, to be a code library they can copy and paste in whatever company they currently work at. And not even acknowledge that they are reusing code they wrote years ago.
So that sort of dishonesty irritates me, but generally I think taking advantage of existing software is the right thing to do and is the number one factor in what non-technical people would perceive as productivity. There is a huge difference in the time required to build a system using an easily configurable base like WordPress with plugins or components or by integrating open source software or libraries versus building various parts or all of the system from scratch.
That's where you will see orders of magnitude difference in productivity where one person or group is reinventing a bunch of wheels that another group is not. And again there are many different ways of avoiding reinventing wheels, from using Google to basing software on existing open source programs to selecting more practical application programming languages/tools (like ones with things like garbage collection or more straightforward syntax or support for interactively configurable graphical components/plugins), or for example doing test driven development or in general having any type of automated QA rather than relying on a manual QA process entirely.
"An outfit in San Francisco called 'tenXer' has begun testing a service that aims to help people boost their mental accomplishments by up to tenfold—hence its name. . . . "
in San Francisco is currently offering a beta version of its tracking software to help people analyse their own performance, set goals for themselves and improve their lot. At present, the tracking software works only for programmers. But down the road, the same techniques could just as readily be applied to other occupations. . . . "
The great thing about this is that many of us on Hacker News can put the claim to the test. Commenting on the claims of tenXer, which is what the writer for The Economist and several of the commenters here on Hacker News are doing, is interesting, and trying this out and seeing whether or not it works might be even more interesting. It's still unclear how much most individual human beings can improve themselves if they apply effort to self-improvement. It might be a good opportunity for career development for many HN participants to try this out.
The only example here given of the powers of these superprogrammers, is that they have banded together to make an OS that’s technically impressive, but gets no traction in the marketplace and ends up all but forgotten (PC/GEOS)? Not very enticing…
Having written both application and system code for PC/GEOS, it never appeared to have been written by 10Xers.
For the record I've worked with EPOC-32, Windows, NextStep, Linux, VxWorks, and Nucleus - none of them was as painful as PC/GEOS (although EPOC-32 sometimes came close...)
I see no comments yet on one of the biggest factors underlying highly productive programming: choosing the part of a development at which you can most easily excel. Sometimes there are legitimate areas where you have expertise, but mostly it's the well-defined areas that have no external roadblocks. And the rough parts get left for everyone else!
Articles like this only depress me, making me realize that I'm a mere web developer and not even a REAL programmer, and in the realm of web developers probably not even remarkable. Which makes me probably LESS than mediocre as a programmer.
if you replace "productivity" with "effectiveness", it feels even more true, with the emphasis on doing the right work, rather volume of work. as programmers, if we do the right work, our software "produces" more, so we readily conflate the two, but in other fields where the product is less animate, effectiveness means your unit of output is the right output, even if near & local productivity doesn't vary much.
I'd see the most value in a programmer who could combine simple ideas/algorithms for some practical, yet useful application. Not necessarily a 10x l33t haxor genius.
one aspect this "superstar"-worship drivel^Harticle seems to ignore is the organizational contribution to success.
i'd argue that individual contribution to any given project of substance is greatly overestimated, while contribution of the team (organization/culture/whatever) is unfairly underestimated.
even in professional sports, superstar a winning team does not make.
True, I remember one study (I can't recall the link sadly) that measured performance of several programming teams and found strong correlation between women in workforce and team performance.
Another study in the book "Imagine" by Lehrer Jonah implies that creativity depends on the Q factor, inside a group. Here is a quote:
"Uzzi’s data clearly demonstrates that the best Broadway shows were produced with intermediate levels of social intimacy. A musical produced at the ideal level of Q (2.6) was two and a half times more likely to be a commercial success than a musical produced with a low Q (<1.4) or a high Q (>3.2). It was also three times more likely to be lauded by the critics."
I can see how hiring only rockstars which probably knew each other would increase the Q beyond the ideal level.
What really genius-level things has Fog Creek done? I remember demoing FogBugz, and it was marginally better than its competitors, but it was far from a complete game-changer. Trello is ok, but it's only marginally better than a whiteboard with post-it notes. None of these reinvent an industry, or provide something so outstanding that people rush toward it.
Don't get me wrong; I think Joel has a lot of good ideas in general, and Stack Exchange is indeed a game-changer. But as someone who continually talks about hiring "rock star programmers", I simply don't see a result that I'd consider "rock star" equivalent. But, maybe I'm missing something?
Edit: More than that, I don't even consider myself a rock star developer, just a normal developer who loves learning new things and getting a lot of stuff done. I'd be thrilled to work for a Google or Microsoft, but what would attract "rock star developers" to Fog Creek? Basically, are fancy chairs enough? Isn't the most attractive thing a super interesting problem domain?