As a programmer, one of the most effective forms of deliberate practice I've ever experienced is to write a complicated piece of software, and then completely re-write it from scratch with a different approach (and then do it again).
Of course, that almost never flies in the business world. They'll ask you why on Earth the company should pay to build something again before they make money from it. However, it truly is a powerful engineering process for many types of problems.
The point is not to merely rebuild it for arbitrary reasons. Rather, to take stock of the benefits and the pain that arises from the current design, and to try completely different approaches. You do it because there's a nagging voice in the back of your head that tells you that you've left some "goodness" on the table ... that even though your current implementation "works", the problem could be solved better.
You've already "loaded the problem into your head," because you've solved it once. The second solution will improve in some ways and possibly regress in some ways. Often, the 3rd or 4th solution will be "right" -- which means, not perfect, but significantly better than your earlier approaches.
It seems kind of like learning the card game of Bridge by playing the same hand over and over to see if you can get it better. Or learning chess by starting from the same canned starting position and seeing if you can play it to completion better.
That's actually a fairly well-known thing. Look for "Second System Effect" (See "Mythical Man Month").
Basically, v1 sort of gets the features working, but has lots of work-arounds. v2 is trying to solve all the issues you got wrong in v1, and wants to be the most beautifully architected solution.
As a result, it's almost always completely over-engineered and in danger of collapsing under its own feature weight.
Brooks does talk about this topic in "The Mythical Man Month", but this isn't what he calls the "second system effect". The "second system" is the second significant system you build in your career, not a re-write of the first system. Brooks' "second system" was OS/360, written after he had worked on an operating system for an earlier machine.[1]
The idea of building a prototype and then re-writing it is what Brooks talks about when he says "plan to throw one [version] away; you will anyhow."[2]
I have heard of that (and also, unfortunately, experienced it), but that is not what I was talking about.
The second-system effect is a product of naiveté. I've made that mistake more than once, but I believe I know better than it now.
But this practice of rebuilding from scratch I'm describing is not that. It is more a matter of practicing craftsmanship.
It's like a sculptor who sculpts a wolf, sees its good and bad, then gets the exact same size block of wood and sculpts it again -- but differently. And again. Not out of naiveté, but out of a patient determination to get better.
I used to do the exact same thing in Math classes. Iterate on proofs, until I got the perfect one. It vastly enhanced my understanding and gave me these really pretty proofs to look at!
In one class where I was especially obsessed, my (rather famous in the field) professor told me I understood the subject better than he did. That was pretty awesome.
"second system effect" is not naiveté, in my experience. The extent of it might be controlled by experience, but the 2nd one always learns from the first one and overshoots.
It's similar to "bracketing" when you fire artillery. You need one in front and one in the back of the target to be able to hit with the third one.
It seems to me to be one of the central aspects to design, of any persuasion. It is a deliberate, thoughtful act which requires understanding rather than 'just' knowledge. It is a process of questioning and revision rather than merely production. All (good) designers do this.
I think the interesting idea in this article is actually it's corollary. Good knowledge workers ARE actually bad at 'just' working. They think to much, they spend too much time on things that give diminishing returns, they invest in dead ends... they learn. These things are not often valued in the job market.
Good knowledge workers ARE actually bad at 'just' working. They think to much, they spend too much time on things that give diminishing returns, they invest in dead ends... they learn. These things are not often valued in the job market.
I'd be curious to see some real data on the long-term evolution of (a) the insubordinate, always-learning curious people who get new knowledge out of every job even if they have to steal an education from work, and (b) the order-following ladder climbers. Within companies, (b) always wins. Who wins in the long-term career game? I honestly have no idea.
You also have people who do 40 hours of work in 20 hours and spend the other 20 learning. They satisfy the requirements of both (a) and (b). And they may very well have the advantages of both.
Great point! I think this is what happens. I've seen this person, and sometimes, I've been this person.
The employer/client often knows that some of the work this type of person does doesn't directly pay immediate tangible benefits. Sometimes they're even frustrated by it. But, on the other hand, the programmer is just so damn good that they have to put up with it.
Overall, the boss who doesn't understand this principle of "deep work" feels frustrated, but still has to acknowledge that these deep workers are his most effective producers. So the boss ends up feeling he has to "take the good with the bad," almost like you'd feel about a great painter who is a prima donna or something. (But in this case, the "prima donna-ness" is not due to a character defect, but rather, the underlying nature of software development).
At a large corporation, it's generally a good idea to avoid letting your boss know how much free time you have. The problem is that if you volunteer your extra time, you'll end up doing 80 hours of work in 40 hours. This is not nearly as beneficial to your career as spending the 20 hours developing deep skills.
I'm not saying lie. If your boss has something that needs to be done and asks you to do it, then get it done. But don't volunteer for extra work.
The times that I have been the person getting 40 hours worth of work done in 20 hours, I was using the other 20 hours to rewrite things with a different approach ... in other words, the "deep work" that the business didn't see a need for.
I knew that the deep work was why I could get 40 hours worth done in 20 hours, so I made sure I did the deep work. That, of course, put me behind in my directly assigned (non-deep-work) tasks. However, the deep work paid off in more powerful ways (more powerful abstractions, allowing me to find solutions to problems we thought we would be unable to solve, discovering real problems that we didn't yet know we had, etc.), and ultimately made it possible for me get my tasks done in the other 20 hours (or, to deliver something else of enough value that they decided my original tasks didn't matter).
It's not nearly as neat as "40 hours worth in 20 hours", and it certainly didn't lead to a bunch of free time sitting on my hands. It was just a case of knowing the deep work was necessary, even if those assigning me work didn't, and doing that deep work anyway, and having things work out well because the deep work really was necessary and powerful.
And I think that happens a lot with good developers in the "enterprise" software world.
I would be interested too (and BTW you have a great blog with interesting theories)
I just would NOT call good knowledge workers "bad" like the OP, "investing in dead ends".
Good knowledge workers care about the baseline and the consequences of their actions. The (b) type only care about improving their own profile - baseline be damned if it helps them.
Michael, it seems like maybe those of type (a) become independent consultants, and those of type (b) stay in one place and put a lot of energy into politics.
But that may be an overly cynical simplification :)
Of course, that almost never flies in the business world. They'll ask you why on Earth the company should pay to build something again before they make money from it. However, it truly is a powerful engineering process for many types of problems.
The point is not to merely rebuild it for arbitrary reasons. Rather, to take stock of the benefits and the pain that arises from the current design, and to try completely different approaches. You do it because there's a nagging voice in the back of your head that tells you that you've left some "goodness" on the table ... that even though your current implementation "works", the problem could be solved better.
You've already "loaded the problem into your head," because you've solved it once. The second solution will improve in some ways and possibly regress in some ways. Often, the 3rd or 4th solution will be "right" -- which means, not perfect, but significantly better than your earlier approaches.
It seems kind of like learning the card game of Bridge by playing the same hand over and over to see if you can get it better. Or learning chess by starting from the same canned starting position and seeing if you can play it to completion better.