As a programmer, one of the most effective forms of deliberate practice I've ever experienced is to write a complicated piece of software, and then completely re-write it from scratch with a different approach (and then do it again).
Of course, that almost never flies in the business world. They'll ask you why on Earth the company should pay to build something again before they make money from it. However, it truly is a powerful engineering process for many types of problems.
The point is not to merely rebuild it for arbitrary reasons. Rather, to take stock of the benefits and the pain that arises from the current design, and to try completely different approaches. You do it because there's a nagging voice in the back of your head that tells you that you've left some "goodness" on the table ... that even though your current implementation "works", the problem could be solved better.
You've already "loaded the problem into your head," because you've solved it once. The second solution will improve in some ways and possibly regress in some ways. Often, the 3rd or 4th solution will be "right" -- which means, not perfect, but significantly better than your earlier approaches.
It seems kind of like learning the card game of Bridge by playing the same hand over and over to see if you can get it better. Or learning chess by starting from the same canned starting position and seeing if you can play it to completion better.
That's actually a fairly well-known thing. Look for "Second System Effect" (See "Mythical Man Month").
Basically, v1 sort of gets the features working, but has lots of work-arounds. v2 is trying to solve all the issues you got wrong in v1, and wants to be the most beautifully architected solution.
As a result, it's almost always completely over-engineered and in danger of collapsing under its own feature weight.
Brooks does talk about this topic in "The Mythical Man Month", but this isn't what he calls the "second system effect". The "second system" is the second significant system you build in your career, not a re-write of the first system. Brooks' "second system" was OS/360, written after he had worked on an operating system for an earlier machine.[1]
The idea of building a prototype and then re-writing it is what Brooks talks about when he says "plan to throw one [version] away; you will anyhow."[2]
I have heard of that (and also, unfortunately, experienced it), but that is not what I was talking about.
The second-system effect is a product of naiveté. I've made that mistake more than once, but I believe I know better than it now.
But this practice of rebuilding from scratch I'm describing is not that. It is more a matter of practicing craftsmanship.
It's like a sculptor who sculpts a wolf, sees its good and bad, then gets the exact same size block of wood and sculpts it again -- but differently. And again. Not out of naiveté, but out of a patient determination to get better.
I used to do the exact same thing in Math classes. Iterate on proofs, until I got the perfect one. It vastly enhanced my understanding and gave me these really pretty proofs to look at!
In one class where I was especially obsessed, my (rather famous in the field) professor told me I understood the subject better than he did. That was pretty awesome.
"second system effect" is not naiveté, in my experience. The extent of it might be controlled by experience, but the 2nd one always learns from the first one and overshoots.
It's similar to "bracketing" when you fire artillery. You need one in front and one in the back of the target to be able to hit with the third one.
It seems to me to be one of the central aspects to design, of any persuasion. It is a deliberate, thoughtful act which requires understanding rather than 'just' knowledge. It is a process of questioning and revision rather than merely production. All (good) designers do this.
I think the interesting idea in this article is actually it's corollary. Good knowledge workers ARE actually bad at 'just' working. They think to much, they spend too much time on things that give diminishing returns, they invest in dead ends... they learn. These things are not often valued in the job market.
Good knowledge workers ARE actually bad at 'just' working. They think to much, they spend too much time on things that give diminishing returns, they invest in dead ends... they learn. These things are not often valued in the job market.
I'd be curious to see some real data on the long-term evolution of (a) the insubordinate, always-learning curious people who get new knowledge out of every job even if they have to steal an education from work, and (b) the order-following ladder climbers. Within companies, (b) always wins. Who wins in the long-term career game? I honestly have no idea.
You also have people who do 40 hours of work in 20 hours and spend the other 20 learning. They satisfy the requirements of both (a) and (b). And they may very well have the advantages of both.
Great point! I think this is what happens. I've seen this person, and sometimes, I've been this person.
The employer/client often knows that some of the work this type of person does doesn't directly pay immediate tangible benefits. Sometimes they're even frustrated by it. But, on the other hand, the programmer is just so damn good that they have to put up with it.
Overall, the boss who doesn't understand this principle of "deep work" feels frustrated, but still has to acknowledge that these deep workers are his most effective producers. So the boss ends up feeling he has to "take the good with the bad," almost like you'd feel about a great painter who is a prima donna or something. (But in this case, the "prima donna-ness" is not due to a character defect, but rather, the underlying nature of software development).
At a large corporation, it's generally a good idea to avoid letting your boss know how much free time you have. The problem is that if you volunteer your extra time, you'll end up doing 80 hours of work in 40 hours. This is not nearly as beneficial to your career as spending the 20 hours developing deep skills.
I'm not saying lie. If your boss has something that needs to be done and asks you to do it, then get it done. But don't volunteer for extra work.
The times that I have been the person getting 40 hours worth of work done in 20 hours, I was using the other 20 hours to rewrite things with a different approach ... in other words, the "deep work" that the business didn't see a need for.
I knew that the deep work was why I could get 40 hours worth done in 20 hours, so I made sure I did the deep work. That, of course, put me behind in my directly assigned (non-deep-work) tasks. However, the deep work paid off in more powerful ways (more powerful abstractions, allowing me to find solutions to problems we thought we would be unable to solve, discovering real problems that we didn't yet know we had, etc.), and ultimately made it possible for me get my tasks done in the other 20 hours (or, to deliver something else of enough value that they decided my original tasks didn't matter).
It's not nearly as neat as "40 hours worth in 20 hours", and it certainly didn't lead to a bunch of free time sitting on my hands. It was just a case of knowing the deep work was necessary, even if those assigning me work didn't, and doing that deep work anyway, and having things work out well because the deep work really was necessary and powerful.
And I think that happens a lot with good developers in the "enterprise" software world.
I would be interested too (and BTW you have a great blog with interesting theories)
I just would NOT call good knowledge workers "bad" like the OP, "investing in dead ends".
Good knowledge workers care about the baseline and the consequences of their actions. The (b) type only care about improving their own profile - baseline be damned if it helps them.
Michael, it seems like maybe those of type (a) become independent consultants, and those of type (b) stay in one place and put a lot of energy into politics.
But that may be an overly cynical simplification :)
Some context: Here Cal Newport is extending some of the ideas presented in his latest book "So Good They Can't Ignore You".
In the book he says (among other things) that how you become 'so good that...' etc is by what he calls 'deliberate practice' - which in turn has two components -
(1) 'deep work', the doing of which actively stretches you while you are doing it (in term of guitar playing, this would be acquiring new rhythmic abilities, vs just playing/performing songs you already know). The 'practice' in deliberate practice should maximise 'deep work' (as per Newport)
(2) fast feedback loops - either by other (and ideally better than you) people, or built in (if you play a note wrong, you know instantly)
This blog entry and its neighbors seem to explore how he applies this philosophy to his chosen activity (research in CS). It is not entirely clear how to apply it to programming.
> In order to achieve consistent deep work, I usually need to first immerse myself in the relevant research literature, seeking a problem that it is recognized as important, unanswered, but probably answerable with my skill set. This is not easy.
I want to run out and repeat this to every PhD student I've met in the last 2 years.
The claim that knowledge workers spend all day answering email and do nothing to focus on self-improvement is just plain bullshit. The fact that we're presently using a website that essentially serves as a crowdsourced, real-time trade journal counters that argument. The existence and wild success of engineering-focused conferences counters that argument. The proliferation of easy-to-access information for professional growth allows for a self-selecting group of people to improve their lot in life just by picking up some new skills and getting some experience with them. You can learn a new skill literally by browsing the Internet these days. As such, I don't really view the knowledge worker as a person at risk of stagnation without actually making an effort to be lazy.
There are some people who feel that they already know everything they need to know, and don't want to better themselves. It is entirely attitudinal, and in no way unique to the "knowledge worker" classification. I would argue the problem is far more significant in other industries, such as manufacturing and other heavy industry. When a specific kind of job replaced by a machine, or an entire industry gets outsourced to another country, the individuals previously responsible for that kind of work are likely not to have been trained for any other function. That would probably be a better audience for this kind of thing, compared to people who spend their free time reading about new technologies and playing with them on the weekends for fun.
> You can learn a new skill literally by browsing the Internet these days
Nope, you can learn of a new skill by browsing. And that's quintessentially shallow work.
I'd also point out that HN ( like the original reddit) was full of "hard" articles, that would take me hours to really understand and work through. While this kind of article still pops up occasionally here, its more common to just see easily digested blog posts now. I assume this is a normal consequence of HN's increased readership and regression to the mean.
The problem that this blog post addresses may be more generally applicable than the author thinks, but the existence of HN is hardly a counterexample.
The fact that we're presently using a website that essentially serves as a crowdsourced, real-time trade journal counters that argument.
I'm pretty sure that "HN members" form a VERY small percentage of the overall population of knowledge workers. I don't have numbers available to "do the math", but I'm guessing "we" are well less than 1%.
On a related note, there is a very real risk in larger, more bureaucratic, organizations that it becomes easier to be recognized for shallow work than for deep work, managers like metrics and the shallow work is typically very much easier to quantify - leading to the Dilbert "I'm gonna right me a new mini-van" scenario - or whatever the social media equivalent is.
I get the sense that a lot of organizations go out of their way to make sure individuals are recognized for shallow work, because we know by experience it's less self-fulfilling for most of us (if not outright boring).
I still proudly keep the multimeter I won for "Tech of the Month" (I was the tech who accounted for the highest percentage of time in the time tracking system that month.)
I agree with the OP. My day job is somewhat interesting but I have a feeling once I've figured out how it works that I'll be back in the same boat of doing less of my craft and more process. I am very lucky that in software engineering the body of knowledge is wide and always changing and that I genuinely like it.
At night I've become more accustomed to reading tech books on the newest technology instead of whatever book for book club. Our profession almost forces us to keep up or get left behind. I love it. Always gets me motivated to think and expand my knowledge.
This is a great article, I can totally relate.
I have been doing work that had not involved deep work for maybe a year, I kinda had plateaued my learning as a result of an unchallenging and demotivating work environment, I realised this is not what 'I' wanted to do, so I got out. I have hacker news to thank for this partly too.
4 months later I feel like a kid again my mind is like a sponge for knowledge, and because I have cleared my desk of non beneficial work, and as a result I have been learning at a fantastic rate.
Things I have been spending my team on include, functional programming techniques, JavaScript client architecture, pure REST api design, algorithms, visualisations in the browser (d3), geojson, node, mongo, redis, to name but a few
I think this article is a great summary on how to improve your skills as a knowledge worker. I see people that believe their mental capabilities will never improve. It really comes down to step #3: the stretch. Many people skip over "the stretch" in all areas of life and reach "steady state" relatively early on.
Please spare us this trite bullshit. I see people trying to justify their inability to improve themselves all the time. "If I'm having trouble, everyone else must be too!", they think, instead of considering the obvious: maybe they just suck.
There will always be time-wasters and people who get off on bureaucracy. But I'm willing to bet many of us, especially here on HN, routinely get admonished for "forgetting" to do menial, time wasting tasks in favor of doing what we love. If you're wasting your time instead of taking every opportunity to learn you'll get no sympathy from me.
I posted it not because I agree with his assertion (I'm not sure I buy everything he's saying) but because I thought it would spark interesting discussion. Since "knowledge worker" is roughly synonymous with "white-collar office worker" (not necessarily programmer) I am inclined to agree with him.
I'm willing to bet many of us, especially here on HN, routinely get admonished for "forgetting" to do menial, time wasting tasks in favor of doing what we love.
Sure, we do. Most programmers and especially most white-collar workers don't. They waste their time instead because it's the more comfortable way of dealing with management, which means they never improve and we have to clean up their messes and deal with their shitty code. It becomes everyone's problem.
I did not mean to direct that at you, but at the author. I agree that there are some interesting points in the article, but the pervading attitude in this article and others on the site is that you can use these methods as a substitute for real passion about what you do. I submit that going through such motions mechanically won't be easy because, after all, it's still "work." If you'd rather be doing email or browsing reddit, a disciplined method of self-study isn't going to help. Only if you're already passionate about what you do will you be able to consistently engage in the "deep thinking" the author talks about.
One thing OP discusses in his self-help literature (which I haven't read, but I've heard it's pretty legit) is that rather than trying to rearrange their lives around finding some "passion", people should try to bring passion to their work. It's a counterbalance against the "do what you love and the money will follow" bullshit that has wrecked thousands, if not millions, of lives.
There's some sense in this. For example, when I was a teenager, I thought being a novelist or poet was what I wanted to do, and I had no idea that I'd ever find things like "machine learning" and "functional programming" interesting because I didn't even know what they were. I realized that the market signals were probably right in steering me along the mathematical direction, because (at least for this point in my life) I'm a better fit for the high-end CS and mathematics than I would be for the literary world.
All that said, there are definite limits to that approach. He's right that people need to be more flexible regarding what to be passionate about, but for a lot of jobs, there just is no way to become passionate without also becoming unemployable and insubordinate. What most self-help writers fail to grasp fully is that most people face inflexible, life-wrecking constraints and limitations that they're powerless to change.
For a lot of jobs, there just is no way to become passionate without also becoming unemployable and insubordinate.
FYI, Cal actually covers this in his book-he has a very specific framework for figuring out if a career has a reasonable possibility of giving you the autonomy, competence, and freedom you need down the line.
Of course, that almost never flies in the business world. They'll ask you why on Earth the company should pay to build something again before they make money from it. However, it truly is a powerful engineering process for many types of problems.
The point is not to merely rebuild it for arbitrary reasons. Rather, to take stock of the benefits and the pain that arises from the current design, and to try completely different approaches. You do it because there's a nagging voice in the back of your head that tells you that you've left some "goodness" on the table ... that even though your current implementation "works", the problem could be solved better.
You've already "loaded the problem into your head," because you've solved it once. The second solution will improve in some ways and possibly regress in some ways. Often, the 3rd or 4th solution will be "right" -- which means, not perfect, but significantly better than your earlier approaches.
It seems kind of like learning the card game of Bridge by playing the same hand over and over to see if you can get it better. Or learning chess by starting from the same canned starting position and seeing if you can play it to completion better.