I used to believe that every company gets one rewrite, but only because I have seen that most places have the patience and the stamina for a little bit less than a single rewrite, but I was on the fence about whether they were a good idea anyway.
Trouble is, I could never put my finger on why, other than that it never seemed to fix the problem. It was a bit like moving to a new city to start over and finding out you brought all your problems with you.
In the last couple of years I have begin to figure out what I know in a way I can articulate. The people who have the skills and discipline to take advantage of a rewrite don't need a rewrite. It's a short distance from that skill set to being able to break the problem down and fix it piece by piece. They just need permission to fix key bits, one bite at a time, and they've probably already insisted on the space to do it, although they may be clever enough never to have said it out loud. They just do it.
Then you have the people who don't have the skills and discipline to continuously improve their code. What the rewrite buys them is a year of nobody yelling at them about how crappy the code is. They have all the goodwill and the hopes of the organization riding on their magical rewrite. They've reset the Animosity Clock and get a do-over. However, as the time starts to run down, they will make all of the same mistakes because they couldn't ever decompose big problems in the first place, and they lack the patience to stick with their course and not chicken out. They go back to creating messes as they go, if they ever actually stopped to begin with. Some of them wouldn't recognize the mess until after its done anyway.
In short, people who ask for a rewrite don't deserve a rewrite, and would squander it if they did. It's a stall tactic and an expensive dream. The ones who can handle a rewrite already are. They never stopped rewriting.
Now, I've left out all of the problems that can come from the business side, and in many cases the blame lies squarely with them (and the pattern isn't all that different). In that case it might take the rewrite for everyone to see how inflexible, unreasonable and demanding the business side is, and so Garbage In, Garbage Out rules the day. But the people with the self respect to not put up with it have moved on, or gotten jaded and bitter in the process.
I was expecting some real wisdom in this story, but I don't think this qualifies him as a technical leader in any way. I don't even think this needed the cargo-cult-y "leaving the room and coming back with a paper that changes everyone's mind." Couldn't he have just asked for a plan? And the ROI? That seems like management 101.
I have no love for him, but he is a genius. This story doesn't do him any justice (nor is the paper particularly compelling).
My point was not to qualify Jeff as a technical leader (though he is an excellent technical leader, IMO). Rather, my goal was to give people a sense for his leadership style, and what it's like to be in a high level meeting at Amazon.
I'm sorry you find the Rickover memo uncompelling. Speaking as an experienced engineer, I happen think it's among the best pieces of engineering writing ever produced.
I have successfully done a full rewrite. 60kLOC of VBScript/ASP to 20kLOC python/django. Added a ton of missing features that wouldn't have been possible in the old, clipboard inheritance style. There was one page on the old site that I had tried to fix up a couple of times and failed completely at. It was ~2500 lines. New system it was nothing special, 25 lines in the view and a big but manageable template.
I did this whole rewrite in about 9 months including all the work to do the database migration which took a bunch of dry runs and then the actual switchover which took nearly 12 hours to run.
What's the catch? I had been working at the company for about 7 years at that point as the sole caretaker of the application so I knew the business needs inside and out, I knew where all the problem areas were and I had plenty of ideas about what features we desperately wanted to add but couldn't. I didn't necessarily add all of them up-front but I was able to make good schema decisions to support those later features.
Rewrites aren't impossible by any means, but I suspect my story is more of the exception than the rule.
I've done two rewrites at the startup I joined 2.5 years ago of codebases that were our core competitive advantage and that were no longer maintained when I joined. Both of the rewrites were successful, but they were small codebases (5k & 10k LoC) and I was the expert on what they were meant to be doing. The first one was our client-side analytics product that I ported from a single 5k LoC CoffeeScript file to TypeScript, the second was a Node.JS/CoffeeScript application that I ported to Java/Groovy.
I think the move to TypeScript was probably the one with the least clear benefit; Our lack of experience with CoffeeScript and the compiler's lax semantics were causing us problems, but we could have moved to pure JavaScript. I like TypeScript and I feel more confident refactoring and merging with a statically typed language, but it's hard to quantify. This took about 6 weeks, though most of my attention going to fighting customer-facing fires during this period. Launching this was excruciatingly stressful though because there was a bug we could only tell existed through aggregate A/B test statistics.
The second one from Node.JS/CoffeeScript to Java/Groovy has seen more clear benefits from being on the JVM since our team has more experience with it overall, both in Dev & Ops, and a direct port boosted perf 3x and we've benefited from the more extensive library support and interoperability with other JVM languages we use elsewhere in our system. This took about 2 months to go from nothing to prod. It was also useful organizationally since we'd talked about rewriting it for so long that we'd been treating the existing system as legacy and not wanting to invest any extra time in that codebase for that reason. Releasing this was comparatively easy since it essentially acted as a streaming transformer, taking JSON in from one queue and emitting JSON out another queue without any state and I could confirm that the new system had only irrelevant changes. Still, half of this time was spent on testing.
I'm currently at my first job, working at a budding unicorn company in silicon valley. I've been here 1.75 years and am already in the midst of a 3rd rewrite - although not my decision (from higher ups)
The last paragraph really hits home for me and i've 100% gotten very jaded and very very bitter.
I used to believe that every company gets one rewrite, but only because I have seen that most places have the patience and the stamina for a little bit less than a single rewrite, but I was on the fence about whether they were a good idea anyway.
Trouble is, I could never put my finger on why, other than that it never seemed to fix the problem. It was a bit like moving to a new city to start over and finding out you brought all your problems with you.
In the last couple of years I have begin to figure out what I know in a way I can articulate. The people who have the skills and discipline to take advantage of a rewrite don't need a rewrite. It's a short distance from that skill set to being able to break the problem down and fix it piece by piece. They just need permission to fix key bits, one bite at a time, and they've probably already insisted on the space to do it, although they may be clever enough never to have said it out loud. They just do it.
Then you have the people who don't have the skills and discipline to continuously improve their code. What the rewrite buys them is a year of nobody yelling at them about how crappy the code is. They have all the goodwill and the hopes of the organization riding on their magical rewrite. They've reset the Animosity Clock and get a do-over. However, as the time starts to run down, they will make all of the same mistakes because they couldn't ever decompose big problems in the first place, and they lack the patience to stick with their course and not chicken out. They go back to creating messes as they go, if they ever actually stopped to begin with. Some of them wouldn't recognize the mess until after its done anyway.
In short, people who ask for a rewrite don't deserve a rewrite, and would squander it if they did. It's a stall tactic and an expensive dream. The ones who can handle a rewrite already are. They never stopped rewriting.
Now, I've left out all of the problems that can come from the business side, and in many cases the blame lies squarely with them (and the pattern isn't all that different). In that case it might take the rewrite for everyone to see how inflexible, unreasonable and demanding the business side is, and so Garbage In, Garbage Out rules the day. But the people with the self respect to not put up with it have moved on, or gotten jaded and bitter in the process.