Throwing out all your code and rewriting it from scratch is like demolishing your building and constructing a replacement from scratch. It's usually not the right thing to do, it occasionally and rarely is the right thing to do, and it's always the tempting thing to do.
It's stupid to argue, "You should never tear down a building" (like http://www.joelonsoftware.com/articles/fog0000000069.html implies) and it's also kinda stupid to write an article entitled, "Why We Tore Down Our Building (And Why You Should Too)"
Neither is universally right. The right thing to do in a given situation is a difficult and complicated question to answer, and it depends on the building in question: how big it is, how well it was designed, how well it's been maintained, etc.
>"like demolishing your building and constructing a replacement from scratch. It's usually not the right thing to do"
Actually, tearing down a building and rebuilding is very often the right thing to do, but isn't done for the same reasons people don't toss their code and rewrite from scratch - sunk costs.
Unlike software, new buildings are almost invariably significantly more efficient and less resource intensive than old ones - and green building has, of course, gone from a fringe movement in the wake of the Rio Summit, to the mainstream and is now standard for public and corporate buildings.
Like poorly written software old buildings are expensive to operate and maintain (and unlike well written software buildings still deteriorate over time when left only to their own devices). Finally, like software one must go through old buildings and figure how systems were designed and installed and patched in order to modify them - and this is very time consuming. Per square foot costs of renovation are commonly as much or more than new construction.
How often do you build a prototype building? Or a building as a side project?
Your analogy fails, sir.
It's easier and more common to write dirty code that functions as it should than build a building with poor foundation but that looks and functions as it should.
> How often do you build a prototype building? Or a building as a side project?
This is one of the most amazing things about rapidly growing cities in most of the world. Neighborhoods that start out as informal shanty-towns full of lean-tos with leaky roofs become, over the course of years, first neighborhoods with modest but permanent cement block buildings and tin roofs, and sprout all the typical urban amenities like power lines and sewers (often at first pirated ad-hoc from other nearby neighborhoods), have their roads paved, etc., and a few decades later look like normal neighborhoods, sometimes even surprisingly gentrified ones.
Absolutely, as someone who grew up in a city like Mumbai, I know much about shanty towns and from my knowledge of them, they are far from ideal. ie. They are not know for a strong foundation, cleanliness, longevity etc. They are constantly hacked and patched by its residents to ensure basics like protection from rain, electricity and basics like a toilet.
But wait, we were talking about buildings, not cities. A city is made up of many buildings. And buildings are torn down across cities on a daily basis.
I agree with the position that it depends, code may be a building, or a city or any number of things.
I disagree with the OP that deciding to recode can simply be compared to tearing down and making a new building for no reason. That's oversimplification.
First, there are times when buildings must be demolished and built from scratch. The OP makes this idea seem ridiculous but it happens everyday when buildings are in fact demolished and rightly so. Second, fewer buildings are demolished and built from scratch relative to programming projects demolished and built from scratch due to the inherent differences between a coding project and a building project. For one, the former is more regulated and carries more physical risk than the latter(a coding project).
People build temporary structures all the time -- event tents, for example. And plenty of people finish a room, redo a bathroom, build a treehouse or a shed, etc, as a side project. And believe me, it takes a high level of expertise to inspect a building. Many of the most important kinds of flaw are only evident to the eye of an expert.
I think it's a good one. Think of commercial real estate. You have an old shopping center that is cash flowing. Its maintenance costs are high, and the rent is low. Once upon a time it made sense for the market and the style was perfect. Now it's out of date and the units are not exactly what people are looking for.
You could tear it down and build a fancy new one that would cash flow even more with higher rents and lower costs. It's exactly what the market needs today. Green energy, nice, open spaces, new paint. Do you break out the bulldozers?
My local zoo actually built a full-size prototype of their new penguin exhibit ahead of building the exhibit itself. Once people could actually walk around in the space, some improvements were obvious.
"The building provides space to temporarily house academic or administrative units that are displaced because of renovations to their home buildings. The facility was designed using a pre-engineered steel frame with its exterior clad in metal panels and precast concrete-like panel accents. The building is designed to be disassembled and recycled in 10 or 15 years.
One floor; 45,000 square feet. Find a public access defibrillator in the hall intersect."
+100 - the biggest questions (to me) were HOW MUCH code did they throw out, and how old was it? Throwing out a few months of code you just finished writing (and which you're still intimately familliar with) is vastly different to throwing out a code-base that would be entering middle-school if it was a person.
I don't know what the old codebase is like, but the current one is just 2145 LOC, not counting tests. 62K total.
I know that startups do way more that just write code, so I'm not criticizing them at all. And it seems they're not afflicted with the typical engineer modesty when it comes to their own codebase, which is a good thing. On the other hand it's not exactly a superhuman hacking achievement to rewrite this from scratch.
Netscape 6 is (or was) the oft quoted example of why not to do this. All that old code may seem old and boring, but it has a huge amount of knowledge, wisdoms, learnings, fixed mistakes, performance improvements, great ideas, deleted bad ideas and security fixes built into it.
When you throw out your code you're throwing out all that knowledge on the assumption that you can build it all back with improvements.
Evolving an application from a known good state to a known better state is usually the best approach unless your codebase is small or your app has no traction.
Rewriting your app from the ground up is like getting divorced and dating a new girlfriend. The first few months are a lot of fun until you figure out how much you lost.
Without Netscape 6, we wouldn't have had Phoenix/Firebird/Firefox, and probably wouldn't have IE7, IE8, IE9 and KHTML/Webkit would have come about very differently and all these would have had much less competition.
Netscape 6, or at least the mere fact that it existed and worked well enough and the fact that it was the first browser ever that did standards well and focused on not much else other than standards, made it the most important browser ever created.
Netscape 6 was the most revolutionary browser project ever created and had the development effort behind Netscape 6 not been done, we would possibly have had a much worse very IE-only web today.
Never mind that beyond all that, Netscape 4 was terrible and deserved to be thrown out.
> Netscape 6 was the most revolutionary browser project ever created
I dont disagree with the general sentiment of your comment. But as someone who attempted to use Netscape 6 at various stages of its development, I fear that statement might be stretching facts a little bit.
A few people have pointed out that it’s easy to throw out your code when you (a) have tests for it, and (b) only have three months invested in it. I agree with both of those points, but want to tip my hat to a team that rewrites their code after three months.
The reason some code bases can’t be rewritten after a few years is because when the code was only three months old, the team passed on the chance to rewrite it while it was still a tractable exercise.
Very much agree. An extra layer on the danger cake (for web apps) is if your user base grows slowly while keeping the business profitable. Suddenly (sic) you hit real capacity limits after n years, and have no idea how the "organically grown" code in question works.
It's so much easier to write from scratch than it is to refactor. So many programmers have no idea how to tackle an existing codebase, even if it's one they've wrote themselves.
Depends on the angle. The person that wrote the codebase is usually the one holding onto the old codebase due to ego. On the other hand, new people that come in and don't like things can be full of ego in the sense of, "I could program this better without any limbs!"
Egotistical engineers always think this time things will be different, because they're smart enough to do it right. The reality is that every rewrite brings a new and different set of problems that need to be addressed. You don't escape problems by scrapping it all and starting over and this is one of the most valuable lessons to learn as an engineer.
You don't need to be egotistical to see that you can do things better the second time around because you've learned stuff along the way. In this case, the writer provided multiple examples of how they were able to use the lessons they'd learned from the old codebase to improve the new one. They did, in fact, escape some of their problems. Did they introduce new problems? I'm sure they did, but they're probably still better off overall.
Sometimes ego is involved, but sometimes rewriting is just the rational thing to do from a cost/benefit standpoint.
Justified or not, I think a project owes it to its users to change major version numbers when a major rewrite (much less a complete one) has occurred.
As perfect as your code might be, customers deserve a hint when major changes were done under the hood. Moving from 0.6 to 0.7 screams "just a few changes, some new features", not "we threw everything out and started over". This should have been made into "1.0beta", I would think.
There are other factors involved when deciding a version number than the code under the hood: API changes, new features, is it production ready, 'this is the now stable API that we'll be maintaining for a long time', etc.
Edit: for perspective, a thoughtful proposition on how to approach software versioning: http://semver.org
I agree that there are many factors. I'm just saying that some changes are important enough to trump anything else; the presence of a rewrite should decide the version all by itself.
sigh; a headline like this is little more than HN linkbait. As others have mentioned, throwing out all your code is generally a bad idea, especially if your project is of significant size. How many lines is NowJS anyway? I doubt it was much of a loss.
I'm still trying to figure out what NowJS does other than expose an RPC call using socket.io. All of the features on the website are actually socket.io features.
Please bear in mind the counter article everyone is citing (http://www.joelonsoftware.com/articles/fog0000000069.html) was written 11 years ago. I'm not saying that it's wrong because it's from 11 years ago, but you really want to consider its central example.
Netscape 6.0 was a complete rewrite. The article claims that this was a mistake.
Now, if you would recall, one of the reasons Netscape 6.0 was completely rewritten was to use Raptor/NGLayout. Raptor/NGLayout was later renamed to Gecko. Gecko is the currently developed Firefox rendering engine, and Netscape 6.0 marked the start of the Mozilla foundation.
I would harshly argue that Gecko would not have taken off nearly as well without Netscape's rewrite. But this is definitely not a story about Gecko's popularity. It's about the reason why Gecko was the decision they took at the time, and the current impact of Gecko.
Besides the fact that Gecko was one of the early open-source success stories, the reason why Netscape decided to scrap everything and start from scratch was simple: Netscape 4.0 was horrid. Remember back to the days of IE vs Netscape. Netscape was significantly slower, and had no dynamic HTML among many other features. It was buggy and would crash frequently. And the development on the "continuation" of 4.0 code was simply too slow to compete, because of outdated, not modulated code, and stuff stuck in that simply "worked," and never changed to allow for future changes.
Now, people attribute Netscape 6 to Netscape's death because of this radical decision. But take a look at Netscape's usage statistics: http://upload.wikimedia.org/wikipedia/en/1/16/Netscape_Navig... . Hey, guess when Netscape 4.0 came out and IE started to dominate? 1996-1997. By 2000, when Netscape 6 was released, Netscape's usage already dwindled down to nothing.
Now, you're thinking, why didn't Gecko save Netscape? It takes too long to reverse these tides. Netscape was already dead. AOL acquired Netscape in 1998, to combat IE. Then, in 2003, AOL won an antitrust suit against Microsoft, and allowed AOL to use IE royalty-free. Netscape was pretty much scrapped at that moment because the damage had been done.
But wait.
Mozilla released Phoenix in 2004, based on Gecko. You should now the story from there. (Clue: Phoenix -> Firefox)
Disclaimer: I'm not saying that Firefox > IE or Chrome or whatever. I'm saying that the codebase Netscape 4.0 was based on was doomed to fail. Gecko still lives on today, and was started as a rewrite of Netscape.
Anyway, my point is, there are times to rewrite things, and there are times to keep them the same. If you have a good, legitimate reason to rewrite, as Netscape or NowJS did, you should. If you are just the stubborn programmer that always implements his own Queue or QuickSort when libraries exist, don't.
Throwing away Netscape 4 and rewriting it from scratch wasn't a bad decision because nothing good ever came of it; it was a bad decision because it killed the company. The shattered remnant of Netscape that hobbled across the finish line to release "Netscape 6" wasn't even Netscape anymore, it was a rapidly fading division of AOL, IIRC, that just stamped its name on a Mozilla product it had little to do with producing at that point, to widespread derision.
As bad as Netscape 4/5 may have been, while I can't guarantee trying to refactor instead of rewrite would have gone better, I can guarantee it couldn't have had any worse a result than the Total Death that was the actual result.
And the phoenix-return of Mozilla is the exception, not the rule.
In support of your comment, I think some are forgetting how horrible Netscape 6 actually was to use. (Or am I just getting old? It wasn't that long ago!)
Compared to earlier versions it was incredibly slow—even by the standards of the time—and crashed a bit more frequently. It took much longer to load or perform some common tasks. Sure, the rendering engine was better for many pages, but browser sniffers were still common, and (like now) weren't updated often enough, so bad versions of sites got served occasionally too.
There were a host of reasons why Netscape 6 was bad from a user's perspective. Spolsky has it right when he says Netscape 6 was a piece of junk. Let's not allow time passing to permit us to remember it more fondly.
Sure, future versions got better, but that was due to he creation of a new "corporate" memory. It took a while to get over having discarded so much learning.
To be pedantic, there is at least one fate worse than the Total Death we observed: Total Death + no Gecko.
Even though ths external case is stronger, this holds true even from the company's perspective. AOL continued to release Gecko-derived browsers through 2007. They got something from that (even if it was just AIM installs and propping up the fading Netscape brand for other purposes).
Holding onto code just because is a foolish notion.
Sometimes approaching things with a fresh perspective is what's best - it allows you to apply what you've learned so far and actually improve not just one section of code, but the whole kit & caboodle.
This is one of the reasons you should write tests as you develop - it ensures you have a consistent baseline you can apply.
As a developer you are constantly learning & evolving.
If you're not confident enough to draw a line under what you've done, step back and evaluate your position and rewrite it when needed - that's fine, but others find it an effective approach to building something bigger, better and more sustainable.
I think that this applies to the technology you're using as well.
I've been thinking about this because I'm writing an application in Rails and realized relatively recently that it would be better suited as a Node application. My motivation for using Rails was that it was what I knew the best. Luckily for me, the most time-consuming part was the front-end, not the Rails part, so I'm hoping my switch to Node will be relatively painless.
Well, just because some prominent luminary says it shouldn't be done never means it's always a bad idea. A lot of the basis against rewrites are for relatively mature projects still on a high growth trajectory or large installation base. However, there are exceptions to that as well. Foursquare went from LAMP to Scala/Lift, PostgreSQL, and MongoDB with minimal complications and it's paid off handsomely to support their amazing growth.
Also, the money quote: "It's managed to survive our vicious benchmarking tools and our ridiculously comprehensive test cases. " My assumption is they are using the same languages and tool sets (not porting from language X to Y).
I'm curious as to how long the rewrite took though.
Rewrites are always fun and I applaud any man brave enough to undertake such an effort.
I've often been faced with a situation warranting a rewrite, but I never mustered the courage to actually go with a full rewrite I don't think. Gradual rewrites are much more my thing.
I've been really really tempted to do one, but sadly the pressures of the job take precedence and so it usually ends up being the case that you stick with the code you've got and try and hack extra things in, knowing full well you really don't want to and aren't at all happy with it.
So my 'rewrites' sometimes come in the form of superficial organisation. Looks moderately nicer, doesn't work any better.
I guess it is courage though. You don't really know that your rewrite will be an actual improvement, or if you'll even pull it off. Brave move indeed.
Once you start making revenues from your code, a complete rewriting is probably the worst thing you can do for your company. Sometimes it might be warranted, but customers care about solving problems, not about technology.
Chances are if you wrote properly modularized code, you can rewrite modules one by one, which is perfectly legit. But just dropping everything and trying to rewrite a product from scratch is a mistake that a company usually only makes once (before they die).
Especially when getting into a codebase that is not yours you are tempted to think "why dont i just rewrite this in a better an cleaner way". Then you go on todo that and fall into most of the pitfalls the original author fell in before, have to rethink every idea behind and it usually is alot more work than you think.
That said it depends heavily on the size of the project and a bunch of other things, for a 3 month old project it might make sense.
I'm glad they're happy with the rewrite, but honestly, this makes me feel a bit nervous about updating my Node app utilizing NowJS to the 0.7 release.
Maybe it is better. Maybe it is more stable, smaller, and faster, and well-tested. But it hasn't had as much "real world" use thrown at it as the previous codebase had.
I'm not saying I won't upgrade, but I'll likely wait a bit until I do and stay with what I know works right now, at least in production.
Frequently, throwing out "all your code" is a sign you did pretty much everything wrong to begin with. Personally, the sign of a decent design is when you can wind up doing the old "Grandpa's Mattock" trick:
"This is grandpa's mattock; my father replaced the handle and I replaced the blade".
I'd be dubious that they actually rewrote everything - having working stuff lying around that already worked, some of which was probably just fine, I'd hope it got copy-pasted into the new code...
Whether their decision was right or wrong, they made a decision that big companies tend not to make. Startups should probably have a bias towards decisions like that.
An attention grabbing headline but all the same I have been using now.js and its been a great tool (although a cut down on the print statements would be nice).
It's stupid to argue, "You should never tear down a building" (like http://www.joelonsoftware.com/articles/fog0000000069.html implies) and it's also kinda stupid to write an article entitled, "Why We Tore Down Our Building (And Why You Should Too)"
Neither is universally right. The right thing to do in a given situation is a difficult and complicated question to answer, and it depends on the building in question: how big it is, how well it was designed, how well it's been maintained, etc.