I've had occasion to think a lot about the relationship of quality to long term productivity. The remarkable thing about building software quickly is that if you focus on speed, cutting quality where needed to "move forward", you end up slowing down after a while, due to the build-up of technical debt. A continued focus on speed will typically result in slower development, and a low-quality product, a sort of worst of both worlds scenario.
On the other hand, if you focus on quality, you end up making choices that enable fast development. You make sure you have high-quality requirements and a deep business domain insight in the whole team, which means you end up writing only that code which you truly need, and write it in the right order (most important features first). You make sure you have code that is well-covered by tests, which as a happy side-effect makes it easy to adapt to new requirements. You focus on having a high-quality deployment process, so the amount of customer issues that result from a broken deployment go down reducing the load on your dev and support teams.
Focusing on quality first is not about gold-plating code, it's about making conscious decisions to have a high-quality process from front to back, and those decisions typically result in a highly efficient organization. It's no accident that those teams which manage to ship releases every few weeks (e.g. firefox, chrome, facebook) have world-class products. You have to focus on quality to go fast.
I agree with this. I think sometimes it's hard to keep the balance between the developers who want to 'do things right' but spend far too much time designing a perfect solution, as opposed to many coders who rush to 'get something done' and build systems which quickly become impossible to maintain.
I think after about 5-6 years of coding was when I came to the realization that everything you build 'the best way' will become awful coding practice in a few short years. But that writing things at blazing speed just to hack it together and get to work invariably leads to code which is written to work a single way and is a nightmare to maintain.
I try to build small, flexible solutions which are as adaptable and maintainable as I can, with the awareness that whatever I build will be looked as as archaic in a few years, but it might still need to be maintained and upgraded.
So, slow down a little, think a little, code a little slower, but don't spend 6 months planning the perfect code weapon either.
> everything you build 'the best way' will become awful coding practice in a few short years
Disagree.
Good architecture is almost timeless. It isn't always en vogue, but it is recognizable by being pleasant to maintain, conceptually cohesive, and simple. It usually resembles a domain model, with the UI, persistence, and everything else being built around it.
It is a force at least as strong as the language used.
As long as nothing changes, I can agree with you. But that's the point of architecture in the first place, to be there for you when things change. The focus of architects should be to ensure that changes are as easy as possible and not to have an elegant design. I've seen some of the most elegant designs that are easy to maintain trashed due to changes in user requirements or changes to the market place.
For this reason I advocate, and have advocated since I started developing in the early 90s, micro service/modular architectures. Java OSGi is currently my favorite environment (though I'm also investigating similar approaches in Clojure), but I've done this with C/C++ in the past as well using shared objects/dlls as the modularity mechanism.
Good architectures adapt to change, they doesn't stay the same with time.
Elegance doesn't matter; simplicity does in the face of changing requirements. Or rather, elegance falls out of this simplicity. It's an emergent property, rather than an explicit aim. Thus, my favorite designs have traditionally resembled a microkernel in approach, where the kernel of the app exists only to facilitate disparate services to find and use one another. Interfaces are kept to an absolute minimum, and concerns like UIs or persistence are pushed to the edge of the system, where they should be. This design facilitates division of labor well; more novice developers can make a mess within their subsystem, yet have the damage contained. I usually bolt a pub/sub mechanism on relatively quickly, as this requirement is almost a given with a UI.
Dynamic loading is indeed the ultimate modularity. Combined with this microkernel approach, extensions are just as powerful as built-in services. The only problem is you enter a new hell: managing dependencies/versions of the core vs the extensions. If you can control the release of the extensions as well as the core, you're set.
Never dismiss the dangers of over-engineering. While not as terrifying as under-engineering, how useful is a feature that exists only for the purity of design?
Over-engineering is a problem because it usually results in a much more complicated design than you really need and wastes a lot of time.
However, over-engineering and spending quality time thinking about the design and domain are two separate things and typically the more you think about the design and domain the less likely you are to over-engineer.
And Over-Engineering is something people only learn to spot once they've designed and built a few systems and had to maintain them for a few years. Its fairly easy to teach design-principle-du-jour but quite hard to teach about over engineering, KISS and YAGNI.
Developers tend to want to generalise everything because it makes it more interesting and has the appearance of good design. But generalising in one direction reduces flexibility in other directions. Better to copy and paste a bit at first and wait a bit before adding that extra extraction layer that generalises Foo.
I think a lot of it comes down to developers focusing on their own personal learning and development and not what is best for the current project.
Very few workplaces will let you work hard for 5 hours solving problems in the most time effective way and then spend the next 3 hours on your personal development.
On the other hand you can always spread the work over 8 hours and "experiment" with "on the job learning" leaving all sorts of sub-optimal solutions in the project. Then the next guy goes "Why on earth did Jim use X here?" and the truthful answer is because Jim wanted to learn more about X.
What we're looking for in design is simpler. Simpler becomes conceptually harder very quickly, because it means challenging a lot of assumptions and opening up some unknowns.
The tried-and-true implementation has a known initial cost and cost scaling factor as it's repeated throughout an application, while an experimental (and overdesigned) one is extremely sensitive to context and could become a net negative if it fails.
When we talk about "technical debt," I think we're describing something that isn't an experimental design, it's just "below" the known standard - it's faster to implement, but it cannot scale, and we already know that. Conversely, a standard practice is one that works and doesn't usually get pushed to its tolerance limits.
The corollary of this is that for any engineering project taking on a big enough problem, you have to assume some technical debt to ground the system in a place where each module can be tested as part of a whole: otherwise your work will go too deep into the experimental/overdesigned zone before it ever completely functions.
Yeah, in my experience, "no engineering" is the most abundant; everything is thrown together willy-nilly to code as fast as possible. I have come across very few real cases of over engineering.
Part of my point is that you don't feel over-engineering nearly as much. If it is under-engineered, it probably doesn't work, and at least has a list of pain points that are fairly obvious.
Over-engineered components will work (sometimes flawlessly), so you don't realize you've spent too much time and money on something until long after you built it.
Yes, never dismiss it. In fact a lot of the time over-engineering is just another form of technical debt, but it also has an upfront cost as well. At least the under engineered solution is cheap up front.
I would much rather come across something under engineered than over engineered.
So, at one time, relational databases were considered 'new'. Entire programs and the data they used were stored in a single file by most development teams.
It was considered good architecture to keep everything together. Many established and well-respected developers argued against relational data, because they said 'it splinters all of the data' and modular programming 'splits up the application logic all over God-knows-where'.
Some people still argue if Linux and it's monolithic kernel is better or worse than Windows modular design. These are two completely different architectural schools, and yet both have people who consider one good design, and the other flawed beyond repair. It doesn't matter which, take your pick, they both have zealots and detractors.
So, saying 'good architecture is timeless' is kind of general for me.
Things you consider to be absolute today will change and be defunct in five years.
If you're into small modules, stay consistent and separate things. If you're into OO-design, try to encapsulate and make your objects re-usable.
When you start losing this consistency and drift away from the architectural vision in order to make quick-wins ("lets just tuck this feature to this module even though it might not really belong there..."), that's when things turn into mud.
Understanding the architecture and its philosophy gives you a big insight into the code, the way its structured, where things fit together. It saves you time.
With solid architecture you know where to change things and where to add them. I think it's timeless in this sense. Its consistency applies everywhere. Once you 'get it', and stick to it, things make sense.
This has been my experience as well. There is only one speed, and that is doing things well.
I'm convinced a lot of developers have been completely brainwashed by the Business in believing they don't need to ship good code. They absolutely do, and the cost of it is high. The cost of maintaining bad code is much, much higher, however. Abdicating this responsibility reflects poorly on you, the developer.
Much of the populism in software targets the insecurity of those who don't push themselves to ship great code: "getting things right is so hard! Just do the best you can and don't sweat those edge cases too much!" I sweat the edge cases when I'm writing the code in the first place because I'm incredibly lazy: I know the mental cost of re-creating all the context necessary to solve the problem later is very high, and I'd rather not return to the feature ever again.
"A week of coding saved me 30 minutes of planning."
That's one of my favorite sayings.
>I sweat the edge cases when I'm writing the code in the first place because I'm incredibly lazy
I like to front-load any problem solving and make sure I understand what I am getting myself into. What that looks like in practice is that, when designing architecture, I will write up a list of every scenario I can think of (within reason of course, also including future plans), and verbally test each scenario against my ideas. Essentially I try to build up the largest image I can of all the moving parts, and then narrow the scope back down to the present problem. Initially expanding the scope to include edge cases and potentially fictitious scenarios not only helps me isolate concerns, but I consequently understand the limitations of my implementation (which of those scenarios failed my verbal testing) and I have my notes for when/if I return. It's how I sate my natural desire to be overly-comprehensive without actually over-engineering everything; I am confident that what I build today is flexible and extensible enough to grow as it needs to. Sometimes I open a file I haven't opened in ages and I read a comment saying, "You're welcome to my future self", and I can't help but be thankful.
There is a lot of truth to what you're saying, but while it sounds great for software that is built for its own sake, it sounds too uncompromising in most business settings. The fact is that most software is built in service to a business and the developer is (quite rightly) judged by the extent to which the software serves the business well, rather than the extent to which the code behind it is "good" or "bad". Ignoring that and attaching your identity to that quality of the code rather than the service to the business is folly. Having said that, it is all a short-term / long-term trade-off; "bad" code can be expedient and beneficial short-term while "good" code is almost always better in the long term. Deciding when to serve short-term and when to serve long-term business goals is the Hard Part, but the answer isn't to stubbornly apply one rule or another.
This is a giant mess waiting to happen. Eventually the pace of development will slow to a crawl as incoherence of the technical decisions made build up. Then when the developers leave, forcing the next guy to work to understand now-legacy projects, all of a sudden the velocity the company has heretofore enjoyed will be gone.
It has been this way for 10 years and predates me working there. The pace of development remains unchanged as far as I can tell over the time I've worked there, regardless of whom left and was hired.
I was pointing this out not to say it was a good thing but rather that the premise of [Quality == Speed] is flawed.
It sounds like releasing daily is part of your testing strategy in that you simply let your customers do the testing for you.
Making small incremental changes with extensive manual testing can work. It just looks like you don't have to pay for the test effort.
I've seen it a fair bit with internal projects at larger companies but most external customers I have met in my career would quickly switch products.
Edit: It also sounds like you are confusing best practices with good code. I've met teams, usually without a formal cs education, who haven't used version control and didn't have any unit tests, but still had a well designed codebase. Of course best practices are best practices for a reason....
> Edit: It also sounds like you are confusing best practices with good code. I've met teams, usually without a formal cs education, who haven't used version control and didn't have any unit tests, but still had a well designed codebase. Of course best practices are best practices for a reason....
1) They both have degrees in comp sci.
2) For very narrow definitions of the word "good" you would be correct. Its technically functional and reliable. However, it regularly has problems with security and/or changes to the common library breaking other people's services.
The critical thing is that the architecture remain coherent. Modules have to have clear responsibilities and not get cluttered with special-case hacks that really belong somewhere else. If you keep things reasonably clean that way, I think you can survive being sloppy about other recommended practices... barely, and with effort, but you can survive.
But if no one can understand how the system works and predict the effects of changes, the thing is doomed.
Question: What is on-boarding new employees like with that system? How long does it take you to get people up to speed and being productive? I would suppose the lack of rules makes them fast in the beginning and in the end they are contributing to the mess in their own special way, doubling down on the technical debt.
I can't really answer that question completely accurately. I was a very odd case/hire compared to those both before and after as far as I can tell.
I mostly interface with outside developers for integration purposes and I never really had a normal on boarding process.
I'm just a random developer with no authority/power.
That said, my best guess is:
> 1) What is on-boarding new employees like with that system?
They get an intranet project that is relatively harmless and helps them learn the common library -> they get assigned projects on the main site and get progressively larger features to implement. There is no real documentation beyond the actual codebase and asking people questions.
> 2) How long does it take you to get people up to speed and being productive?
No clue. I'd guess 2 months based on how quickly I switched from "intranet projects" to integrating Amazon MWS, 3rd parties, etc.
> 3) I would suppose the lack of rules makes them fast in the beginning and in the end they are contributing to the mess in their own special way, doubling down on the technical debt.
More or less, yes. An example would be we now have 3 partial versions of the same intranet application. The super old one, the one I built before handing it off to the new guy, and the new guy's version [because he didn't like my version].
> I've had occasion to think a lot about the relationship of quality to long term productivity. The remarkable thing about building software quickly is that if you focus on speed, cutting quality where needed to "move forward", you end up slowing down after a while, due to the build-up of technical debt. A continued focus on speed will typically result in slower development, and a low-quality product, a sort of worst of both worlds scenario.
We're a typical "launch in 3 months!" start-up that is now 7 years old, and we've were so focused on growth and feature development to meet the market that technical debt just accumulated and accumulated.
Eventually, technical debt began to have a real impact on business feature delivery, so we talked about it with our business team and came to an agreement that 30% of development time was ours to reduce this major risk.
The end result is that over time, as quality improved, our business feature delivery has gotten better and faster - less bugs, and more features per sprint. We're very lucky to have our business team, I think.
Very well said. It's a shame so many project managers (and coders!) don't understand that any sufficiently sized project is a million times better off when investing more time at the beginning to lay a solid foundation. I like that more and more people are beginning to understand nowadays.
It's possible to minimise the technical debt you take on (without spending the full amount up front) by being smart about your application's design/architecture.
Using the user roles example given in the article, it's not beyond the wit of man to foresee that you're going to need user roles at some point in the future, so you can make provisions for adding it in later. Done right, the extra effort is minimal but means that you'll be able to plug in the new functionality at a later date without needing to refactor the entire codebase.
It might be as simple as figuring out where the permissions/role check will go and including a "# Future permissions check will go here" comment. Or you might actually include a check that calls a dummy service/function which always responds to a "Can this $user do $action to $resource?" query with "Yes". Then, in a few months, you can simply replace the dummy service/function with a real one.
You don't even have to design/architect the whole thing up front. You just need to avoid painting yourself into a corner. Ask yourself "What am I likely to need in the future and can I do something cheap now that will save me a lot of time/effort further down the line?"
If you ever need to explain this concept to a non-technical manager, tell them that it's like applying a kind of net present value calculation to development time/resources: Spending a little extra now means you can potentially save a lot further down the line.
> It might be as simple as figuring out where the permissions/role check will go and including a "# Future permissions check will go here" comment.
Very good point. I was thinking about this issue not too long ago. And the ability to do that, comes with experience and often separates normal programmers from much better programmers.
It also takes a while to detect this, because there needs to be enough "Futures" that become present to tell if the original architect predicted or foresaw or did they just over-abstract or and overloaded the code with potential apis or features.
More often than not I see the later. "This code never runs","why is this here","we have 4 levels of abstraction for this, why?". "Oh because I thought we'd have this and that in the future so I added in".
Quite rarely I see the former. When a developer is asked "can we add this?" they come back and say "yes easily, I knew you'd come and ask for this so I added the abstraction to handle it or add stuff to it later". The one thing I noticed is that this becomes a pattern for people. Some people create the former type software some the later.
Exactly. I really like writing future focused interfaces like that. They're light weight, and can save a lot of time later. It's like building good credit to get a lower interest rate on your technical debt.
I don't understand why bad code is "not technical debt". It seems to me technical debt, poorly spent. Someone who takes a payday loan to buy beer is still incurring a debt.
If there is some more meaningful distinction here that I'm missing, can someone clarify?
'Technical Debt' is a case where a specific thing was named before the general concept. Another case is the phrase 'mock object'. In the early 2000s, the original mock objects paper coined the term and defined them as testing stubs with builtin expectations. For years, people used the term "mock object" for any sort of stub.
Technical Debt is a very specific metaphor that Ward Cunningham framed as a strategy. Ever since, though, people have used the phrase for all sorts of entropy in software. If you look around the literature, you'll see people talking about type 1 and type 2 technical debt.
In my experience (and I'm the guy who wrote the Legacy Code book mentioned in the article), the typical entropy thing that we see in software has more to do with just taking the easy way and avoiding refactoring - different from Ward's concept.
That was my thought too. I guess OP was trying to distinguish between debt deliberately incurred vs wasted.
But maybe a better way to think about it is "leverage" vs "debt". Leverage is something you use to improve the product with a known and (hopefully) quantifiable return, whereas plain old debt represents those poor non-decisions.
If the bad code is still tested, it can easily be changed to be "good code". If you have good code, but untested, you have no way of refactoring the code without actually knowing that you making it better.
Bad code with unit tests cannot easily be changed to "good code". Most unit tests of bad code are very brittle. Consider changing the permissions system in the article: All a typical rushed set of unit tests would do in this case is form a brittle specification.
In fact in my career I've never seen bad code + good tests unless the tests were added at a later date (typically by another developer trying to do what they can to stop code rot).
On the other hand it is a joy and straightforward to add unit tests to good code and typically they are much less brittle.
To me, all code is debt so long as it requires maintenance. It will be read and modified hundred of times after being initially written hence why you want to have as few as possible. Code also need to be hosted on servers somewhere and that's a sort of debt too since you now are forced to maintain the infrastructure, keep the software updated, ensure it's always running. However, you have to accept a certain amount of debt otherwise you will never be able to launch anything. Everytime you can invest some time to reduce maintenance later down the road, it's like getting a lower rate on your debt. However no matter how hard you'll try, if the project is used it'll require love, time and hard work to keep it going. Then comes a time where either the project fails and you take it down (and stop paying any debt) or, if you're lucky, the balance between ongoing cost and benefits start to even out and you're finally out of the woods.
While a good post, the real issue is that non technical managers of technical projects is quickly becoming more of a risk than many projects can prudently accept.
The old, and unfair, characterization of programmers as smelly trolls that can't handle client interactions is more of a crutch for fools than a reality.
The truth is my clients complain bitterly about how they'd prefer to just talk directly to the dev's and not be bothered by the various hand waivers trying to justify their line item on an invoice. Often this is a great way to annoy the dev team, but it's a good expression of the perceived value of a non technical intermediary.
In the end the old truism of "You can't manage what you don't understand" still applies to this new fangled industry of software development.
That being said, ramping dev's up on management techniques is another challenge, for another internet rant.
I don't see why the analogy to debt is necessary when the direct analogy is to capital underinvestment: in fact what we're talking about is or very nearly is just that, capital underinvestment, not simply something analogous to it. And the hypothetical businessy, non-technical types whom the debt analogy is intended to enlighten understand very clearly what deferring capital investment is and what the likely consequences are, at least if they actually know anything much about business.
Most of the people who are pushing for "features now, someone else's problem later" are probably not upper management but rather middle management. Upper management might understand capital underinvestment. Capital underinvestment is very nearly the sole job middle management is there for. And berating people for not being done yet. At least at a couple of the places I've worked.
I like the concept of Minimum Lovable Product, although MLP isn't as catchy as MVP (thanks to sports). It's probably good to think of an MVP as a step on the way to an MLP - the MVP is the one you release to specific audiences of likely users or show to angels or seed investors to show that it can work, and be very explicit about it not even being a beta version - more like a version 0.1.
>Thousands of startups have launched and failed precisely for the lack of quality.
That was an emphatic statement in the article and I feel the discussion would really be constructive if the author (or posters in this thread) listed concrete examples of startups that failed primarily because of software quality.
If success has many fathers, failures are not orphans and also have many fathers as well. Perhaps software quality was not the 1st but actually the 2nd or 3rd reason why the company failed. I'm not saying such examples of software execution killing the company don't exist. I just can't think of any prominent examples at the moment.
I'm leaving out poor software examples such as the Atari 2600 E.T Extra Terrestrial game cartridge because that was not a startup situation.[1]
On the other hand, an internet company like Friendster looks like an interesting case study. Their infrastructure software was not efficiently traversing their social graph. It was something Facebook programmers conquered but Friendster did not. But was software quality the #1 reason behind Friendster's demise? Apparently, the board-of-directors didn't prioritize and throw money at the engineering challenges that Friendster was facing.[2] So the their failure can be reinterpreted as a combination of management of priorities and software execution. Maybe if they did throw money at the issue, it wouldn't have mattered because their programming staff didn't have the unique competencies to tackle at-scale social graph problems. In that case, it's not a "technical debt" issue but a "hiring talent" issue.
Another case study might be the Chandler software (chronicled in the book "Dreaming in Code"). However, it appears that the product got bogged down by architecture astronauts unable to make decisions and not "technical debt".
If there are thousands of companies that failed on software quality, it'd be interesting to see which ones can be disentangled from non-software issues and we can confidently say, "yep, that technical debt is what killed them."
(My curiosity for real examples does not mean I'm endorsing technical debt. I'd just like to get some real-world perspective on this topic.)
Here's the analogy I made up a bit ago trying to explain the concept.
As programmers We build code paths. Design mazes. In the beginning the maze is simple, there's only a few paths. To make things simple we separate our ideas into many small mazes. Since they're small, it takes a short amount of time and concentration to predict the different possible paths in our heads.
Technical debt is when, due to to time constraints the decision is made that it is quicker to draw one more line on an existing maze, rather than create a whole new small maze. In the beginning it seems not so bad, you can still take in the whole maze at once. But as time goes on, that one maze gets larger and larger, and finding a successful path through it takes longer, and longer. Eventually you get to the point where you can't separate the maze because too many of the paths are built on top of each other, but you also can't manage the maze because its to large itself. That's the default. That's the rewrite.
This isn't a 'primer' about Technical debt - it's an opinionated piece about the value of the concept of technical debt, which I think is at least a 102 or possibly 201 level course on the subject, and needs to be taken in the context of other material. I suspect it hasn't been tested out on many nontechnical managers, either - using the concept of electron spin (for a nontechnical audience, remember) as an analogy for the concept of analogies is the kind of meta-analysis that I think will lose them pretty early on.
I think the real problem of this piece is that it flatters nontechnical managers, as businesspeople, by describing technical debt as a sensible business decision they can get into consciously through clearheaded negotiation with their technical team; then characterizes 'bad code' as something different, that happens when the technical team just don't do their job. This alleviates the nontechnical manager of any responsibility for bad code. The fact is that knowing about technical debt is not necessary to find yourself up to your neck in it. A team which goes for years taking shortcuts, being forced to hit unrealistic deadlines, or which lacks the skills to really build a good, robust system, is like a kid with a credit card who doesn't get that just paying the minimum payment each month isn't going to work forever. Every shortcut they get asked to take, they swipe the card, and they get to walk out of the store with the goods, and it seems to just keep on working; thousands of dollars worth of stuff, and they just have to pay a few hundred each month. It's magic!
But the consequences will show up in the end, either in the form of a long period of hard graft to pay it down, or declaring bankruptcy and abandoning the codebase.
That's why I love the technical debt analogy - different people manage financial debt differently, and most businesspeople understand the concept that planned debt is necessary and beneficial for growth; arguing that Ward Cunningham and the author don't think you should call other technical debt by that name seems petty, and not particularly useful, when extending the analogy to runaway debts works so well.
Overall the problem I really have with this piece is that if I tell a manager "This project is carrying a bunch of technical debt," now they'll google it, find this, and come back to me with "You mean it's a pile of bad code." Which is true, but how does that help us? When I make the argument that a troubled project needs to pay down some technical debt, I'm trying to make clear that we need to start paying off more than just the minimum payment every month. If the technical debt analogy is removed from the table, I'm left with just saying the team needs to replace the bad code with good code. And then they'll tell me that this piece also told them that the 'big rewrite' was a bad idea too... So how has this helped?
I am financially astute, and I've been working for small and large financial services firms.
When someone talks about taking on debt, it is critical to know the interest rate charged. Also very important are whether the debt is collateralized (example: car loan, house loan) or non-collateralized (credit card), along with how easily it would be to get out from under the debt if you cannot pay (can it be discharged in bankruptcy? how does it affect your credit rating?)
Very few (if any) financial people and senior managers could make the company liable to replay this debt unless these variables were known with relative certainty. To take on debt without this information could be deemed reckless and may expose the company to shareholder lawsuits, etc, and be job-ending moves for the people involved.
Back to technical debt? What kind of debt is it? Is it collateralized? What if you can't pay it? Can you write it off, or do you lose the company? Can you get out of it in bankruptcy? What's the interest rate?
I argue that "technical debt" is like an unknown debt. The interest rate is unknown, it is collateralized with the organization itself, and cannot be discharged in bankruptcy, short of bankrupting the entire company.
The prudent and reasonable thing to do would be to not take on the debt at all. Remember what Toyoda (the founder of Toyota used to say): Cash is your best friend, but debt is your worse enemy.
Who would borrow money to go to college and get an education at 40% interest rate a year? It would only be worth it in very rare cases. What about 20% a year? My wife put her college education on credit cards, paying foreign tuition rates, and we had fun paying it back, let me tell you. When I went to college, I went slower, paying as I went, but had no debt at the end.
If any company takes on debt, it is irresponsible for them to not do a thorough financial analysis of the impact of the debt.
What's the interest rate on technical debt? Who can say?
I'm not really a fan of the story-point example. The article begins with:
> it’s something that you get when you delay design decisions for later
But then presents us with a problem where we can "spend" 21 story points now, knowing we won't have to do much refactoring later. How do we know this? I thought the whole point was we couldn't see enough of the picture to make a stable design.
Instead of talking about how patches, hacks, and workarounds impact code, I'd like to see discussion on how you move forward when you know the requirements are going to change, but don't know how.
If you implement a robust permissioning system that has permissions related to group membership and one-off permissions related to a single user then MOST of your problems are solved. Nearly all new permission requirements can then be made without touching the code at all, or in more than a couple of places.
The idea is to choose a new design that scales better. If you have M permissions and N permission checks then each new requirement might cause M * N changes to the codebase.
If you have a reasonable permission system you might end up having to make N changes to the codebase (worst-case) or perhaps only M modifications to permissions or perhaps both. But in that case it's M + N changes, rather than M * N. For any values of M and N greater than 2 (and very nearly all of them will be) M + N is smaller than M * N.
The issue I deal with isn't in choosing a design that scales better, it is uncertainty in how things will scale. Say I need to handle group permissions depending on the context. I could hard-code the permissions changes for the one group that needs it now, or I could abstract and generalize context-switching behavior into its own component, ensuring changes to future groups will be easy and low-cost.
But, what if I never get a second group that needs this? While hard-coding values was an ugly shortcut, I may never have a need to revisit the code. If I had gone with a generalized solution, I'd still have to contend with maintaining all that code.
If you ask someone who knows about the business rules if something could EVER happen and they say "no" they're probably wrong, but not maliciously so. Be cautious when taking their word for something. Even if the CEO signs off on the idea of "such and such never needs to be able to do x, y or z" the second a big contract does need it, it all goes out the window.
As a result whenever possible choose solutions that give you SOME flexibility without necessarily creating a supremely robust, exhaustive module.
In this particular case, once you realize that you're going to have more in the way of permissions other than global can/can't do, it's time to be smart. Being smart might only mean creating a function which takes some context like userid and "permissions token" and/or the URL that the person is visiting and then doing a hardcoded permissions check for say one of a dozen userids who are basically root versus everyone else who isn't.
Yes you didn't make a giant scalable system of awesomeness including a web-based permission editor. But you did make a function you can insert every time a new permission item gets created and eventually a system could be created for storing those in the database such that users can eventually be mapped to permissions also in the database.
So if you want to hard-code those values into a single page, which then grows into two pages, which then grows into four pages which then grows into 29 pages that's the wrong solution. But if you want to hardcode those into a function which you then call, that's a good enough solution for now that doesn't hobble you later. Putting the function call in adds very minimal overhead in terms of execution and ranges from slightly negative to moderately positive on maintainability.
Parts of software development are similar to being a speculator in a financial market. You need to have a fairly intuitive grasp of what's going to be a "good bet" for spending a little more time now which has the potential to pay off big in the future.
"I'd like to see discussion on how you move forward when you know the requirements are going to change, but don't know how"
I've found requirements do not change nearly as often as you would think. Only maybe 10% of the changes I make come from a requirements change.
Take for instance the articles permissions example. Is that a change in requirements or a lack of understanding the requirements and domain?
Sometimes it is a genuine requirements change. Maybe the company has recently adopted a new security policy, or maybe you are selling to a new customer and they require permissions even though none of your existing customers do etc. More often than not though the requirement has always existed but the dev team didn't know about it or didn't have time to deal with it.
In my line of work, either I'm building a new component, or I'm dealing with requirements change. So dealing with the shifting sands of what the system should do is a major concern.
Requirements come and go for me. And losing requirements can be just as disruptive as getting them. Worried about how component X will handle users? Well, we don't ship component X anymore, so any refactoring work you've done is wasted.
I might be the odd one out with requirements changing all the time, but I don't get the choice between good design and fast design. I have to creep forward in my architecture, like I'm fumbling around in the dark, because what my system should do today may not hold until tomorrow.
"In my line of work, either I'm building a new component, or I'm dealing with requirements change. So dealing with the shifting sands of what the system should do is a major concern."
And that is simply the case with certain lines of work.
To help me determine which requirements changes are simply a failure in the requirements process I ask myself:
"If I had shipped feature X a month ago would it have had business value?"
If the answer is no then I'm likely dealing with a legitimate requirements change.
For example:
If I worked in the tax industry and the government changes tax laws then shipping these changes months before would not deliver any business value.
Or if I have to make a requirements change because of an external dependency changing. No extra value.
Sounds like you have a good idea of what I'm on about. Still leaves me searching for a way to deal with my system.
Take your tax-software example. You need to start work before the law is written, but there could be changes in the law that require a rework of business logic. What do you work on? How do you prioritize? And how do you deal with that subsystem that no longer has a reason to exist?
In my domain, technical debt is not really a choice I make. It is something that happens as the system ages. I wish I could get someone to speak to that.
In the case of the tax software there's a big question I have:
Do you have a rules engine already or not?
If you're hodge-podging it all together and all the rules are implicitly defined by the code as it exists you're in a bad spot. Knowing that the rules will change every year your goal isn't to grind out this year's changes really hard for six months (and then even harder for another moth to change the final changes) and then slack off for five months.
Instead you should implement a system such that the specifics of the changes are captured in the rules. Once the new drafts come out you need to look for what all the possible new rule categories are, what additional data needs to be captured, etc. But then once things are finalized it's a matter of tweaking the rules slightly to capture the final wording of the bill.
This may not be something that applies to you, though. What area do you work in? How does the debt pile up without choices being made?
I work in government contracts, so your example of complying with the tax code was oddly relevant.
There are plenty of places that follow along your example: I can build a general solution that will withstand a wide range of changes. This doesn't change the fact that a change could come along that I didn't anticipate. What I would call "debt" is more the result of changes in requirements than choices the team made.
You can still frame dealing with change as classic "technical debt", except the full cost/benefit of each design can't be well known. Instead it looks more like "technical investment"; you try make a design that will pay off, but you don't have a guarantee that it will.
I find myself hedging a lot of my designs, and rarely am I able to justify spending a lot of time up front on any architecture.
"I thought the whole point was we couldn't see enough of the picture to make a stable design."
While still technical debt it is not the type the article is talking about or using in the example.
FTA:
"At this point the developers realize that the code is starting to get messy and the solution is refactoring it to have a decent permission system."
I.e. You are at the point where you realise you need a better design and know the rough solution. Do you keep putting band aids on your solution or do you properly refactor?
I think analogies generally suck. Better to analyze things for what they are. Analogies are good for explaining difficult to understand concepts, but all to often people say, x is analogous with y, y has something good/bad about it, therefore x has something good/bad about it.
Technical debt is a good example of a bad analogy :
1. Assume some piece of code works, technical debt is only an issue if that code ever needs to be modified. Financial debt is always an issue regardless of what happens, even if you die.
2. Writing some code that generates the so called technical debt will often have an upfront cost saving, so whilst it might incur some extra work down the track, the extra work isn't a straight cost. The cost = (extra work - time saved up front).
3. Attempting to avoid technical debt often ends up with over-engineered code.
4. Being dogmatic and avoiding technical debt in all projects is often not good for the project / business / investors. Being early to market at the cost of code quality is often very important.
I think point 3 is the worst scourge of all. It is quite common amongst mid-range developers. After working for a while they start to get good development skills and start to think too much, looking for ways to express their skills.
In summary, it is like most things in life, there is a balance to be found.
Something that gives a counter opinion to group think is never appreciated. Better we all sit around and stroke our egos, with every statement implying how magnificent we are and at the same time implying how everyone else is churning out debt filled shit.
I am not sure "delaying business decisions for when you have better information" causes technical debt. Under this premise, pretty much every agile development project would be technically bankrupt.
I like its target audience, but am I supposed to just forward this to my superiors? Can't imagine they won't feel at least a little insulted, even if it's a concept they need to understand.
Always an interesting subject. I can't help but feel it would reach a much wider audience and achieve a greater impact if it was subjected to an automated grammar check, or edited by a native English speaker. I could not get through all of the distracting grammar errors, and I suspect most non-technical English speaking managers (who seem to be the intended audience?) would have the same reaction.
As a native English speaker and someone who prides themselves on their ability to clearly convey complex ideas through prose, I don't recall a point where a grammatical error detracted from my understanding.
"Bad code" is a completely subjective measure which everyone is sure they will know when they see it, but few agree on the actual definition.
Extreme examples will be floated, but in practice what people consider "bad code" are things that made decisions that aren't sustainable. For instance a complete lack of permissions. A lack of logging. A lack of multithreading if it's a CPU bound process. Locks. Dependencies. Etc.
Each person and team really does define bad code differently, and often in complete contrast with each other. There are people who considered DI heavy code bad code. Heavily abstracted and layered Java code is considered great code by some, horrendous code by others. And on and on.
On the other hand, if you focus on quality, you end up making choices that enable fast development. You make sure you have high-quality requirements and a deep business domain insight in the whole team, which means you end up writing only that code which you truly need, and write it in the right order (most important features first). You make sure you have code that is well-covered by tests, which as a happy side-effect makes it easy to adapt to new requirements. You focus on having a high-quality deployment process, so the amount of customer issues that result from a broken deployment go down reducing the load on your dev and support teams.
Focusing on quality first is not about gold-plating code, it's about making conscious decisions to have a high-quality process from front to back, and those decisions typically result in a highly efficient organization. It's no accident that those teams which manage to ship releases every few weeks (e.g. firefox, chrome, facebook) have world-class products. You have to focus on quality to go fast.
This isn't just theory. Capers Jones actually measured this. The fastest delivery schedules were produced by teams with the highest quality product. http://www.parasoft.com/products/article.jsp?articleId=2769