It's not always easy to find the right words to describe why a feature pause to refactor is needed. This quote was a succinct statement that avoids analogy:
"Postponing a small cleanup can transform it into a big cleanup because, over time, code builds up around the problem, and it too must be refactored."
I recently needed to make a change in how something common is done across several services. I made the change in one place (and have some passing unit tests), but then I realized that I’d have to duplicate more code than I’d like. So I’ll take a couple days to pull all related functionality out into a shared library.
This refactoring could’ve been done ages ago — there’s already some messy and duplicate code there — but now’s the perfect time to do it because 1) I’m already touching and destabilizing that area of the product, and 2) the clearer, de-duplicated code will be easier to spot check.
Makes sense. I guess for any given refactoring, the advantage of doing it sooner is that you get residual recurring benefits sooner. And the advantage of doing it later is you have more information about the requirements of your application and the "correct" way to do the refactor (or even if that code is going to be a long-lasting part of the app, cc Lindy effect). So whenever you touch any given piece of code which could use a refactor, you could try to figure out which advantage is more important right now.
Of course, if the total effort of doing the refactor plus building the feature on top of the refactor is less than building the feature on top of the debt, you should always do the refactor first. That's pretty much a no-brainer.
Another idea I've thought about is informally keeping track of possible refactors, maybe give them an informal prioritization, and taking a glance at your list before each task. Maybe there's some value in letting a refactor stew in your brain a little bit before actually putting it in to action. And continually maintaining a rough refactoring plan could help prevent your codebase from entering a state where it's actually unrecoverable.
Interesting, I think this is part of what the article is alluding to: what happens with tech debt is more to do with the team’s process rather than with the tech debt itself. Some teams are stuck in a process that makes them unable to tackle the problems even if they want to due to that process.
Of course you could just say “change the process” but it’s also obvious that there are many other factors and pressures that are non trivial
Also, as you alluded to, if you're changing related code which will require manual testing, then that decreases testing costs, if they're shared across the refactor and the new code.
Yeah I’ve got about five ideas for refactors that will make my life noticeably better. I’ve been pondering some of them for more than a year, slowly improving the design, but I’m still waiting for the right time when the ROI is higher.
This isn't refactoring - it's just hygienic coding.
"Refactoring as you go" sets up an adversarial relationship between dealing with tech debt and the feature work that that the time and effort was actually assigned to. IMHE, human nature and engineering politics then means that tech debt doesn't get addressed at all until it's already too late.
Approaching with this intention also takes refactoring entirely off the roadmap - from the eyes of product management, refactoring becomes invisible and "free", and no longer requires budgeting for. "Why do you need time to handle tech debt? I thought we were refactoring as we go?"
By all means, it's good practice to leave the code better than you find it, but don't just leave the tech debt that isn't trivial, is too large to address immediately, to fester.
Instead, take notice of inefficiencies and cruft while working, and write tickets for it, so it can be seen and brought up for discussion, prioritized against other tasks. Perhaps even leave a comment on the ticket whenever you run up against the same issue, so there's a paper trail of the how serious the issue.
This way, at least management has visibility into the state of the codebase and can't act surprised when tech debt brings productivity to a grinding halt.
A lesser issue, but still important IMHE, is that actually trying to "refactor as you go" makes PRs inscrutable. What's refactoring? What's feature work? A little bit of noise here and there is unavoidable, but any refactoring substantial enough to increase development velocity is likely to make meaningful code review very difficult.
Agreed about mixing a refactor with the incremental feature add makes PRs confusing.
"and write tickets for it, so it can be seen and brought up for discussion, prioritized against other tasks"
I have been doing this for almost a decade now and I have at no point ever seen even a top-ticket tech debt item get prioritized. Ever. It's always too much effort for little payoff (in the eyes of the business).
I explicitly tell juniors that when I cut this kind of ticket, I do it because it is right, not because I think it will ever get worked unless I myself (or someone similarly insane) directly disobey marching orders to work on it.
And even when someone does disobey orders and refactor, it's often a pet peeve that isn't one of the more major ones. So that's fine but nobody ever one-man-armies against the real problems.
And to illustrate what I mean by "the real problems", let me explain. I like to talk about tech debt as high interest or low interest.
Low interest tech debt is the shitty python script I wrote the other day that took 1-2 hours of manual monkey work we'd have to do in Presto sql and that only 1-2 people even knew how to do, and makes it so literally anyone at the company can do it perfectly right in a minute or two. It's a shitty script, but it knows it's shitty, and tries to have no pretenses about it and make it as simple as possible to debug and make incremental changes to it. So yes, if I spent more than an hour or two on it, I would not have done it like this at all. And yea it'll probably be a little annoying to change the next time someone has to do it. But it saves a lot of time, didn't cost much, and it's not going to change how we design the system at all.
That last bit hints at what I mean by "high internet tech debt". High interest tech debt is how it takes 8 hours of babysitting to deploy a schema change. It causes blip outages in every region as you deploy it. It's so painful it perverts our schema design to minimize how often we have to migrate the database. And it's all because, shortly after the Big Bang, right in between photons existing and the first hydrogen nucleus existing, someone had a cool idea for a simple trick about how they could eat their cake and have it too with how our databases work. And we have paid for it ever since. And they built on top of the primitives this offers so deeply that we'd have to change a ton of shit to make it work. The best time to fix this was 5 years ago, the second best time is now, and I guarantee you we will never ever do it. We'll replace the whole thing with something else because it's shiny and will be worse for years before we actually just pay down this tech debt
Seems like maybe shops have a tendency to get stuck in one extreme or another, either shoving crap out the door or else excessive concern with prettifying minutia.
I’ve literally never seen the latter: I think companies that do that just disappear. The former though is a great driver: Let your debt pile up sky high because it doesn’t matter as long as it lets you make more money in short term vs dealing with it.
you may say it’s short term thinking, but if you’re optimising for investment, that’s the right thinking. Make money now, you can always move it elsewhere that makes better use of it.
It’s not what I personally agree with, but just my observations so far
With regard to "excessive concern with prettifying minutia", I'm thinking of e.g. code reviews where people spend significant attention going back and forth on minor stylistic issues.
Another example: I remember a coworker reviewing code I wrote many years ago. I came up with a solution that was reasonably simple and workable, but wasn't the "right" way to do it. My coworker complained, I pointed out various reasons the "right" way wasn't practical, and they said something like "yeah but it has to be right" (without suggesting any concrete plan). Very frustrating -- I don't think it was particularly important code.
Working on my own, I already have a sense of how much effort I want to spend on code quality. When you add code review on top of that, it can feel a little excessive, depending on the importance of the application.
I actually have a lot more memories of code review frustration than code review gratitude. At my next job I would like to experiment more with code "previews" or design reviews -- that seems more efficient than rewriting code which already works.
I sometimes catch myself discussing “correctness” in PRs because it’s important to know what tradeoffs you’re making, and whether a neater/simpler/better solution exists. So yea, sometimes my suggestions can be misinterpreted as blockers.
I have since learned that giving actionable feedback is the trick to smooth review process, as well as trusting the authors (in a way). Their implementation may not be the way I would have done it, but I have to ask myself “will it still work?” as a basis for pass.
But I digress, I can see what you mean in your original reply, and agree, that weird “arranging deck chairs on a sinking ship” does indeed happen
"Always leave the code you're editing a little better than you
found it"
- Robert C. Martin (Uncle Bob)
There's no point in refactoring the whole thing. Maybe add a longer comment explaining the logic you had to decipher when you encountered the code. Rename a few variables from foo, bar and baz tom something more descriptive etc.
Agreed. The architecture mismatch paper [1] identifies common assumptions that software can make, such as "I own the main thread of control and other modules will do my bidding", that tend to be baked-in from the start.
> Deep architectural/design flaws in a codebase can't always be addressed using a series of small independent changes.
Not sure that's true. At the extreme end, you introduce a replacement with a better architecture and run it side by side with the old one, incrementally switching over dependents. Of course, that may take more overall work, but maybe the incrementality is sometimes worth it.
In my experience you need to do ‘the big refactor’ when a codebase has to handle something big that was not part of the original design such that you find yourself having to break the simplicity, elegance, completeness, coherence, etc of the existing system by tacking on some lopsided or alternate route or structure. What you really want in place of doing the ‘tack-on’ is a new simple, elegant, coherent, etc system that can handle both the old requirements and the new. In other words, you want to do the big refactor when you have new requirements that really should have been known at the time the system was designed such that you would have done things differently to accommodate them along with all of the requirements that _were_known at the time. This is easier to do the more monolithic and strongly-typed the application is.
Naturally then, you do want to know as many requirements up front as possible, which is the basic point of the article. Even though it’s not always possible, it’s still the best path to try.
All of this ‘screw design lets just roll up our sleeves and start coding’ is a great way to end up with spaghetti code and technical debt.
The key is to do as much requirements gathering as you can up front because your initial design will address only the requirements you know about, and the initial design constraints future updates.
"I'm sorry, I didn't understand much of that. Are you saying you can commit to finishing the feature on a shorter timeframe than you originally asked for? Would it help if we forgo writing tests?"
"I'm confused. When I greenfielded this app, several thousand commits ago, I took half the time you've already spent on this feature. You said you were a senior engineer!!"
I’m lucky that I don’t have a boss like this, but I’ve asked myself this question recently. I’ve been with the same company for almost eight years working on the same code base, which I and another dev greenfielded. I know it very, very well.
I’m often dismayed at how long things seem to take these days compared to when we first started out. Am I getting slower? Lazier? So far, I’ve identified the following factors:
- We have more customers, and those customers are much more demanding (used to be b2c, now we’re b2b). The cost of making a mistake is much higher.
- It’s just a lot of code. Parts of it are fairly complex, as much as I try to keep it simple. When a core component is changed, multiple services might need refactoring.
- We have many more features, and they sometimes interact in surprising ways. I’ve been around longer than our product people, so we often have to spend time iterating when they come up with a design that doesn’t fit with what’s already there.
- It’s important to refactor your database from time to time as you learn more about the domain and find simpler ways to do things. But refactoring a database is terrifying. I spend a lot of time triple-checking my work.
Working on such a large code base for years is super satisfying though. I’ve learned so much about system design, just from noticing how easy or hard stuff is to maintain.
I wish I had more than one upvote to give you for this post.
> I’ve asked myself this question recently.
I suspect both are true. There are real complexities that have grown around you and working in the same way on the same stuff for so long has caused you to habituate to a few inefficiencies. I suggest shaking up your world view a little and seeing what falls out. There are probably a few big gains you could make.
I would approach it—at least initially—as a mental exercise. What are your assumptions about the role, the code, the product? How could you (in)validate those. What would it look like to take each thing you think you know and invert them one at a time? What if things that you think are bad are actually good? What if things you think are fast could be twice as fast? Etc.
The first time, I lacked the self-confidence to speak up in the face of dominating people. So I internalized their behavior as "proof" of my incompetence - in the face of the patently obvious facts to the contrary, in the face of my own experience and judgement. Naturally, this made things much worse - I consider myself "part of the problem" in that case, though not the largest part by any means. Anyway, I was fired a few months later.
The second time, it was a breaking point - the CEO, who said those things was incorrigible, and the situation was unworkable. I called my boss in the morning (the fucker CEO had been yelling at me at 11 at night) and gave notice. He quit too - exhausted of losing engineers and being party to the abuse. Within the month, they lost most of their engineering team, and the few who stayed had received promotions and substantial pay increases to incentivize sticking around. They also "saw the light" and halted feature work for several months while (I assume) the worked on fixing their tech debt problem.
I've done really substantial work on myself since then, and I feel like I'm in a much better place to appropriately execute the soft skills required by my position. So I'd like to think that if (or rather, when :) ) the first situation occurs again, I will be self-assured enough to push back in an effective non-confrontational way, or at least speak my mind instead of being silenced by the unreasonable shame of an inappropriate dressing down. I would find ways of halting the narrative every time bullshit was spoken, and address the "inaccuracy" instead of behaving in a way that that manager took as confirming his suspicions that I was the problem.
And, in the second case, I'd have quit way, way earlier, when I saw all the previous red flags.
Mostly though, I'm not going to work for hotheaded, first-time founder-engineers so recently graduated from college, so bereft of the experience required to lead an engineering team. :)
I’ve come to realise that there’s no valid business case for dealing with tech debt early, nor adding tests to an existing project (bar some special circumstances / legacy change, critical outage, etc)
It’s like a lot of things have to be aligned for “good” development practices to reap benefits, most shops are much less organised, and a bit of chaos and early/quick iterative shipping will always yield better results.
Having said that, my core belief is still that if you take care and do things properly you’ll go fast in places where all other companies get bogged down
I'd argue that there's definitely a business case for frequent and early refactoring - it just needs to be refactoring worth doing in the first place. IMHE a most code gets touched rarely, and some parts get touched all the time, so refactoring needs to be strategic if it's going to have any utility.
This also argues for ongoing refactoring - addressing pain points as they arise, while they are fresh in people's minds, rather than suffering through them until Stockholm Syndrome sets in, people can no longer see the forest for the trees, and much of the velocity refactoring would provide is lost because the subsequent work is already done.
But the biggest issue I've seen with not handling tech debt on an ongoing basis is that there's never a good time to start. So if it's not built into the development cadence, then resistance builds for doing it at all. Product starts pointing fingers at "slow" engineering, who point back at the "breakneck" demand for feature work, and negotiations start for unrealistic (for both sides) halts in feature work - neither sufficient to resolve the problems nor short enough to avoid hurting the business.
Then product breathes a sigh of relief - the tech debt is "resolved" and will never need to be addressed again, engineering returns to wading through a codebase that is only marginally less swampy than it was before the cursory refactoring sprint, and the downward spiral (and finger pointing) resumes.
At least that's how it always seems to happen around me :)
As far as tests, IMHO refactoring without them (with BDD style tests being greatly preferable) is fraught with peril. But unless the team is bought in on BDD tests and using them to guide development, I agree they are a time sink to write them early. However, writing them later (and around code that might have been touched by multiple hands) rather than maintaining them is flavor of pain - like the refactoring, it's harder to be sure they won't miss things and you'll break something.
I think it goes back to experience. If everyone involved has suffered at least once, they’ll be keen to spot issues before they cause pain. both developers and managers.
So therefore my current reasoning is this sort of thing can’t be taught or explained /shrug.
I’m currently on a quest to teach/explain this now, so depending on how that goes I may change my mind, but i’m not holding my breath
I do opportunistic collection - when I am working on a feature, and spot opportunities for a refactoring/cleanup in code that is more or less directly related to the code I'm touching, I will keep making small incremental changes and keep testing them until my feature is implemented and the cleanup is done as well. I also ensure to not make a breaking change while doing this e.g. no change to the user facing api signature. If I see issues in unrelated code, that just gets a TODO and waits its turn when we have to do changes there for some feature request.
The reason is that I can atleast somewhat justify the changes under the umbrella of my feature while utilizing the allocated time budget. If I don't do this, we will never get a dedicated release to do tech cleanup - the backlog of feature requests is just too big and too little appetite on the decision makers' side for purely tech debt releases.
Big fan of this. Here in New Zealand we have a slogan “be a tidy kiwi” that encourages people to pick up their litter and be good stewards of for our natural environment
Imo the same mentality is good to have in software, and I’ve always appreciated being in a team that makes codebase improvements alongside feature additions. It makes things a lot more pleasant
I have done this in the past and it has worked well. On some teams though, I have been met with:
“Why did you do this refactor with the feature? Can you pull the feature into another PR, then we’ll leave the refactor in the original PR to be merged at another time? (read: never)”
I'd definitely agree with the putting the refactor into a separate PR and then the new functionality in another.
Aside from the obvious that people are more likely to be willing to review your change when it's small, it makes it much easier reason about both when they are separate.
There will be a lot of noise generated from the refactor that will drown out the new feature, but self-contained in its own PR it's a lot easier to understand the new feature and spot mistakes.
A simple refactor can likewise be likely skimmed over quickly, looking for the patterns of how it was done and assuming that most of the changes were similar and mostly just looking out for the differences.
As for accepting the feature PR but not the refactor PR, that suggests that the feature doesn't actually rely on the refactor. In that case, it's even clearer that they should be separated.
Personally, I'd always even create a new ticket for the refactor so that there's some justification for the work, and maybe you can say that this ticket blocks the one for the actual work. Maybe that's just me though, because I like to make sure that every non-trivial commit is tied to an issue in the bug/issue tracking software. This means that every bit of code is a simple "git blame" away from a justification of why it was changed.
A refactor should, at the very least, be in another commit if not another PR.
The refactor being separate in a separate commit/PR makes your code easier to review.
> Why did you do this refactor with the feature?
This indicates that the team doesn't think the refactor is needed or it wasn't a priority for them.
Teams that push back probably have a different mindset than you. Some teams don't understand the negative impact of tech debt, others don't know how to prioritize it.
This is the only way I've found that we can do the needed refactor. As consultants, we never ask the permission of the client to refactor the code. Ever. That's not for them to decide. The reason they hired us in the first place is because we're the experts and they are not. We evaluate the needed refactor that puts the code in a decent state and put the time in the quote we provide the client. And even then we never create tasks or log time; when we find something that needs to be done, then our time is billed to the task relating to that issue.
Consider a case where every non trivial refactor ends up in several rabbit holes, requiring lengthy meetings with lots of people to discuss, and many disagreements about direction. And in light of that the trivial refactors start to feel like you’re taking a bucketful out of a tsunami.
What ends up happening is it’s much more impactful for the team to just focus on shipping what they can, partly because they don’t have the tools/procedures/experience (anymore) for dealing with tech debt in this context
And that is exactly how the cruft I see has built over many years (well before I joined the team). It is not that devs before me were not smart; they just focused on doing the bare minimum without any attention to keeping the code easier to manage and reason with. An append only garden.
Wow, what a weird feeling. The authors of this article seem like super smart and experienced guys, but the article itself is introduced with incorrect statements in the very first two sentences and slides downhill from there.
"There is a kind of design distortion that happens when a team chooses to build iteratively instead of looking at all of the requirements at once. Ward Cunningham coined the term technical debt to describe those design distortions."
1) There isn't an option to build software by "looking at all of the requirements at once." The requirements of any (sufficiently complex) project will emerge over the course of development, regardless of whether it's built in an iterative style, or with a much larger investment in planning and design up front. We often work iteratively because we acknowledge this particular aspect of reality and it helps everyone when we work closer to reality, rather than fighting it.
2) Technical Debt does not describe design distortions that arise due to "iterative" development, it describes design problems that arise in every project.
Quantizing the choices for dealing with tech debt into 4 buckets doesn't feel right either. Changing the design of a running system happens along a continuum and the need and benefit vary widely over the course of any given software lifecycle and surrounding (business or other) environment.
Maybe I'm just grumpy this morning, and I think reasonable folks could disagree, but the metaphor at the center of the thesis (tech debt ≅ soon-to-be-deallocated-memory) doesn't hold up for me at all.
Tech debt is sometimes left in place, intentionally or not, for many years. Sometimes we chip away at it, sometimes we stop the world and push on it. Sometimes it's bad enough to throw the whole system away and start over. It's like debt for businesses. It can be valuable to take on some debt if it lets you stay alive long enough to pay it back.
Finally, I'm struggling with this article because tech debt is already a metaphor, that includes information about how, and when to take it on and pay it down.
It's not helpful to layer another, unrelated metaphor on top of it.
Ward Cunningham's original idea of tech debt (see [1], a beauty of concision at just 300 words) is that iterative development distorts your code because you start writing code before you know the requirements, but even so, it's better than waterfall. "The traditional waterfall development cycle has endeavored to avoid programming catastrophy by working out a program in detail before programming begins. We ... [instead use] the alternative, incremental growth ..."
Today, the term "tech debt" includes sloppy code, shortcuts, novice code -- really any kind of bad code. The original conception of tech debt is more limited and tied to waterfall vs iterative process choices. [2]
I would argue that there is the option to look at all the requirements at once. The disconnect with your mental model is in the definition of the piece of software being designed. You’re considering all iterations to be the same piece of software, despite ostensibly having different sets of features and functionality (e.g the ones added over time).
Consider the alternative perspective that each new feature actually makes it a new piece of software. You were able to fully design and specify the original piece up front, thus avoiding technical debt on that piece of software.
I don't think that perspective really holds up to practical scrutiny. The most common sources of tech debt in the real world are when new features are tacked onto existing software without integrating with them fully/smoothly.
As a relatable example, think about how the Control Panel in Windows still uses a 2000s-era UI, even as the Settings menu, and most of the rest of the OS, uses a new UI. The likely reason for this is that it was faster to tack on a new UI and leave the Control Panel as-is. It would've taken more time to refactor the Control Panel from the ground up to accomodate a new UI.
The end result is that there are now two separate settings interfaces, and there is probably some ongoing engineering effort required in maintaining both and keeping them coherent with one another. That's a classic example of tech debt - save time now, but it may cost you later.
But, by your definition, the old UI and the new UI are separate pieces of software, therefore there is no debt. How does that track?
There are some bits of software that are simple enough to describe first and then implement, but just about any software that has more than one person working on it is more complex than that (IME), and for those cases, one might as well just implement it anyway.
There are varying degrees to which a team or business may invest in collecting and negotiating requirements before starting development, and there is obviously some amount of this work that is essential and valuable.
That said, as this planning investment grows beyond some reasonable point, and moves deeper into technical details, it will be composed of more waste and this waste will only become apparent as the people executing discover the real technical and product requirements.
The technical debt we're discussing will become an option immediately as the engineering work begins, and it's up to the team to decide how much, if any debt they are interested in taking on at whatever point in time.
I'm not convinced that it's helpful to conceive of the thing we're all working on and becoming more and more intimately familiar with as a new thing over some randomly selected interval.
There is no reality in which we can fully design or specify everything up front. If the specification were comprehensive, it would be the solution.
In practice, *practice*, we use the term iterative pretty loosely. In fact sort of like this IEEE article, it feels dated now, and was never really true in my experience.
It seemed like the word iterative became a euphemism for “dont worry about it”. I have worked across the entire industry for 30 years, and I have never seen a “true iteration”. On any feature or application. Anything at the end of the sprint that doesnt work is just a bug. Thats not iteration.
Iterative development was just a powerpoint slide for the Scrum Industrial Complex which was hard selling itself and it worked!
This article which seems even more antiquated than Id like to describe, seems to confuse iterative with “having all the requirements at once”. Iteration and the quality of requirements, and the bandwidth to analyze them and design a system dont really have much to do with each other.
And thats why iterative came to be a stand in for “dont worry about it”.
I personally feel that the creators of Scrum or some responsible group of modern architects, should audit the results of supposed iterative development, and do an honest assessment of what it actually is in practice. Because its not what we were sold, and its not iterative. It does result in awfully bad software but there have been other shifts in development practice that have accelerated the awfulness of delivered work in the industry, and Ill just leave it at that.
Please resist the temptation to comment, “because your not doing scrum right”. That was the genius of the Scrum Industrial Complex, to suggest that somehow this simple thing is really magical and complex and keep repeating the same things over and over. Its not magical or complex, its just bad.
As far as the article goes, I have been on projects that did spend over a year ingesting requirements and doing design before starting to code. Its true they usually fail. But thats a money/commitment/time to market problem.
Starting to “code” right away creates some better optics and fewer arguments in meetings. The one who pays the piper calls the tune as always.
> Building software iteratively leads inevitably to tech debt because we choose to deliver systems before we have looked at all the requirements. Not knowing what’s next distorts our designs, and that distortion is the tech debt.
This article frames technical debt as something that happens passively because you can't know future requirements. That's sometimes true, of course, but in my experience the majority of technical debt is accrued deliberately in a much more active process.
When developing a new feature that doesn't neatly fit into the existing system, you must choose between two compromises:
1. Build it the "fast way", shoehorning the feature into the system just enough to work, compromising quality for velocity and accruing technical debt; or
2. Build it the "right way", adapting the system to accommodate the new feature, compromising velocity for quality to avoid technical debt.
This is usually a deliberate decision, so choosing to accrue technical debt is an active process. The only way it could be passive like the article describes is if the developers don't know or otherwise don't consider the "right way" and go straight for the "fast way". I hope to never work on a team that operates like that.
The problem is that there is never an objective right way. There are infinite wrong ways, and usually a handful of ways that are just fine with different pros and cons that are not always clear up front.
A lot of the time when people say technical debt they mean a developer not taking the time to understand some code they inherited (or they wrote and forgot about), and wanting to throw the baby out with the bathwater.
If think you ask ten developers the best way to refactor a complex program you'll get 100 answers.
But I do agree that deliberate technical debt is more common on a decent team. I definitely have left many comments like "I know it would be better if I did XYZ, or ABC, but the boss wants it now, and I'm tired, so it's going to be a monstrosity"
the 10 -> 100 bit is absolutely right. a cross functional team has to learn how to work together well, and spend a non trivial amount of time discussing the problem and tempering solutions until something is achieved where each of them are some state of “happy”. This is I think the crux of many tech debt issues in companies
The other possibility (which is common in startups) is that often the “right way” is different depending on the scale of the system you need to design for. In cases like this you end up with technical debt a year down the line, but at the time the feature was shipped the engineering decisions made were extremely reasonable.
I’ve seen a few colleagues jump to writing off all technical debt as being inherently bad, but in cases like this it’s a sign of success and something that’s largely impossible to avoid (the EV of building for 10-100x current scale is generally negative, factoring in the risk of the business going bust). There’s a kind of entropy at play here.
Big fan of tidying things up incrementally as you go [1], because it enables teams to at least mitigate this natural degradation over time
> The only way it could be passive like the article describes is if the developers don't know or otherwise don't consider the "right way" and go straight for the "fast way". I hope to never work on a team that operates like that.
I've been on teams like that and it's absentee management. There's two reasons: the management is technically inept or the execs and other stakeholders are taking up too much of their time. Sometimes it's both.
This creates a situation where either the most technically competent team member takes over responsibilities or the team just loses coherence.
Beyond that there's also often a severe lack of involvement from the broader organization where everything is siloed.
The quality of the dev team reflects the quality of the business. Implementation details are not so different from "operations" in other fields. Makes no sense we have such a terrible state of things in software at some places other than a lack of talent and residual people who should have retired or pivoted careers a long time ago.
There is often a third alternative: do not shoehorn it into something, nor rebuild what you have to fit this new thing, instead build the new thing on the side.
I sometimes have worked with engineers who believe they know what “the right way” is, or spend a lot of time trying to figure it out. And I have certainly worked on legacy systems persons like that have built. It’s not all fun and games.
The less we entangle things the easier it is to remove cruft when it is no longer needed.
You’re completely right about entanglement: Id go further to say that simplicity in its purest form is almost always more useful than clever upfront design, but it’s funny because clever upfront design process can be tweaked to bias simplicity.
What I’m trying to say is that there are two ways of designing systems: Make them flexible to meet unknown future objectives by incurring tech debt, or build them so simple that they are easy to change when those unknowns come in.
You may say that there’s no “right way” to build systems, but some ways are certainly better, perhaps it’s possible to distil a way from that, but i agree that it can be paralysis inducing. can’t go wrong with building Simple working systems, imo.
This is a perfect example of when deliberately taking on technical debt is the smart thing to do. You build the feature the "fast way", deliberately incurring technical debt to get faster feedback on the feature. If the feature doesn't work out, you can remove it quickly. If it does work out, then you can be confident that building it the "right way" is now worth the investment.
Technical debt is just like regular financial debt in that it can be a powerful tool when leveraged correctly.
So, from my 30+years building software products, I have a couple of problems with this:
- we don't choose iterative development as an alternative to waterfall because "waterfall is bad mmkay". When developing a new product, we don't know all the requirements. Part of the process of iterative development is discovering the requirements. Every waterfall project is secretly an iterative project because there's always a Phase 2 where the requirements get updated (even <especially> when every stakeholder pinky-swore that the requirements were final before Phase 1 started).
- The optimum amount of tech debt is not zero. Tech debt is a product of changing the product plan/vision/design in reaction to learning from customers. If you're not accruing tech debt then you're not learning from your users.
- Changing the design to accommodate the new feature is great in theory, but in a lot of cases the new feature is an experiment itself. We're optimising for customer learning: get the new feature in quickly with the minimum of effort, see if the users actually use it, and if not then roll back. If the users do like it, then refactor to tidy up any tech debt. Doing all the refactoring first is a complete waste of time if the feature ends up getting rolled back.
- Tech debt slows down development but only as a result of delivering early features quicker. It's the sharpening the axe analogy - sooner or later you have to stop chopping the tree to sharpen the axe. Changing the way you chop the tree so you don't have to sharpen the axe is not optimising for getting the most timber quickest. The trick here is communicating with stakeholders about what tech debt is and how it accumulates. They're usually OK with feature pauses if they understand the situation correctly. In other words, like a lot of problems in this industry, this is a communication problem not actually a tech problem.
> Every waterfall project is secretly an iterative project because there's always a Phase 2 where the requirements get updated
This is a false equivalence. Waterfall is iterative in requirements but not in code. Changing the requirements document is much, much simpler than changing the code to conform to new requirements.
> The optimum amount of tech debt is not zero. Tech debt is a product of changing the product plan/vision/design in reaction to learning from customers. If you're not accruing tech debt then you're not learning from your users.
Not sure I agree with the implication "learning from users entails accruing technical debt". Technical debt is a rough measure of the inherent resistance of your program to adapt to customer requirements. Some architectures are flexible enough that you could learn how to configure the architecture differently based on what you've learned from users, but I don't see how that necessarily entails the addition of technical debt, ie. as a rule.
If the requirements change, then so must the code. The code you wrote for phase 1 cannot (by definition) include the changes needed for phase 2. Especially because you don't discover the changes needed for phase 2 until partway through phase 1.
Writing code so that it can be easily changed to conform to new requirements is a trade-off, like everything. You will incur some cost to do this, probably longer dev times. In many cases this trade-off is not worth it.
And in my experience, shipping a new feature faster is always the goal. Dealing with the tech debt incurred is a tomorrow problem, and tomorrow we will have more funding/staff/time to deal with it than we do today. It's not always true, of course, but that's the attitude generally.
RAII your technical debt: make a note of every time you slap something together the expedient way, and schedule a future time to refactor after the legitimacy of expediency goes "out of scope", e.g., after a major product demo.
One could even add a CI/CD task to remind the user of when refactorings are due (and refuse to successfully build until they are addressed). Since debt is created by borrowing, one could call this the "borrow checker".
I think this is the absolute worst way of dealing with the issue, and I truly hope everyone who sees this in a code-review has the sense to reject it.
By all means, have such a check as a compile-time error, but not as something that gets shipped to customers. The code you have now works, even if as a developer you'd like to refactor it. However, what if you ship this software to a paying customer, and then your company goes bust? At unknown point in the future, their perfectly good application suddenly stops working, without warning or obvious cause, and your company is no longer in business to simply change the time to a later one and recompile. Even if your company hasn't gone bust, why should the customer have to spend ages trying to figure out if they've done something wrong or it's the software that's at fault, before contacting your company, possibly being told to pay more money for an update even if they were otherwise happy with the old version they'd purchased.
Basically, this change is a liability for everybody working on the project and your customers. It'll soon get forgotten about, until you have people complaining that something just suddenly stopped working without warning. You might be able to "fix" it in seconds, but it can still require hours to triage the issue to discover what caused it before it ends up as your problem again, and probably significantly longer after your few seconds to fix it before the working version is back in the hands of the customer. For all that time, they've been inconvenienced just to save you the hassle of sticking a reminder in your own personal calendar or raising an issue in your bug tracking system or wherever else.
Ooops, I guess I missed the part where he said "in tests". That makes it a bit better, although when it triggers, it'll still need someone to triage why the test failed and either nudge the date up / delete the test / do the refactor. While that's going on, potentially you'll miss other failed tests because your overall build is red, or maybe you'll force someone to backout their change for no reason and stop other checking in that day. I actually worked at a place where if the build ever went red, all commits since the last successful build had to be backed out and nobody was allowed to check in again until the build was green again. This sounds drastic, but having lived with it for a couple of years, it grew on me - it stopped the repeated "Oh, that's just a one-line fix" that then broke something else and so on... Better to immediately revert the change and re-commit when you're sure everything will work. We also disabled and/or removed tests that will intermittently fail due to a fault in the test, and created an issue to write a better test if it was still relevant. The GP's test would have just deleted when it was discovered, rather than achieving its aim of getting someone else to do the refactor.
The best approach is still to do the refactor straight away if it's genuinely needed, or else create a task in your issue tracker to do it a later time if it's going to require resource that you don't have right now. Then PM can prioritise it accordingly, not have to react to an arbitrary timebomb placed in the code, which will invariably be discovered at a time that's not good for anyone.
Good in theory, but try that on a project with real customers and you’ll get a very simple “remove it and we’re done here” /shrug. Team’s processes and decision chain must have agreed to enforce this already, and that above is just a symptom
Does anyone else get the feeling that how we deal with technical debt in software is fundamentally and critically misguided?
How a pot hole gets filled in isn't a technical challenge, its a political challenge.
Developers can talk about the _technical_ side of technical debt all day long, but its a) generally well understood already, and b) blowing in the wind. How technical debt is handled is fundamentally a political problem.
We may understand this at a glance, but the solution of "gain knowledge and bring it up at standup" is not only not getting us anywhere, it's deliberately misleading us into thinking its up to us individually to change the organisation. It's as silly as telling the labourer that part of their job is to lobby the city council to spend more time and resources on filling pot holes.
Discussions on technical debt should be within the bounds of how to identify organisations where technical debt is identified as a real issue from the top down, and a survival guide for working in organisations where it is not.
A quirk of IEEE's publishing system is that it drops the abstract, instead using the first paragraph. Here is the abstract [1]:
The iterative process that a team follows is a bit like a garbage collection algorithm, and we can compare software development processes like we can any algorithm. A process can help developers do two things: clean up tech debt after it exists, or avoid creating it. When an iterative process does neither, tech debt buildup will lead to bankruptcy, so it is only suitable for projects with a short lifespan. A process that does both has the best chance at minimizing tech debt over a long lifespan. In particular, focusing on the system’s design will keep tech debt low.
Oh, of course, of course. Why, this is so mundane, why would anyone even bother to read it?
> For content processed or stored on Adobe servers, Adobe may use technologies and other processes, including escalation for manual (human) review, to screen for certain types of illegal content (such as child sexual abuse material), or other abusive content or behavior (for example, patterns of activity that indicate spam or phishing).
They are going to use AI to scan through all content...for your protection. Does Adobe have a problem with people using their cloud to produce child abuse material? I think we all recognize Bitcoin is used for illegal activity. But I never considered Adobe being associated with CP. I guess that's what they're going with...
I feel like this is a trope at this point. Just like Microsoft Recall, no one believes or trusts this won't eventually turn into them using your IP for themselves via AI, and then eventually by the government. That's the bottom line.
To really put a point on it, they are saying they're swipping a spider off us while they slip their hands down our pants.
"Postponing a small cleanup can transform it into a big cleanup because, over time, code builds up around the problem, and it too must be refactored."