tl;dw: Technical debt is equivalent to the interest you are paying on it, where interest == pain. Some debt is planned and low interest, like a mortgage. Other debt is unplanned and high interest, like a credit card or payday loan. Sometimes that interest allows you to get larger returns. For example, if you could take out a loan at 3% in order to invest at 8% over the life of the loan, that's a pretty good deal, as long as you can pay it down. but, if you let that interest get out of control, if you don't pay down the principal, then it becomes more painful.
I really like a post I saw awhile back that refines the idea of technical debt as an unhedged call option. (Original link is dead, so here's the wayback machine link and the HN thread.)
> Call options are a better model than debt for cruddy code (without tests) because they capture the unpredictability of what we do. If I slap in an a feature without cleaning up then I get the benefit immediately, I collect the premium. If I never see that code again, then I’m ahead and, in retrospect, it would have been foolish to have spent time cleaning it up.
> On the other hand, if a radical new feature comes in that I have to do, all those quick fixes suddenly become very expensive to work with. Examples I’ve seen are a big new client that requires a port to a different platform, or a new regulatory requirement that needs a new report. I get equivalent problems if there’s a failure I have to interpret and fix just before a deadline, or the team members turn over completely and no-one remembers the tacit knowledge that helps the code make sense. The market has moved away from where I thought it was going to be and my option has been called.
Originally I wanted to write that there are two different things that get called "technical debt". In one case, the choice is between "clean code now" and "quick and dirty code now, sell version 1.0 to pay the bills, refactor the code, publish version 1.1". In the other case, the choice is between "clean code for more money" and "quick and dirty code for less money" with no itention to fix it later.
Technically, only the first case should be called "debt". But sometimes managers pretend it's the first case, when actually it is the second case, to help developers save some face. So the developers keep adding "refactor XY, priority: low" to the backlog, and then at some moment the developers are moved to a new project, and the backlog gets silently deleted.
Then again, sometimes a big change is needed unexpectedly, and all those issues that were originally planned to silently die in the backlog suddenly need to be addressed. So, in this case, the "technical debt" was actually a bet that this situation is unlikely to happen. (Or maybe that it will happen in a sufficiently far future so that it will be another manager's problem.)
interesting fact is that selling options is strategy called Selling Volatility, meaning you bet that things will slow down and normalize.
When applied to tech debt, it can be thought of as betting that product velocity will slow down, and you will have time to refactor/reliminate tech debt sometime later.
Of course when developing new product/startup product velocity only increases over time. And if you did accumulate tech debt - there will be some time when you will have to rewrite stuff from scratch, because refactoring and/or maintaining codebase will become unfeasible.
and it leads to conclusion that the best time to take on technical debt - is maintaining stable product, when product velocity is slow. you implement change quickly, then over time you refactor it. Which actually perfectly aligns with some agile methodologies (XP).
I’ve had some success with non technical people and a warehouse metaphor.
Can you keep shipping products if you never tidy up? Yes it’s possible but eventually it catches up to you.
There’s no specific point in time at which tidying up a warehouse is going to make or break a business but the untidy ones are more likely to be hard to hire for and more likely to catch fire.
And they’ll definitely slow down ability to ship products.
Warehouses that stay tidy have a predictable rate of shipping and reduced risks of accidents and fires.
Do you need to put that one single item in the right place every time to continue to ship anything? Well the answer is always no, but permanent short term thinking like that without ever tidying will always eventually cost you in the end.
Much easier to keep on top of it. Workers and new hires are also more effective.
I find that this metaphor doesn’t feel forced or stretched and it’s pretty intuitive and nicely harmonises with products and shipping terms that people use for software anyway.
Makes sense. I can imagine it working on non-tech people more easily. It sounds similar to an explanation of why people should keep their house tidy then.
changing requirement is just semantic/language trick.
the reality (market forces) will eventually catch up in form of angry customer/regulator demanding the asked feature asap, or frustrated clients leaving for competition, etc
> Weird phrases that we often mock, such as "action item", or "circling back", are actually highly efficient vectors of meaning. They fit non trivial concepts in very few words, and are understood by everybody.
I hold the opposite belief. Management speak is meant to obfuscate. Instead of just telling a report who disagrees with your decision based on expected outcome to think the way you tell them to, just say "think outside of the box". When you commit to "circling back", you're committing to nothing but having this idea cross the manager's mind at some point in the future.
It's weird that this is the conclusion the author came to from their linked article, considering this is in it:
> This is why you might be tempted to use the readily accessible management metaphor laced language which you’re familiar with. Don’t.
"Technical Debt" is this level of crummy metaphor. I love Cunningham's original definition. Do your best to model the business problem with the understanding you have. Later, pay down that debt by refactoring.
That's not what it means anymore. Tech debt means whatever the speaker wants it to mean. It's a get out of jail free card for any bad software choice.
"That choice will severely limit our capabilities in the future, and be incredibly hard to change."
"I'm willing to take on some tech debt to prioritize shipping."
All those phrases originally meant something. I think they become management speak via two mechanisms: 1. People use them shorn of their original meaning to deliver the appearance of being managerial without the substances or 2. People raise them to rock-solid rules and demand slavish conformance to them without thought about costs/benefits or appropriateness.
For bonus points, mix the two up, so that not only is someone trying to sound managerial but failing by using the terms completely incorrectly, but then also rigidly enforce their incorrect understandings.
For instance, if you have lots of meetings and those meetings almost never produce "action items", and/or those action items never get done, you really do have a problem. If you don't like calling them action items, fine, find another word, but it'll be the same thing. But if you get a manager that insists that meetings always produce action items, no matter what, you get another sort of failure. It is a perfectly valid use of a meeting to get two groups together and just make sure they are on the same page about something. If that's all you ever meet about, there may be a deeper problem you should dig into, this shouldn't be your dominant form of meeting (unless you're in a position where that's your direct job), but it does sometimes need to happen.
I find it useful to dig down to the "original" meaning of a lot of these terms, think about what they mean and how to use them in my own head to put terminology on things, but not to be rigid about it.
I also observe that there are some managers who use the terms essentially correctly, and some that clearly fit into at least one of the two categories I listed at the beginning. They should be treated differently.
I agree with you but I think there is something to the business side. No Sales/PjM/Manager/CFO is likely to want to spend 3 months of resources on non-revenue generating updates. They would rather pay the fiddler when the system has a failure. Just like us humans, people don't change their diet and exercise until AFTER the heart attack.
I'm not actually arguing against prioritizing shipping. My criticism is with the way that people wield language to enforce their choices and avoid responsibility.
Defending a choice means actually defending it.
"If we don't ship this by X date, there won't even be a company to have the problem you're talking about."
"This choice will have less impact than you think for X reasons."
The problem is the specific reasons are often more in the family of "Shipping this is one of my KPIs and I'll be on some other team when the problems you're talking about are realized."
It's not just business vs. engineering anyway. Slow to ship? It's cause of all the tech debt. No tests? It's okay, just tech debt. No docs? Poor abstractions? Lack of understanding about the problem? Low skill with the technology at hand? It's all fine! Tech debt can be your scapegoat or hall pass for anything.
I don't quite agree with the 'for anything'. For poor/missing technical assets that area direct cause of insufficient upfront investment (of time, money, etc), that's 'tech debt'.
Missing/outdated docs/tests? Tech debt.
"Low skill with technology" - probably not as easy to lump it as 'tech debt', imo.
"Lack of understanding about the problem" - again, doesn't seem like tech debt, but... business debt. Tribal knowledge debt. Operational debt, but not explicitly 'tech' debt.
But... perhaps that's your point? It's easier to just say 'tech debt' for everything, even if it's plainly the result of underinvestment on non-tech things?
>"Technical Debt" is this level of crummy metaphor. I love Cunningham's original definition. Do your best to model the business problem with the understanding you have. Later, pay down that debt by refactoring.
I on the other hand hate Cunningham's original definition because the metaphor maps badly, debt is not something that is poorly understand that becomes clearer the more information you have. Maybe that would be Technical Risk.
Your second example:
>"That choice will severely limit our capabilities in the future, and be incredibly hard to change."
>"I'm willing to take on some tech debt to prioritize shipping."
is a better metaphor because debt is the thing you take on that you need to pay off later with some knowledge of what level of debt you have. That said it's still sort of bad because debt would be nice to have sort of a ball park estimate of how much must be paid to get rid of it. If you don't have that estimate to go on it's beyond technical debt, and maybe more like technical favor owed to the mob.
That might be the outcome of not thinking or it might be the outcome of thinking “going out of business [or just canceling the project] will severely limit our capabilities even more and also be incredibly hard to change” and concluding “taking on tech debt is the better choice.”
Weird phrases and jargon are a burden in healthcare. There are literally thousands of terms you can learn that don't have applications elsewhere. As a healthcare IT specialist, you're spoken to in this language of doctors and medicine, and must take diligent notes and learn what all those 3 letter acronyms are so you can get the right diagnoses or drugs in your analyses.
There is intentional or incidental technical debt which is debt taken for good reason because of lack of enough context, YAGNI, or whatever with the understanding that it will need to be revisited and the developers are aware of it. This one is easier to deal with and is usually contained well because it is a known-known.
Then there is accidental technical debt that is mostly due to lack of design skill / judgement that results in code smells, spaghetti code, poor abstractions and a codebase that sucks the life out of everyone.
When most people write about technical debt, they refer to the former while it is the latter that is much more costly and prevalent and yet, avoidable.
With a name like 'gilfoyle', I'm compelled to remind you that son-of-anton's debugging capabilities proved rather useful at creating a third category of technical debt - namely technical debt resulting from unintentionally deleted code.
Because code is written by humans, it behaves in ways that are not too much of a stretch to call “organic”.
We need more literal and figurative gardeners in programming, but everyone is a hurry to get their 10k hours in before they learn how to adult properly.
Gardeners make suggestions to plants, but the plants end up doing their own thing about half the time, so everything becomes a negotiation. When you try to assert your will too hard, you can kill the plant.
If you’re pruning a tree, the tree reacts by putting out more branches. If you do it too aggressively, the result is an ugly, unhealthy tree. If you stay below a threshold, everything keeps working as expected. So you have a pruning budget every year, and renovating a tree nobody has maintained may become a four or five year project.
The first year you cut off the sickly branches, and the one you keep walking into. Second and third year you do preventative maintenance, removing crossed branches and bark inclusions (future calamities). By year three you might begin making aesthetic decisions, and by year five you either finish up or start over because the tree has changed so much in the last four years.
Like software, usually this plan is in your head, not on paper. But for software it doesn’t have to be this way, we just haven’t done anything about it yet. I hope that 15 years from now we have ways to record code reviews not attached to PRs, noting targets of opportunity and storing them with the code. That way, when I’m adding a feature I can also look at the refactorings suggested by others, especially the ones that are not locally apparent, like architecture and code duplication issues. Having to fix a problem the moment you see it is a very ADHD mode of thinking.
Even the first one can become an issue if the follow up refactoring is not prioritized and the team moves on to the next feature, the POC becomes a Product. It needs discipline to make sure the follow up investment happens.
I would refer to the latter as "technical death", as in a terminal disease you never recover from, because it comes from an incurable culture of dysfunction. I say "incurable" anecdotally, and I'd like to be wrong about that, but I suspect I'm not, and maybe it's better to recognize you're swimming against the current and just get out of there instead of hoping to pay off this idea of debt someday.
Shameless plug: I also wrote about how we have overused the term "Tech debt". The concept of "debt" for me should include some sort of "investment" - ie you don't do something well because you need or you want to invest the time on something else.
But if something is done with plain bad code, it shouldn't be called tech debt. Something that outgrew its objective shouldn't be called tech debt. If something is a bug it shouldn't be called tech debt.
Something that outgrew its objective shouldn't be called tech debt
I agree with the rest of your comment, but something that outgrew its objective is almost text book tech debt. Instead of taking the time to build something new, or refactoring, you tacked it on to something existing.
I think the idea of short-term vs long-term tradeoff is key. And whether you chose the short-term option decisively or naively, doesn't alter the fact that the tradeoff existed.
Technical debt also manifests itself differently. Some of it might not be felt on a daily basis, but acts more like extra mass or like drag, impeding acceleration. That is slowing you down when making or changing stuff.
This is in contrast to the debt you have to pay each day in production.
We have a fair bit of that "extra mass", due to shortcuts taken many moons ago under tight deadlines. On a daily basis the code works fine, and does what the customer needs.
However due to it we spend extra time when we need to extend it or change it.
> But if something is done with plain bad code, it shouldn't be called tech debt. Something that outgrew its objective shouldn't be called tech debt. If something is a bug it shouldn't be called tech debt.
I don't think anyone refers to bugs as technical debt per-se. What I have seen is bugs that result from technical debt, and hasty bug fixes that create more technical debt.
I think the whole thing is hogwash. Debt is an instrument of contract, nothing more, nothing less. It's packaged, legally binding and sold on the markets. There is nothing good or bad about it, it is what it is on its own.
Even thinking in terms of "investment" is gap thinking and treats human systems without human traits.
In a system of joint activity, people and artefacts work together in harmony. If they're not working together, it's not joint activity and if its not joint activity, that doesn't mean its debt either. It's dysfunctional.
Well on an organizational level badly written code is imo tech debt. It's worse quality in exchange for paying less money (paying less capable / inexperienced devs).
I always feel like these academic approaches miss the point of why technical dept is just a thing that happens. Sure, there is the bit where business and development don’t speak the same language, which is outlined well in this article. But I think you have to zoom out beyond the size of the single project, if you want to know why technical dept piles up, almost regardless of what you do.
Development, maintenance and operations are three fundamentally different and often opposing forces that no management is ever going to tackle right.
As a result you’ll have developers who build programs and then move on to build other things. You’ll have operations that’ll have enough know how to operate and error-handle but not enough to re-develop when the real world changes. By then, the developers have either moved on, are too busy or have forgotten enough of the project that no one is actually available to change the system to match the new reality. If by some happenstance, you actually do have someone who can redesign and reimplement the system, they will be given too little time to do a proper job, because developer resources are expensive. When the real world has changed enough times, you’ll have technical depth.
The only projects that can avoid it, are those big enough, to have their development team never leave and management willing enough to let the developers spend 1/5 of their time on maintaining the system.
That’s not to say that you should do nothing. Often it’s proven itself valuable, for us at least, to design systems with the full intention of throwing them away 5-10 years down the line. By not applying any sort of best-practices that are meant to make the systems easier to readjust during their life times, building them takes maybe 40% of the time, and if they are simple enough, you’ll never have an issue with them. Then 5-10 years later. You build their replacement. Which by theoretical means, would be a terrible business case after 15-30 years, but ask yourself how many smaller systems your enterprise organisation has, that are 15 years old. Out of the 300+ (and I say + because we don’t have the full picture) systems we operate, less than ten of them are that old, and every one of those systems is large enough to have full time developers working on them, often with a 9/10 focus on maintaining them, and only 1/10 on building stuff like more modern APIs for them.
I’ve always wondered how many of our problems with technical debt are self inflicted by trying to design for some future that may never exist. Simple, direct, and declarative code with tests are very maintainable.
... works only in an ideal world where nothing changes, i.e. where software written once is so good that you don't need to touch it.
In reality though, what happens to successful software projects is that once the management and the customers see that it works and is good enough they always, always start to request more features. In fact in the minds of managers adding a feature is just adding a feature, should be easy right? Well no, a lot of the times it's not. Too often adding features affects some core abstractions in the code in such ways known only to the developers that adds technical debt.
"Let's redesign the home feed. Should be easy, right?"
Software just rots like a smashed watermelon in a tropical jungle these days if you aren't actively developing it all the time.
You can't get off the treadmill of updating and paying attention, because if you take your eye off the ball for just a few months, you wake up with outages because some service you depended on has deprecated their API and no longer support the way you were calling it before.
" they always, always start to request more features."
It's not even that they'll request things in the future. There's often a backlog of stuff that will be coming up. I've still had to fight on people pushing back with "quit 'overengineering'! YAGNI! etc" comments.
"Yes, we ARE going to need it. You agreed to it 3 weeks ago in meeting X! Just because it's not in this week's workload doesn't mean it will never happen. We're slated to have to do XYZ in 5 weeks - we understand enough of the problems that will face to be laying some of the groundwork (some extra refactoring/cleanup today in prep for work that will come down the pipeline 5-6 weeks from now)."
But... that will impact the schedule for today so... ignore it.
It's not always so blatant, and current projects don't suffer from this approach nearly as much as some past ones, but it happens.
You have to first identify the percentage of developers that fall into the over-engineer class. If it’s 30%, then it’s possible 30% of your team will over-abstract, and if you make them a principal, they will over engineer most of the codebase. These are scary people, because at least the duck-tape ‘just get it done’ people write sloppy but simple code (always needs to be cleaned up, but this is a lower load cognitively than unraveling the mental models of the over-engineers). Both can test one’s patience.
To keep running with the analogy, the sloppy developers will just take a credit card and max it out. The over-engineers will do credit default swaps on mortgage backed securities and tie up the whole economy.
Great extension to the analogy! In my experience over-engineering class are the most damaging of all and I feel like the article and general discussion of technical debt largely ignores this.
I'd also add two subsets of the over-engineers. One subset are those that create frameworks to solve problems vs. address the problem directly. The other are those that introduce (language / datastore / build tool / etc.)-of-the-week to a codebase in an ad-hoc manner. And I'll admit, I've been both of those once upon a time...
Business usually values speed. If you over engineer or just duck tape everything, so long as you are fast about it, you’ll be on the short list to be a principal.
Add to this that many developers/engineering managers are trying to virtue signal to business leaders about priorities. Unless your business leaders grok code development, that translation layer is always rife with grifting aiming to pad resumes and career trajectories over serving the organization's need.
In a recent informal pointing session I suggested we attach a volatility score in addition to a point score to tickets related to an ongoing project.
The goal being, by including a point estimation AND volatility score, we are able to create much better signals for unknown unknowns.
A ticket pointed at a 3 with a volatility of 5 (volatility being 1-5) indicates that this should be an easy ticket but might require a much deeper amount of work for unknown unknowns.
I got this idea after learning about options where price is partly a factor of a security's volatility and thought it would also be an interesting way of modeling tickets.
Lately my biggest problem with technical debt as been that it accrues even if you do nothing to invite it. Maybe a better term than "technical debt" in this case is "deferred maintenance". It's slightly different, in that it doesn't happen as a result of decisions and compromises during the development phase, but the similarity is that it grows more costly over time.
Right now I have running on servers that I manage several old websites. One is a static site but it has content that has become broken over time. Flash animations and videos being the primary culprit. Another is a Drupal site running on an unsupported Linux release. These were developed many years ago, and met all their goals at the time. The people who developed them moved on. There were no requirements changes, no functionality that needed to be added or refined, the sites just simply needed to run as-is. But they were built on shifting sands. Operating systems are patched and updated and eventually EOLed. Frameworks like Drupal get patched and updated, and later versions are not backwards compatible but the prior versions are eventually not maintained, so there is significant work imposed upon site owners just to keep them running and safe. The same is true of any application framework/stack. Nobody wants to do the work to keep these sites updated, there is always a higher priority, but nobody wants them shut down either. So they stagnate, accumulating problem after problem just by existing, until they eventually collapse due to a critical failure that can no longer be repaired. Just like the house that eventually collapses due to deferred maintenance, the only solution at that point is an expensive rebuild.
Ironically the only computing environment I can think of that has largely sidestepped this is the mainframe. Yes the stodgy old corporate tech from the 70s that no self-respecting modern tech worker wants to touch is probably the only place you will find code written by your grandfather still running unchanged in production.
In the worst case, debt is a trap that can compound and be difficult to come back from. In the best case though, it can be a very useful tool. Every homeowner who takes out a mortgage and gets to live in their home with their family for 30 years before they've fully paid off their house is effectively and productively using debt.
The one thing I would add is sometimes tech debt needs to be paid off at unexpected times. There have been several instances in my current work where a piece of the application broke due to poorly written code, and refactoring was an immediate need, no matter what else was going on.
I imagine home loans would look a little different if there was a small percent chance on any given day that the bank would show up and ask you to pony up.
But in general, debt can be good given the right agreements in place.
If you were to compare it to a homeowner, tech debt for the average startup is more like someone deciding to build their own house with neither relevant skills, planning permission nor expert advice and desperately trying to keep the whole thing from collapsing into the foundation while simultaneously building balconies and extensions on every floor (think of the worst episodes of that Grand Designs TV show).
> Every homeowner who takes out a mortgage ... is effectively and productively using debt
I understand taking a loan from a business perspective, e.g buy more tractors for my farm to increase productivity. But how is using debt to buy a house increasing productivity?
In the monetary sense the the farm productivity may equate into growing more of something you can later on sell. In the case of the house, it equates to gaining an asset (or at least equity in an asset) that you can later on sell, also.
But in a more abstract sense it's not all about money - it's about getting around roadblocks, for a cost. For example, being productive can include things like getting stuff done that you've been neglecting. Maybe a person bought a house to have space to practice yoga at home or something like that. They're knocking out things they've wanted to do but couldn't before. Also maybe increasing the demand of housing in your community by buying a house makes the local construction industry more productive.
We don't know the future. Since that's not going to change, we need to stop blaming technical debt on the repercussions of that fact.
We want houses before we can save enough money to afford them. We want applications before we know how many users will be slamming our servers. The common denominator here is time. We're trading in time - the only question to ask is whether it's being done for the right reason.
The wrong reasoning is usually something like "it would take too long to understand how to do it better".
When a better solution is understood to have obvious merit but nebulous short term applicability - that's probably good technical debt to take on.
Too often when I see this claim it’s used to defend the opposite idea - that we should assume that the future will continue to look like today.
A good engineer should be thinking about robustness and flexibility from day one. Many choices we make early on close off options in the future or make them achievable only at great cost and at the expense of other business goals.
By this I do not mean “future proofing” or trying to account for all possibilities or to solve for specific scenarios that do not yet exist - those too are traits of an inexperienced engineer.
One of the causes of such technical debt is turnover. I've been in teams where the headcount has stayed the same for 10 years and technical debt is kept low. I've been on projects where people are continually coming and going and adding functionality gets rewarded - this leads to the most debt as you get nothing from cleaning up code.
Personally I think the concept of Technical Debt is more obfuscation than it is really valuable. I know that there is a whole cottage industry of people helping organizations deal with it, but so be it.
To clarify my position I wrote this article below. One aspect is the fact that circumstances and thus requirements may change. This is not a misunderstanding or tech debt. This is just what it is, no need for any metaphors.
I don't think it really makes sense to try and lock down a "true" meaning of the term. It means so many different things to so many people that any discussion about it inevitably becomes mired with emotive meaning ("I hate this code" "Me too") or arguments over what truly "counts" as technical debt (this article included).
Unless you focus the discussion (e.g. on test coverage), which includes tossing out things that could arguably be "tech debt", most discussions break down.
Thought experiment: if you do a quick and dirty code for a feature that never gets used (or dropped). Is that technical debt? I would call it a technical surplus. You didn't waste time on a feature with little business use.
Also, what if the business cost for slow development causes a negative impact? Sure the code and you resume look good, but your startup just became vaporware and ran out of money.
That is the problem with the tech debt analogy. There are many times you can get away with dirty code with no costs.
I wasted time before cleaning up code only for it to be dropped or for the code not needing any additional updates for over 5 years.
You can't always anticipate tech debt in advance so it often better to leave the quick and dirty solution and revisit if it becomes painful to update in the future.
It's wasted effort if it never gets used, but it may be a valuable learning experience for whoever made it.
It can become debt (or a liability in a more general sense) when you realize, a year or two later, that it's what you need but it's no longer compatible with the base system. So now it has to be cleaned up to be reintegrated properly, is it worth the trouble of the fix or should it be scrapped and restarted?
Or, worse, if it is in the system but disabled by something like a feature flag, it helps to ossify the underlying system because you now have something which depends on capabilities you want to change, but can't because it's in the system.
In a statically typed language this means care is taken with your core system to protect a feature that's not in use, but prevents compilation if you change its dependencies. And then you have to determine if the dependency change should be kept or not, if the unused feature should be kept or not, and if kept how to update it to use the changed dependency.
In many dynamically typed and interpreted language this means your core system is not stuck, but when you finally flip that feature flag on you'll realize you pulled the rug out from under yourself. Unless your tests are sufficiently comprehensive to trigger the error after a breaking change in the dependencies.
> Technical debt is caused by a lack of understanding
Engineering in general usually involves a trade-off. If you want a really fast car, it cannot have air conditioning. But also, if you want the car built in 12 hours, the engine might overheat every now and then. The point is, you can have technical debt and have understanding between business and engineering.
Tech debt is deferred value, the cost of which tends to exponentially increase. "We could have feature/efficiency of <some high number> if we spent <high time/people/money cost>. That cost is too high right now -- we have a deadline, we have been given a specific budget, etc -- so we will settle for less value since we believe it to be more affordable. Doing this will likely only make the cost of what we are deferring even higher, but it's the best decision we can justify at the moment."
Tech Debt is a rational tradeoff, but when all decision makers are incentivized to only care about the short term, then it quickly becomes an unassailable barrier to future growth.
Except you are talking about hypothetical technical debt, not actual practice.
In theory you are right, tech debt is a tradeoff.
In practice what is happening is that people decide not to pay attention to design at all and just throw away every single "best practice" with the excuse that there is a short deadline.
Many cases, this short deadline is entirely illusory, especially in large corporations. Many times I just ignored the deadline on my own judgement and everything came out fine and everybody was happy.
Most of the time, in a crunch, you can still follow most of the good practice and identify only small number of things that are just too time-consuming to handle now. You can then plan so that it is easier to fix later.
For example: when in crunch, I tend to focus on overall design and APIs and try to concentrate the tech debt in implementation. It is usually much easier to fix individual modules when they have good APIs and interactions with other systems. Fixing bad design/architecture spanning multiple systems is much bigger issue and usually requires touching implementation anyway.
But that's not what usually happens -- I haven't noticed people actually planning technical debt.
Another issue with technical debt is that there isn't anybody to chase you to pay it off. At least with an actual loan there is a bank that chases you. This aspect is underappreciated -- I personally think it should cause us to err on the side of less tech debt.
imagine what kind of decisions are being incentivized when using "Agile Scrum" methodology that requires you to push new stuff every sprint and refactoring always get postponed because it is not on a roadmap
In my opinion, the biggest source of technical debt in our industry is HR.
Design-by-HR has led to so many awful, awful projects, endless grind, and disastrous architecture.
Technical organizations which dissuade the HR power grab, usually have a lot better technical management and therefore architecture - and the technical debt is minimized. Technical organizations which defer to HR-derived architecture decisions are doomed to become a churning quagmire.
HR people: you've got to learn to stop interfering with technicians and engineers, because of your career choice. You're really not needed by organizations that build their technical resources properly.
Sure, tell that to the HR authorities who are insisting on only hiring one 'type' of developer for the currently exasperated teams, who then have to do things that types way, or else - even when half the team already does 'type' and knows why we should stop doing it.
I know its not a popular opinion, but 30+ years of experience in the software industry, from top to bottom, including stints in HR projects, has led me to conclude: HR is never senior to technical staff. Ever.
HR is not your friend. It is an organizational adversary imposed, usually, by executives who cannot make technical arguments or see things through to completion, no matter the depth of their understanding of the subject in focus, or otherwise.
If you are a technical person, never accept career guidance from someone who cannot do your job. Period.
Technology is about action, and doing things. Its not about identity. HR in the modern zeitgeist, from big corp to startup, typically avoids having to participate in any action of company product-building by instead promoting identity-as-service.
That this gets inverted is a huge cause of technical debt, imposed by nonsense policies and scripts that do not have a place in technological realms, but rather drama theater.
Then you would be inferring that as a professional, I would be incapable of making an objective assessment of the teams I have been asked to manage, recruit for, rescue, reform, refactor, or disband .. over a period of 30 years.
But that's okay, I could as easily dismiss you with the rationale that you are probably a HR person, as you seem mostly interested in psychology and not the very real, counter-productive problems one might encounter were one to work in the technology industry.
As with any metaphor, technical debt has its' limits and its' traps. IMHO the biggest problem is that a comparison with lending implies that technical debt can be managed like people do with a loan. With loan you know how much of money-equivalence you take, how much you owe and how much interest you have accumulated on top.
Most of measurement - if at all possible - is much less obvious with technical debt. And inability to easily quantify it, opens up door for corruption.
> And inability to easily quantify it, opens up door for corruption.
Yes, and since it's not readily quantifiable, any notion of something analogous to a payoff schedule is not possible. This makes it difficult to make a concrete and specific case for doing code cleanup or debt paydown often just before it's absolutely necessary.
Technical debt as disagreement is an interesting take. The pragmatic discussion between two members about practical debt is rarely a contentious thing. If you moved quickly or flexibly to get a piece of functionality up with obvious corners cut, we all recognize it and have no problem with it mostly because the fix is also obvious (as obvious as why it happened in the first place).
If you challenge someone’s larger architectural decision that is the root of most of the problems, now you’ve entered dangerous territory. Usually, the defense will try to reframe the issue as ‘practical debt’ of the kind I described above. ‘We needed to do this, we thought it was the right way to go, you get it right?’ - not really, no. Those decisions are handicapping the team.
Technical debt is the number one contributor to quality of life imho. Good design that isn’t over engineered on one extreme end, or duck tape everywhere on the other extreme end, can help developers withstand everything the project throws at them.
When a developer can easily understand and make meaningful changes to your codebase within an hour or two on any given day, they can withstand any type of siege (business shifts, design shifts, bugs, users, life). When the debt is too big, any one of those things can break the technical debt dam. It’ll be obvious when simple design changes can’t be made, or when a developer can’t adjust their workload down (will appear slow), or when new developers take time to make an impact (will appear dumb), or when the business changes and the product needs 5 new features that get push back (business will perceive friction).
If we agree to disagree on tech debt then we’ll all pay the price.
You cannot just shoehorn this expression. It is overloaded and depends on context:
- fixed price was paid, software delivered, nobody else cares == it is not a debt;
- fixed price was paid, software delivered, but more potential buyers == tech is most likely in a debt as you'll need to accommodate more changes;
- subscription based income, software needs constant improvements == tech is most likely in a debt;
- VC based income, software needs constant improvements == tech is most likely in a debt as you do not know what you're building;
- no income, software delivered, people want more == dunno;
- no income, software delivered, devs say it is crap == no tech debt;
I think debt should be tightly linked to the potential financial gains. But not only it is hard to quantify it, developers are not usually participating in the financial projections. Because of that you get a speculation on the importance of various refactoring exercises that do not yield any income improvements. Besides developers are usually biased towards improvements because otherwise they (devs) are not necessary anymore.
An outrageous example. Imagine you are painting a naturmort. If you're not careful enough and do not do it right from the beginning it might end up being shit and beyond repair. But who cares if nobody buys it.
Despite having 'debt' in the name, it isn't directly coupled to financial incentives. An internal team that produces tools for other teams within the org can have tech debt.
My version of tech debt is anything about the design, implementation or operation that isn't meeting the current needs and is not readily adaptable to meet them--an impediment/mismatch in the original design or implementation to what would be a suitable design/implementation now.
Software is always going to have mistakes and there is always going to be a better way to do things. That said, there are best practices that help to reduce the cost of those inevitabilities. Technical debt is when people throw those best practices out the window which leads to the inevitable outcome of a project that becomes extremely expensive to maintain. If it goes on long enough, and gets bad enough it might become so expensive that the team needs to declare technical bankruptcy and abandon the entire thing.
Another common thing I've seen with most projects that take on significant amounts of technical debt is a chaotic team structure that is constantly in fire fighting mode. While there are times when you are working on a throwaway/POC/low funded MVP for a startup where it might be appropriate to take on that debt, I have seen far more instances of people using the rationalization of "All of these practices are great, but (we don't have time, money etc. etc. etc.)" when in reality, to achieve the end goal everyone is looking for they are actually omitting non-optional things and shooting themselves in the foot.
The real cost of constantly having to firefight: people will inevitably leave or demand a higher pay to stick around. No one wants to be the one wasting 3 hours because a design pattern was poorly / wrongly implemented, or doesn't realize they need to downcast what is essentially a data object, or whatever gotcha the people making it before decided to implement.
It doesn't help further one's career. It feels bad, like a dead end. You're always far behind the curve, and you always feel like a third class citizen behind the whims of management. It goes directly against why most ambitious software devs became a software dev in the first place.
I'm somewhat familiar with McCabe style complexity measures[1], but I'm curious if there's a metric that measures a code base's ease with which it can be refactored or extended. Has anyone heard of such a thing?
I like that definition: the code might be fine as far as code itself goes, but it's still slowing you down as you iterate on the product. Decisions that were somewhat right at the time have slowly started go wrong as new ideas, requirements, and knowledge arrives.
On backends that comprise a database and an API layer on top I find that the sweet spot of speed + iterability can be found by deconstructing the product into its atomic parts. This boils down to modelling the database so that is as close to "reality" (as seen from a business/legal perspective rather than the product's perspective.) as is comfortable.
Reality is somewhat immutable (compared to a product.) Deconstructing (not just necessarily normalize) the API entities into small parts that reflect some "real world idea." The product is an abstraction of "reality" and that abstraction may change, but the underlying parts do not.
Of course once performance comes into account things may be different, but speed is rarely an issue when we start.
* Technical debt is a result of a mismatch between technical architecture and business needs (this comes from Ward Cunningham, the originator of the concept, the OP says)
* OP wants to analyze this mismatch in two main categories:
1) The technical architecture was wrong from the start, as a result of not understanding the business requirements (domain) accurately enough. He calls this "wrong design". (While the OP doesn't say so, I think you could further subdivide this -- you thought you understood the domain but had some misunderstandings; or you knew you didn't understand the domain sufficiently, but had to or chose to proceed anyway to be good enough for now).
2) Originally there was a good match between technical architcture (design) and business requirements (domain), but the environment changed, you have new requirements/demands that didn't even exist in original implementation. He calls this 'rapid evolution'.
In fact, I think those categories can overlap -- the landscape always changes, part of good design is attempting to predict some of that change. This is the trick with software design in general, you are always trying to plan for future needs, and trying to predict the future is always inexact.
Still, these categories seem useful to me conceptually anyway, even if not an exact bifurcation. Some technical debt (design problems) seem more like one or the other.
OP says for the first category "bad design", you reduce it by spending more time understanding the requirements/domain.
For the second category, "rapid evolution", you write code that is easier to refactor. Ah, sez me, but there is the rub, that's exactly what's hard, writing code taht's easy to rrefactor! Especially because it requires predicting the future itself, everything is easy to refactor on some dimensions more than others, requires trying to predict the tthings likely to be invariant vs the things likely to change.
Still, I think these are some good guidelines, it's not really one or the other, think about how much you can invest in both these paths -- understanding your domain better (which really means what kinds of changes are likely in the future), and making code easier to change, even in ways you didn't predict.
A better term than "debt" might be "inertia", technical inertia. Every line adds inertia to a code base, because it adds complexity. Some of that inertia is necessary, and some is accidental. As context changes, or someone attempts to put the code to a secondary use, the inertia becomes apparent. But the perception of inertia changes depending on how much present demands on the code differ from past demands. One person's debt is another's well-oiled machine.
Rust is good, too, because it alludes to external factors that affect the code and decrease its usefulness through no fault of its own. There is no stainless code...
Re: inertia, a lot of old code has no speed at all, which is kind of the point here. Inertia implies stasis as much as speed. If you wanted to emphasize speed, you could use "momentum", I suppose.
But I think the idea of inertia, and inert code that requires energy to change, is a more useful analogy than debt.
"A system has technical debt when the effort needed to keep it fit for purpose is surprisingly high. Zero technical debt means that according to our current knowledge, cost of foreseeable future work and maintenance is the lowest possible."
A good approach to avoid that kind of technical debt is domain-driven design (DDD). It helps to express concepts discussed with stakeholders directly into the code base. Thus the chance to make errors when translating the customer language into the technical language of developers is reduced. DDD could be seen as the continuation of requirements engineering, since it bridges the gap between specification and code. Both, the specification and the implementation would use the same ubiquitous language.
I should know better than to enter a debate of semantics, especially one that kicks off with something as paradoxical as "highly efficient vectors of meaning".
However, those dirty practices that the author dismissed really do pile up and drag you like debt. If you don't deal with them, they get worse until you bust. If those things aren't technical debt, then what are they?
There's nothing wrong with technical debt, per se. Every non-trivial body of code has it, and like real debt, it's an instrument that can be useful. The important thing is for all stake holder to be aware and comfortable with the inherent trade offs. At least the ones that are known.
So, I think a far better way to think about what is called Tech debt is as Maintenance Load. Chelsea Troy has a series about thinking about Tech Debt from that perspective. I also think that the way we often talk about tech debt ignores the fundamentally human nature of software development.
All projects that follow economic constraints have technical debt. It's meaningless for the silent majority of the industry that delivers product and maintains them only when the customer pays for it, otherwise never touches them again.
I liked the coinage of the term "clean to refactor". Which to me means it is easy to reason about the effects of deleting or rewriting a piece of code because it is not tightly coupled to a bunch of stuff subtly or otherwise.
This has too much on how technical debt is generated and use a poorly selected set of quoted terms to reach a bad conclusion.
Technical debt is the ongoing price you pay for making trade-off decisions in design and implementation.
Code management trade-offs? I hate to see all those day.
Quality trade-offs? Yup. Customers tell me about those.
Security trade-offs? Fix those when you find 'em before bad guys do.
Requirements trade-offs? Maybe I can change requirements and package up some of that debt as a requirements-backed security. Go for it PM.
Some come from hastiness, some come from lack of skill. Some come from changing environments and requirements.
Good news: you can avoid all technical debt if you do nothing.
Bad news: all decisions are trade-offs.
I've still seen what I consider tech debt just poorly written overly complicated code. This post seems to ignore the experience and skill set of engineers writing the code.
I think you're both right and that GP presents a decent duck test. That set of problems for which, upon inspection, I think why didn't we/I do this right the first time?
Though there isn't a Right way, there are right ways in the sense that they have undesirable effects that are less severe and slower growing than the wrong ways.
It's an interesting article, and I appreciate the readability (good formatting, images, etc.).
Personally, I am absolutely against any kind of debt. It has served me well, but has also given me a humble lifestyle. On the other hand, I worked for a 100-year-old Japanese corporation that was (and still is) ferociously debt-averse. In my own life, I feel that my debt-aversion has been a great asset. I'm not so sure that debt-aversion was as good for the corporation I worked for.
But then, "good" is relative. They are 100 years old. That's pretty badass. They got to be 100 by being risk-averse, and building on a robust foundation. But they are also relatively small. Their aversion to debt meant that competitors passed them by, in Ferraris, while they were chugging along in their trusty Volvo.
Often passing flaming, expensive wrecks...
But I digress. It also made it a huge pain to develop software for these folks. I feel as if there was so much design (that yellow curve), that the software suitability suffered.
I consider tech debt to be anything that we say "We'll deal with that down the road." It's really that simple. It may be high issue counts, inflexible design, usability issues, resource usage, cost to maintain, etc.
The word "debt" has a lot of shame attached to it. When I first got married, we needed to use a significant portion of our wedding presents to pay off the credit card debt that I brought into the relationship. That was something that caused me great shame, and was a principal motivator for my "live humble and debt-free" philosophy. I have never carried a balance on credit cards, since. So I have a ridiculously high credit score, but I probably would have a hard time actually borrowing money (so I'm told. I haven't actually tried -the mortgage for my house, and leasing cars has never been an issue).
But debt can also be used as leverage to grow, or do more.
"Tech debt," as the author states, is also often assumed to be the result of "poor code." There's a lot of really badly-written code out there, but much of it is relatively debt-free, due to being tested, encapsulated, and well-maintained. It wasn't allowed past the velvet rope, until it had cleaned up its act.
I've also shitcanned dependencies that had well-written code, unit tests, eye-candy Web sites, and great presentation, but suffered serious bugs.
To me, "debt" also includes things that are not necessarily "loans." For example, stock purchases or investment is not always done as a "loan," legally. That said, I still consider it an obligation (so..."debt").
That's why defining tech debt as anything "that we'll handle down the road" works best for me. Covers a lot of ground, and removes the emotional baggage.
Usability and localization are some things that have resulted in a couple of bent-nose guys telling me how it would be a shame if anything would happen to my kneecaps.
Usability, in particular, is a killer. I have designed, implemented, then removed, some pretty fancy widgets, because they proved to be nice on paper, but unsuitable to my needs in practice. An example is my RVS_Spinner widget[0]. It's a solution, in search of a problem. My RVS_Checkbox widget[1], on the other hand, has proven to be marvelously useful.
When it comes to usability, in my experience, there's absolutely no substitute for actually putting the code out there, and letting people play with it. Early beta-testing is crucial, and I have found that it's also important that the code be full ship-quality; not lash-up-test quality. That's because, if Management likes the demo, it becomes ship code; whether or not I want it to be so. The downside is that, if rejected, I toss out ship-quality code. Sometimes, I may keep the code, and polish it up for future use (that's one reason that I have so many small module packages out there).
Localization and security, in my experience, are things that are easy to manage, if baked in from the start, and a fang-toothed nightmare, if added after the fact. This means that not dealing with those from the start, is a fairly obvious example of "I'll deal with that down the road," that I can live to regret.
Since leaving my last employer, I have been experimenting with what I term "ultra-agile" development techniques. I despise tech debt, so I am trying to figure out how to get "the best of both worlds." The results, so far, have been promising, but my scale is, by necessity, fairly small.
First objection: Managers and Business persons understand Debt, like it and use it. I use credit cards and have a car loan. If we, as engineers, present the problem as "You can have what you want now, but you will incur Technical Debt" then any sane person will say "Sure! Good deal!" Ask yourself, how many times a manager or business owner has asked "What is the interest rate?" Zero times.
Even if you really want to call it Debt, don't. Even if you could convince me that Debt really describes the problem, I would suggest that you don't use that term. You are using it because you want to better describe a problem and reach an understanding, and achieve a goal. However, the managers will listen to what you said, agree that it describes the problem, and the proceed to go in the exact opposite direction that you wanted. Calling it Debt does not achieve your goals. You can keep calling debt because it's oh so clever a phrase, but see where it gets you.
Second Objection: It's not debt.
Debt would be if someone said "I need to build a bridge across here that can handle an Army, and I've got $1m", we engineers reply "It will take $2m", and they respond "Ok, I will borrow $1m so you can build the bridge I need".
Instead what happens is they say "Well, build what you can for $1m", and you say "Ok, we can make 'a bridge' for that", and then either (a) your infantry can cross, but the tanks have to get diverted 20 miles out of the way, or (b) the tanks end up in the river along with the bridge. Since (b) is bad, you then have to spend a lot of time planning the routes for the tanks, and making sure the tanks have the right air cover, etc etc, i.e. doing more work. More likely, however, is that the manager, who is not a structural engineer, sees a perfectly good bridge and orders the tanks across anyway, causing the loss of the bridge, the tanks, and the war.
It's not debt. It's just (at best) an incomplete solution or (at worst) a bad solution that fails at the worst possible moment - e.g. database collapses during registration for the largest event of the year.
Ah, but surely, if you build the lightweight solution for $1m, and acknowledge the increased costs of managing the problems that it doesn't solve, then thats fine? Sure, but that's not technical debt either! That is scoping: we (engineers + business) identify a workable solution that provides some business value. And then we do that well. When Cunningham, for example, talks about what to do about debt, e.g. YAGNI, these are all good ideas: scoping. But the term "debt" is incorrect in this case. The term "debt" will only ever get a team in trouble as it indicates that the team is unwilling to confront the actual problem as engineering, but only as a broken metaphor.
https://www.youtube.com/watch?v=XakfJ2spb3w
tl;dw: Technical debt is equivalent to the interest you are paying on it, where interest == pain. Some debt is planned and low interest, like a mortgage. Other debt is unplanned and high interest, like a credit card or payday loan. Sometimes that interest allows you to get larger returns. For example, if you could take out a loan at 3% in order to invest at 8% over the life of the loan, that's a pretty good deal, as long as you can pay it down. but, if you let that interest get out of control, if you don't pay down the principal, then it becomes more painful.