Feb 5, 2016
The inability to conceptually and quantitatively clarify how to map financial concepts captured powerfully within the financial debt metaphor to software hinders our progress in research and application of technical debt in practice. In particular, there is increasing conceptual mismatches in articulating and calculating interest. Consequently, the tendency to equate any code quality issue that a static analysis tool may find to the principal or interest of technical debt in the software realm puts us on the wrong path.
In this note I will map interest and principal to their economic foundations and argue that a fine tuned quantification of technical debt requires defining their counterparts, in some cases completely dispensing them. Let us start by recalling the financial definitions:
Debt – in its simplest form, a financial debt is an obligation to repay an amount you owe. Repaying the amount means repaying the principal and the accrued interest.
Principal – initial amount borrowed
Interest – money paid at a rate or as a fixed amount for the use of money lent
Interest rate – the percentage of the principal to be paid, often regularly until the debt is paid back in full, as compensation for the loss of the initial borrowed amount value
Variable interest rate — an interest rate that fluctuates over time
Period – duration that the debt is paid back, hence interest is calculated against.
Steve McConnell’s technical debt definition has been the one that stuck with the community; a design or construction approach that’s expedient in the short term but that creates a technical context in which the same work will cost more to do later than it would cost to do now. Cost more is a proxy for the interest. What we are borrowing, the thing that we have an obligation to repay, is the construction approach often referred to as a short-cut. We are committing to “pay that back” by calling that approach “technical debt”.
In the literature there are two classes of technical debt quantification, repayment strategies if you will, exist.
Lump sum repayment – Lump sum repayment typically takes a code quality lens. This approach looks at the entire software and equates all the code quality violation found as technical debt principal and interest. While there is acknowledgement that all issues found are not of equal importance, the resulting lump sum quantification does not provide enough context for project managers or team members to decide how to respond.
Variable repayment – Variable repayment focuses on the rework, accumulated rework as often referred to. What is quantified for recognizing the accumulated rework are measures related to maintainability, such as defect proneness, time to add new issues in addition to redoing the initial design approach. This approach differs from the lump sum repayment as it scopes the conditions of the debt. The ideal repayment strategy here is doing the rework as soon as possible, i.e. pay back the principal, to avoid interest, i.e. additional defects. This approach assumes that the cost of required changes can be bucketed cleanly as rework of the original design + eliminating the side effects.
In financial debt, the consequences of just paying the interest is never getting out of debt. Principal in software has a very short span, it is only applicable during decision making. I argue that we never pay back the actual principal in software, we only deal with reducing or eliminating interest and principal gets factored into that activity. One-to-one mapping of the principal is often not feasible. Software changes, but not in the form of a backwards looking lens as in financial debt where when we borrow 5K we pay back 5K + the interest. Consider the following example, in a recent project that we studied the development team did not understand the requirements well enough and picked a UI Framework that was not best suited for the project. After several months of development it finally came to a state where they had to spend more and more of their time trying to figure out the root causes of the odd behavior of the UI framework that they did not foresee. Driven by this problem they decided to step back and rearchitect the UI framework, but since a significant amount of functionality had already been built they also had further knowledge about their expectations from a UI framework and picked a solution that they would not have gone with to start with in the first place. They definitely had technical debt and significant rework, but there is probably not much value in capturing it as principal + interest when they are in a position to make the rearchitecting decision. Given a one-to-one mapping does not exist, philosophically arguing principal quantification and articulation sets us back.
I argue that the goal of any effective technical debt management strategy is to reduce or eliminate the variable interest of technical debt, not pay back principal. Consequently, interest takes three forms and different quantification approaches are required for each:
1) Visible interest: interest already accumulated and currently visible
You have some code clones: a developer did some “cut-and-paste-and-modify” of a fragment of code, about 20 times throughout the system. But the software works fine. Identified classes that are highly coupled through a code analysis tool is also a form of potential visible interest.
2) Probabilistic interest: possible future interest (accumulation) that builds-up if the issue is not fixed.
In our example above, the UI framework has unanticipated behavior, but you have implemented it already and made it work one way or another. The question is what is the likelihood that as the system continues to evolve this behavior will cause other issues and will the team be able to continue development around it.
3) Sweeping interest: the design related added cost as a result of remediation (.e.g. removing additional patching)
If we decide to change the design we need to change the code clones in all of the 20 places (as articulated in #1 above) each of which may require additional design fixes, and create a utility to call the code from a single location. Or in the example in #2, if we decide to change the UI framework we need to ensure that we not only cleanly introduce the new framework and but ensure not to leave any of the workarounds related to fixing the odd behavior.
Cost of remediation does take into account principal. But since we have already established that it is most likely not the same work as the initial decision cost of remediation comes down to the following:
Cost of remediation = work on (sweeping interest + visible interest + new design)
Let us also take a look at the period, duration the debt is paid back and during which interest accumulates. In finance the period of the debt has an influence on the accumulated interest. The longer the period the higher the interest. In technical debt this too is probabilistic, the lower the probability of the changes executing the debt area, the lower the impact of the period.
Not all technical debt accumulates all of the three interests. By articulating technical debt in terms of these three kinds of interests as a starting point concrete payback and quantification strategies can be articulated. For example in the examples above, the visible interest is low because the software works, so the developers should evaluate the probabilistic interest. If for example they have only occasionally new features requiring changes in the cloned code, and when that occurs it is in one of the areas only they may articulate the probabilistic interest to be low, hence decide postpone remediation. In a different scenario, if the developers find that new changes push them to create new clones of the same code, adding to both visible and probabilistic interest, remediation may need to be done.
Solely focusing on interest and its probabilistic and deterministic forms can allow us to better take advantage of existing tools and develop an economic model for technical debt quantification and communication.