Technical pollution: an alternate metaphor?

Joseph Purcell, from Palantir in Madison, Wisconsin has another interesting metaphor: Technical pollution

Slides here: https://josephdpurcell.github.io/Lorax/dist/2015-madison-php/index.html#/
and a video of his presentation:
https://www.youtube.com/watch?v=eiMry8HRLQQ

Technical Debt in Product Lines – on the trouble of differentiation

Traditionally, technical debt research (implicitly) takes a single-system perspective. However, what happens in product lines? Is there also technical debt (you bet) – are there further forms of technical debt? Probably..

In our recent research, my group also addressed the notion of (code) smells in product lines. There is already some research in this direction, but very often from the perspective of: how do existing smells translate to the product line situation. We were more interested in the perspective: what are novel types particular to product lines.

What is particular about a product line is that it typically contains a variability model (describing possible features and their relations) and the code that is related to these features. For a specific feature combination that is selected then the corresponding code is extracted / generated / composed, etc.
We grounded our studies in Linux, a well-known reference case, as it is with about 15.000 features and industrial-size product line. (There code is C-code with pre-processor statements, while variability is expressed using the special language Kconfig)

So what are the kinds of problems that arise there? And are they REALLY technical debt items – or are they defects? (Kruchten argues very strongly on this blog – and I think correctly – that these two are actually very different)

Typical problems relevant for product lines are for example:

  • dead features,
  • dead code (or the inversion),
  • or where we was interested in: over-constrained code

But what are these beasts and if we use the following definitions

“Technical debt = non-user-visible issue in the system implementation that creates a problem for evolution”
“Defect = user-visible issue that leads to a problem for the user (unexpected / unintended) behavior”

are they defects or technical debt?

A dead feature is a feature that cannot be selected, although it is described as selectable. It is probably a safe bet this is a defect. For dead code, it is more difficult: this means that some code is not used in any variant, although it is described as variable in the code. This may have many reasons like added constraints in the build system or in the variability model that interact.
At first sight this may seem clear: certainly the problem is now not activated in some situations for which it was intended. This is surely a defect! – On the other hand, it may be due to the fact that these variants do not even exist any more, making it simply some code that is maintained, but effectively dead code, i..e, technical debt.
The situation of over-constrained code is even more complex as simply in some cases where one might expect this code to be present it won’t be. Whether this qualifies as a defect or technical debt requires a deep analysis of which code should be integrated under what circumstances and what would be required for correct functioning. We found in our analysis that there was hardly any difference in amount of effort spent between identifying whether s.th. had a user-visible effect (i.e., defect) or was technical debt and actually proposing a change for solving the problem altogether. Hence, while the differentiation between technical debt and defect seems rather obvious and intellectually appealing, we also saw that it sometimes is of rather little practical use.

Apart from this specific question of technical debt vs. defect, I am also looking forward to discussing special forms of technical debt like product line technical debt or debt handling in software ecosystems.

Technical Debt – not to be taken at face value

The term “technical debt” has done a lot to help get acceptance for the issues and problems we are dealing with. – However, it is ultimately a metaphor, i.e., an analogy. Analogies are handy tools to get a first intuition about a problem. However, they are not a model of the problem!
So, ultimately, on the way towards a better understanding they turn from support to obstacle. The most famous example is the “atom = planetary system” metaphor. Initially it helped a lot, but over time it became clear that the analogy leads to very much inappropriate reasoning and had to be completely abandoned in favor of quantum mechanics to really understand and solve the problems.

Already now we see that a lot of elements of the “debt”-metaphor do not work [1] (and various people also mentioned the problems on this blog). To give only a few examples:

  • the notion of a principal does not really fit: you do NOT pay off the same debt. – In some cases you will actually never pay of the debt as it makes no sense [2]
  • interest payments: financially there is a fixed rate (typically), but in TD it only depends on how your further development goes.
  • there is no reasonable notion of risk-free, not because there is no possibility for an optimal design, but rather because what was optimal can usually only determined in retrospect when also future requirements are known.

So it seems a reasonable question to ask:

“Is the TD-metaphor (and the financial analogy) actually helping or hurting the cause of dealing with technical problems in development?”

Given the overall amount of work that was poured into this and the fact that still today many arguments are based on this analogy , it must be allowed to at least ask this question. What if we step back, completely forget about the term “debt” and ask what is it we really want to address from a SE perspective?
In some earlier work [2] I actually tried this and as a result was able to propose an approach for the systematic management of technical debt that effectively navigates treacherous waters like the lack of a meaningful definition of principal in TD, paying back being not mandatory, etc. – So from that perspective I am happy, but at the same time this work was still strongly influenced by the metaphor, so I think, we can surely do better.
I say this in particular, because conceptualization is on the agenda for the workshop, hence forbidding the term “debt” and any referral to the analogy for a day or two might actually be a healthy exercise..

[1] K. Schmid. On the Limits of the Technical Debt Metaphor – Some Guidance on Going Beyond. Fourth Workshop on Managing Technical Debt, Workshop at the International Conference on Software Engineering, 2013, pp. 4, IEEE, 2013.
[2] K. Schmid. A formal approach to technical debt decision making. Proceedings of the 9th international ACM Sigsoft conference on Quality of software architectures (QoSA), 2013, pp. 153-162

Price Sensitivity – The Missing Dimension

As I consider the different charts, models and diagrams most often used to explain the concept of technical debt, it occurs to me that they are missing an important dimension, which I will call ‘sensitivity.’ In short, does this item/problem/issue matter within the larger context of its (eco)system? This dimension was also missing from the static analysis tools I wrote and used earlier in my career, which severely impacted their usefulness, and therefore their usage.

In particular, I remember deploying FindBugs at Google [1], and the conclusion that the tool could not determine the impact of the defects it found because it had no context. Putting that context into economic terms, what is the price of an item, or perhaps the price of either fixing or ignoring it? We talk about costs and benefits, which to my mind are generally fixed quantities. But the price of a item is variable, and allows us to bring the powerful laws of supply and demand into the picture.

Supply_demand_1

“What’s it worth to you?” I asked myself. When I was facing 100,000 build targets with missing direct dependencies while attempting to disallow transitive build dependencies [2], my price was very low. Once I had automatically fixed all the easy cases, my price rose as the goal got closer. I priced the final few, most difficult fixes very highly, as the value of all the earlier work depended on their timely completion. Once finished, the build system permanently enforced the desired property, essentially making the price infinite from then on. The debt in question had been paid off.

So, I am arguing that we use some analogy of ‘price’ as the measure of what matters in the larger context, and let the ‘market’ set that price for each TD item. Some items will be quite price sensitive, others quite insensitive, to extend the metaphor. This will bring the social aspects of economics into our models, and perhaps cube the quadrants by adding another orthogonal dimension.

[1] Nathaniel Ayewah, William Pugh, J. David Morgenthaler, John Penix, YuQian Zhou. Evaluating Static Analysis Defect Warnings on Production Software. Proc. 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, ACM Press, New York, NY, USA (2007).

[2] J. David Morgenthaler, Misha Gridnev, Raluca Sauciuc, Sanjay Bhansali. Searching for Build Debt: Experiences Managing Technical Debt at Google. Proc. Third International Workshop on Managing Technical Debt, IEEE (2012).

TD Indexes – What is missing?

We experimented different tools providing some kind of Technical Debt Indexes, we tried to identify and analyze their main features and observe what is missing. I outline below some observations we done with Marco Zanoni on these issues.

Different measures have been proposed to address the estimation of Technical Debt, by analyzing the artifacts contained in a project.
The most recurring characteristics of the proposed measures are:

  • localized hints or issues, with an assigned score;
  • scores are then aggregated by following the containment relationships existing in the software, e.g., modules, and the aggregation operator is usually the sum;
  • scores are assigned to the single issues a priori and arbitrarily; default scores are setup by relying on the experience of tools’ developers and may be customized to adapt to the specific project’s needs.

Most hints contributing to the building of the measure fall under these categories:
1. coding issues regarding violation of best practices
2. metric values and violations to some defined thresholds
3. detection of more structured issues, e.g., architectural/code smells.

This approach is surely reasonable and motivated by practical feasibility, scalability (new knowledge about issues can be coded and added to the additive model), and manageability of the analysis (as pointed out by Jean Louis Letouzey in “Questions that are quite systematically raised about TD”).
The final goal of TD indexes is to allow the **Management** of Technical Debt, i.e., to allow developers (we use “developers” to mean all people designing/developing/operating the software at any level of abstraction) to understand that a choice (conscious or not) has consequences, which are usually non-functional, that can affect both developers (e.g., ease of maintenance and evolution) and users (e.g., performances, security). Given the knowledge about the **risk** derived by each choice, developers should also know how much resources are needed to transform their software in a way that removes or mitigates the risk. So, there are the two widely recognized aspects of the problem: the cost of keeping the system, and the cost of fixing it.

Do the current TD indexes allow estimating these aspects?

Since measures are implemented by summing scores that are assigned to each issue recognized by the analysis tool, the precision of this estimation is tied to two factors:
1. the precision of the single scores
2. the appropriateness of the aggregation model

1. As for the precision of the single scores, in all the models we know about, scores are arbitrarily fixed. They are fixed by experts, but they are fixed. Depending on the index definition, scores represent the cost/time of fixing the issue or its relative contribution (a penalty, usually) to the overall quality of the system. Both aspects lack empirical evidence, as other details like the thresholds applied to metrics when detecting, e.g., size or complexity violations. A more sound result would be obtained, in our opinion, if the maintenance costs and the impact on the quality of the systems could be fitted from empirical data, and customized per domain/technology/organization. This could allow choosing which issues are relevant and which are not, on a statistical base, and to obtain an estimation, e.g., of their *actual* cost of fixing, or to quantify the existing relations with maintenance times or defects.

2. As for the appropriateness of the aggregation model, when trying to estimate the cost of fixing a certain set of TD issues, one should consider that any change applied to a system has consequences that are not obvious. Software systems are structured as complex graphs, where each single change impacts every neighbor node recursively, both at design time and runtime. In this context, aggregation by sum is simplistic. Especially when dealing with design/architecture level issues, fixing an issue may remove an entire other set of related issues, or generate other ones. An ideal MTD tool should be able to understand these inter-relations and exploit them to suggest the sets of fixes that maximize the obtained quality, by following the structure of the system, and not a super-imposed estimation model driven essentially by quality aspects. Quality aspects are relevant, but should be used to understand which aspects of a system should receive more attention at coarse granularity.

Moreover, respect to point 1. , some issues may be underestimated due to their rarity in history. This is where another complementary approach could be used, that cares about rare (but not so much) issues that lead (potentially) to out-of-scale risks. We can borrow this approach from security analysis, where a very small vulnerability can disclose extremely important information. People working on these issues do not think in terms of tradeoffs, but try to follow practices that are proved and *do not allow* some bad things happening. When dealing with security, a lot of effort is spent to be sure that, e.g., a user cannot enter a system without successful authentication. That would have extremely bad consequences, which costs are exponential w.r.t. the effort spent in avoiding the issue.

This complementary approach consists in collecting which issues can have (even anecdotically) generated external issues that are “extreme”, like system shut down, data loss, project failure, and (when they are detectable) remove them from the project, with maximum priority. Do not even associate them to a score, because their effect is out of scale. If you multiply the chance of suffering from this risk by the costs it have, it is probably high anyway, but the problem is that if that happens no one would be able to pay that cost. This goes beyond the usual debt/interest metaphor, and resembles more how “black swans” behave.

**A possible research agenda

What we would like to see and work on in future research is:

  • estimating the relative relevance and/or absolute time/costs associated to hints/issues detectable by software analysis tools, with the aim of providing a TD estimation index with an **empirical base**;
  • collecting evidence regarding the root causes of known large-scale failures in both operations and development, with the aim of generating a blacklist of the issues to absolutely avoid in any project;
  • explore the existing structural and statistical inter-relations among different TD issues:
  • generating alternative estimation models that rely on the structure of a software, and that allow the simulation of changes to estimate with higher precision the effort needed to implement fixes and their consequences.

The Agile Alliance Technical Debt Initiative: First deliverables

A little bit more than one year ago, I get the opportunity to lead an Agile Alliance Program focused on Technical Debt. The purpose of the program (also called Initiative) is to provide practical recommendations and answers to questions like:
• What is Technical Debt?
• How do I make informed decisions on when and how to address it?
• How can I start to manage Technical Debt

We formed a group of six people and we had a kick-off meeting in May 2015 to organize our work on different objectives. Then, we collaborate mostly by Hangouts, as we are located in different countries on both sides of the ocean. We also had a workshop in July 2015 in Washington DC, to present and discuss our first results and share ideas and thoughts.

Some of the first results (available here) of our work have just been published on the Agile Alliance website. This is :
• An introduction to the concept of Technical Debt
• A set of recommendations: Project Management and Technical Debt
• The Agile Alliance Debt Analysis Model (A2DAM)

The last item is a very basic set of code related rules which should be the minimum to comply with in order to produce “right code”. This is not a large inventory of best practices, of requirements to comply with. As the opposite, it is a short and simple “ground level” set of good practices to help teams to start. All the rules are characterized by fields like impact, remediation model etc.

An initial version has been built by the AA Program Team. This initial list has been sent for review to experts in 16 companies developing static code analysis tools. 11 of them have provided their feedback about the relevance and clarity of the practices, the associated thresholds and their tool implementation. Their feedback has been taken into account for delivering the final version.
Now that this list is public, we welcome all kind of feedback (in order to improve and enhance it on a regular basis) and we hope that it will help the community to better manage technical debt.

These are our first deliverables. There will be more this year.

During my consulting engagements within…

During my consulting engagements within large organization, I meet senior managers and exchange with them on the topic of technical debt. I would like to share with you:
• The questions that are quite systematically raised by my contacts
• My current answers to their questions.
• My suggestions, my proposal about what needs to be done to make progress?

Question 3:
What can I do with all the Technical Debt accumulated in my legacy applications?

My current answer:
As you do not have the entire necessary budget to pay off your debt, you have to compromise. I propose to them an approach based on the use of two types of data for each application.

  • 2 external data: the estimated relative added value level and the estimated annual maintenance charge.

-2 internal data: The amount of technical debt and interest as measured by a tool that implements the SQALE method.

The principle is simple. If an application has a low added value and has no maintenance activity, then its technical debt has no impact and improving its debt is not necessary.
If an application provide high added value and has a high maintenance activity then its technical debt should be very low. This application must be given priority in the allocation of budget for improvement. This approach is applied successfully for some time by very large organizations. I have explained it in more detail in a recent article published in the Cutter IT Journal.
Here is a copy (please don’t distribute or copy for copyright reasons)of this article.

I will be happy to present (and demonstrate) this approach with more detail during our seminar.