3 Techniques To Measure Product Debt
How to quantify product (tech) debt to reach better decisions
How to quantify product (tech) debt to reach better decisions
Intro
I wrote about product debt in the past and why I decided not to call it technical debt. However, I still see how this kind of debt is treated in the R&D realm, and most Product people casually ignore it.
This is partly because it is hard to quantify and measure the debt and explain its meaning in business terms. Product and Business people should be aware of the debt and carefully and responsibly decide if/how to handle it (in opposition to arbitrarily adding technical tasks provided by R&D).
The below techniques can help you do exactly that.
#1 — Unused code
Like your dusty warehouse, you don’t touch much; code gets cluttered over time and is hard to navigate. As a garage sale, from time to time, it is a good practice to find unused parts of the code and clean them up.
Unused code can slow delivery due to added complexity, maintenance costs, performance, etc. Some frameworks offer such insights out of the box, while others require some additional work. I’m sure the AI wave will solve this problem as well.
Until then, one tool can spread logs around the code with sequential numbering (for example, “Area 5–68, send an email after cart abandonment). Over time, you’ll see which numbers are not logged, indicating they are not used in the code.
Once you know the above, you can quickly quantify the % of unused code in each product/service the team owns.
#2 — Developers survey
In every company, there are these legacy products that, once touched, developers show fear and frustration. Developers know which areas are messy and require extra effort to work on. Occasionally, send a survey to the team asking them to rate which products/services they prefer working on over others.
Analyzing the surveys over time can indicate the areas in the code that work well versus those that are getting more complex and harder to manage. These are the areas where the team accumulates the most debt. Given the scores for each service/product, you can devise a formula to give the final team score on their developer experience (which is a lead indicator for velocity).
#3 — Complaints and Errors
Customer-facing teams can provide information (using a ticketing or chat system) on the areas they suffer the most complaints. Similarly, the engineering team can point to recurring errors or locations that require more maintenance than others.
The above can be quantified or compared to find the two to three areas that suffer from the most complaints and errors. The reasons could be applicative bugs, performance, infrastructure issues, etc. Handling these areas can improve customer satisfaction and free up teams from the day-to-day repetitive tasks that come with it.
Not all debt is created equal.
The above techniques help quantify the amount of product debt. However, it has nothing to do with the impact of it. Let’s demonstrate it using a concrete example.
You have one area of your product with older versions of packages that use deprecated infra and is hard to work on overall. The team invested around one month of work in this area last year.
On the other end, you have a different area with a decent amount of unused code and degraded performance over time, and the strategy aims to release three new features within two quarters in this area.
Product people should prioritize the above to ensure the debt the team pays off is worth the company's time and aligns with its strategy.
To do that, I accumulate the debt and weigh it with the impact/plans for each product. Here is a made-up example -
Although the Checkout area has the most debt in the example above, given the company strategy and planned work, it might be worth tackling the Order History area first.
Summary
Debt is accumulated for a reason, assuming it will be paid later. Managing the debt is an important responsibility the Product Manager share. The provided techniques can help understand, quantify, and correlate debt to desired impact and strategy to make sure the team is working on valuable deliverables.