Technical Debt and the Lean Startup

A conversation on twitter between @david_harvey,  @rachelcdavies@RonJeffries and myself made me to think it was about time to post on my view of technical debt in a lean startup and how that view has changed since the days of running XP-based projects for established companies.

There’s a lot of information about Technical Debt out there on the web (including Ward’s Video, a clear summary by Martin Fowler, and an interesting, albeit somewhat flawed, article by Rod Hilton) and I’m not going to get into a detailed description here, but at its simplest level it says that making some quick-and-dirty choices today may lead to pain later on, and that pain is like a debt that you’re going to have to pay back at some point.

Its that ‘at some point’ which has led me to believe there is a difference between technical debt as applied to a lean startup vs. that applied to a project for an established business. In a project you are trying to deliver something which meets some kind of business case. You will know roughly who the customer is, roughly what their objectives are, roughly what you can spend (time and money) in order to achieve them, and so on. There’s going to be a lot of stuff that isn’t known but there are some constraints in place. When I used to use XP to deliver these kinds of project we had a very simple view of technical debt: don’t incur it if you can avoid it and, if you can’t avoid it, pay it down at the earliest possible time. Why? Because in these projects sustainability is a primary objective: you know you’ll have to pay the debt down at some point so why let it grow uncontrollably?

In a lean startup not only do you not know what your customers want, you don’t even know who they are, and you usually have less time and money to spend on finding out than most established companies would try and run a project with. Your primary objective is finding those customers and learning what they want, not sustainability. Which isn’t to say sustainability is unimportant, just that its not your primary objective; once you have a business, then you worry about how to make it sustainable. Its a High Quality Problem.

Here are some examples of technical debt we incurred at various stages of our search for customers and what they wanted:

  • Areas of the system understood by only one person. I’m a pair-programming fanatic and would love for everything in our system to be developed by pairs. But when you have only one developer in the company, that isn’t possible. When you have two developers and one has to go support a sales meeting, that isn’t possible. In fact, our experience is that until you have 5+ full-time developers there are going to be times when it isn’t possible to have everything paired. And do you know what? That’s okay. Code can be developed by a single developer but the fact that they built it on their own is a form of technical debt: until others have worked on the code, it is more widely understood, and has had the benefit of many eyes on it, you are in debt to the system.
  • Large scale refactorings left undone. We use a number of underlying platforms and tools. If the platform or tool doesn’t do what we need we develop custom code. It isn’t uncommon for the tool or platform provider to then release something that does what our custom code does at a later date and, until we refactor to use the new platform or tool feature, we have some debt to the system. But that’s okay – the custom code still works, its just not as simple as it could be nor as future-proof if the provider develops the feature further.
  • Limitations left in place. Sometimes a truly scalable solution is much harder than a simple but limited one. We had a situation where using in-memory collections led to a hard limit on the number of objects we could handle but the in-memory version of the code took a couple of hours to produce. We knew how to get around the limit – by adding in a system of creating permanent records and then processing them asynchronously using scheduled jobs – but that was several days work. The limit was okay – the function still worked, just for small collections of objects, and we had a debt to the system as and when our customers couldn’t live with the size limit.
  • Sub-optimal design approaches. Sometimes it is easier to deliver a simplistic design than a well-crafted one. For example we delivered a quick-and-dirty reporting function that was simple but slow because we were making it work, then making it right but hadn’t yet got to making it fast. But that was okay, the information was available to the users and correct, just not very quickly delivered.

But why would we live with such limitations? Aren’t we just delivering a shoddy system?

The answer to the first question is that we’re prioritising learning what makes our business work over some abstract notion of quality. If we develop a feature and it turns out that customers don’t want to pay for it, does it matter if it is only understood by one developer, that it could be refactored, that it is limited or sub-optimal? We develop the feature to get it into customers’ hands as quick as possible, to understand whether it is something valuable or not, to understand how it should work. Only if that features is valuable is it worth paying down any debt we have incurred building it. In fact, never mind the feature, the same applies to the whole system: unless the system is sufficiently valuable to customers we have no business and if we have no business the whole question of quality is moot. Customers pay for features and benefits, not fully-paired, beautifully factored, limitation-free, optimised code.

The answer to the second question is an emphatic ‘no’. We produce production quality, tested code. The testing is something we don’t compromise on simply because it is the mechanism by which we will pay down the debt if and when the time comes to do so. I can live with the fact that a design is sub-optimal if we have tests which will help us optimise it when we have to. I can live with code that need refactoring if we have tests which will help us refactor it when we have to. I can even live with something written by one person if … well you get the picture.

In a startup, technical debt is something to be managed, not minimised. We make sure we understand how much debt we have and which bits of the system it affects. We make sure we have the ability to pay down that debt as and when we need to. And we make sure we work the time and money required to pay down debt into any timsescales or budgets we agree.

Anything less would be irresponsible. Anything more is prioritising some notion of ‘code quality’ over learning what makes our business work and equally irresponsible.

Advertisements

20 thoughts on “Technical Debt and the Lean Startup

  1. Willem van den Ende

    Some of this also holds in established companies. Refactoring code that has no value is a waste of time.
    I had investigated some clients’ legacy code for refactoring opportunities, given to me by one of the programmers. When we eventually found a programmer who knew what this part of the system was supposed to be doing, we learnt that most of it no longer had value.
    So instead of going for the small refactorings we had identified before we knew the business value, it was more productive to find out how we could scrap large parts of it.

    Reply
  2. pauldyson Post author

    I agree, there are some established companies and projects out there that I think could do with acting more like a startup. Although I have to say its a matter of degree – no sane manager would fund a project as badly as many startups are and I do believe there is something unique about the circumstances a startup gives rise to: some of them very good, some of them much less so.

    Reply
  3. JJzD

    When you say these technical debts, can have business value, it becomes like a normal financial debt. Don’t occur it, until you can spend the money on something that will repay the money, the interest and the management/operating. Good point.

    But what will happen when you are near 5 developers? You will still be busy, so you can repay it later. When you have 10 developers, you still have to repay the debt, but the interest has accumulated longer than you envisioned. And probably has the original idea also added to the sum outstanding because a key developer left, or you had to build “temporary” workarounds. And this kind of debt is harder to track. Since nobody will knock on the door, asking for repayment. The bodies will be out of the closet on the worst moment.

    Too many companies have gone burst on their operations not matching growth and stumbling forward and faling eventually. So when your product/market fit aims to find the best combination, and reduce the risk, this will have to be compared to the risk of a future repayment or even default of technical debt.
    When properly tracked this is possible, but still involves many issues.

    Reply
    1. Richard Marr (@richmarr)

      > you still have to repay the debt

      The trick is that most software projects fail, not because of the code quality, but because they didn’t properly meet a need.

      You don’t have to repay technical debt on a failed project.

      If the cost of failure (i.e. the cost of innovation) is low because you saved time and cut corners then your business can take more risks.

      Obviously it’d be silly to apply these arguments to all cases, e.g. banks handling financial transactions, but as a general case they stand.

      Yes, companies do have scaling problems… but if all projects have to bear the up-front cost of building carefully designed and scalable systems then fewer projects will be financed and ultimately fewer developers will have jobs.

      Reply
      1. JJzD

        Allright, defaulting your technical debt because the software wasn’t needed (in that way) afterall, should also become part of the equation to decide if we incur debt or not?
        Because if we do, i have to assume that it will be very soon a good decision to do this.

        I am not a developer, but let’s assume the following ratio’s
        Doing a “fast” technical debted project: 0.75x
        Doing a “full project”: 1x
        Redo a technical debted project: 0.9x

        Projects which were useless: 3
        Project which was good practice needed complete redo: 1
        Project which was spot on: 1

        Scenario a: every project fully done:
        effort: 6x (three wasted, one project done twice, one project spot on)
        Result: 2 good projects

        Scenario b: technical debted projects
        Three wasted projects: 2.25
        two projects redone (good idea or not up to quality): 2 * (.75 + .90) = 3.30
        Total effort: 5.55x
        Result: 2 good projects

        Conclusion: seems like “playing” has a good result, and even has room for extra effort involved for tracking the “in-between” state?

  4. JJzD

    PS: sorry for the double post, but your definition of a HQP actually solves my problem as stated. Before i waste a couple of bytes, i’ll read your referrals first 😉

    Reply
    1. pauldyson Post author

      Thanks for the comment(s) – I’d rather have two posts than none ;).

      I’m sure you’ve heard of the famous book ‘Crossing the Chasm’ by Geoffrey Moore. Its not a bad book but where Moore describes ‘the chasm’ I see many chasms, of varying width and depth, that a start-up crosses every day, week or month. Making you first sale is a chasm to be crossed, making your first hire is another, getting to profitability another, and on it goes.

      Growing the team to a sustainable level is one we have yet to cross but its actually one I’m relatively sanguine about. I’ve run a number of largeish-scale delivery projects and this is my third company (the first two were consultancies rather than product companies) so I know I can grow – or shrink – a team and keep delivering. But sales; thats a unique problem. If you don’t have product market fit or can’t achieve it you don’t have a business. Every other problem other than that is High Quality.

      Reply
      1. JJzD

        I haven’t heard about that book, thanks! But what you call Chasm, would in the Theory of Constraints be called “limitations of the system” and more specific “bottlenecks”.

        I was thinking about your second comment, and paraphrased it as “If we can replace a sales problem by a HQ-problem, we should do it”.
        At first i agreed (the body also makes always sure there is enough oxygen in your brain, and will even take you down to make sure.) but on the second thought i am not too sure.

        Of course are “always” statement not something from reality, but for example Acme gave the same statement to their sales people: Accept almost any order. This put high customization into the orders, which the people delivering couldn’t realize, and created more unsatisfied customers. Bottom line: they lost more business through the customization, and satisfaction upfront, than doing “proper orders”.

        If we replace customization with technical debt, we can very well have similar problems. But both coming from a technical background, a bit more focus on the bottom line could very well be the current bottleneck we are breaking :).
        Not fair to put words in your mouth, but let me know where you would accept the sales domain problem.

        PS: LEAN works on the premise to reduce turn-around time, by reducing deviations and work in progress/inventory. This applies to the domain of production (convergence) and not the domain of creativity (divergence). Assuming that technical debt becomes part of the inventory (being managed, costing money lying around) it should be reduced.
        In that way i like the approach of William Pietri to do hacked experimental stuff and then throw away the result and do it properly in the production phase.

      2. pauldyson Post author

        A Chasm isn’t really a bottleneck, its a transformation you have to go through which may make the business or, perhaps more likely, break it. Moore’s book talks about the Chasm as being the transformation you need to undertake between doing what you do for early adopters (enthusiasts who are inclined to forgive any limitations in your product or business) and pragmatists (people who, by definition, won’t). But the book is written from the point of view of companies that are well past start-up and have already achieve some degree of sustainability. When I say that I see many chasms, of varying width and depth I mean that a start-up has to go through many transformations on a daily, weekly or monthly basis that could bring the business down. Going from the purely theoretical “if we sold this would you buy it?” to actually having it to sell has a few of these: what if you can’t actually build it, what if you can build it but it costs so much more to do so than those potential customers are willing to pay? As you start to grow your business there are many more: what if the first salesperson you hire is an arsehole and alienates your existing customers whilst putting off your potential customers (this really happened in my first start-up and it was a contributing factor to that start-up’s failure)? And so it goes.

        But, hell, if start-up was easy, everyone would be doing it 😉

        Your thoughts about customisation vs. ‘proper orders’ are very pertinent to us. We started with an idea for a mass market ‘shrinkwrapped’ product and pivoted to a much more niche product based on our experiences and advice. The niche we chose (investment banking) is one where people are used to having things ‘just so’ and also one where things are different at a detailed level from customer to customer even if they all need the same high-level features. So every sale we make requires some degree of customisation and, as you rightly say, this is another potential form of debt (regression testing each of our new releases against all our customers customisations for example). Here’s why we do it:

        * Customers pay for customisations. Revenue in the door is always a good thing in a startup even if you’re saving up headaches for later on (HQP)
        * The more customisation we take on, the more we learn about what a potential ‘shrinkwrapped’ product might look like; where we need to build in degrees of configurability so the product remains standard but can act differently for different customers
        * The more customisation we take on, the more we learn about what is truly unique from customer to customer and how we might handle those genuine differences in the future (those differences are a fact of our niche, you can’t just ignore them). Maybe we partner with consultancies who deliver and support the customisations, perhaps we do it ourselves but separate out professional services from core engineering?
        * As a tech team we’re pretty adept at building meta-data driven systems and the platform we chose also is a good fit. So the cost of dealing with those customisations is perhaps an uplift of maybe 15-20% over the cost of building a one-size-fits-all product vs. the perhaps 200-300% it might be if we ‘branched’ the product for every customer. That extra cost is currently more than covered by the extra revenue brought in for customisations; it is a net profitable activity so we are making money as we learn which is pretty much the holy grail for lean start-ups!

        But it does give us headaches on both the technical and the sales side.

        Just a comment on leanness; yes it does work by *reducing* deviations and WIP but not by eliminating them. You can still by your Toyota car in many different colours, with different engine specs, different trim levels etc. The trick is to reduce deviation to the point *the market will find acceptable* so you make the most amount of profit, whether thats a smaller number of high-value sales of a highly-personalised product (Aston Martin and Porsche) or very large numbers of sales of standardised product (Kia and Fiat). Of course the great trick is to combine the two as BMW and Mercedes do but I’ll settle for being an Aston Martin business (or even a Pagani business) to start with 😉

        Thanks again for your comments – great stuff.

  5. William Pietri

    Excellent point, and well put.

    We take it a little farther. When we are doing something pretty experimental, we’re happy to write hacky junk: no tests, ugly code, suspicious names, etc. When the experiment is done, we always throw away the junky code; if the experiment comes out in favor, then we build the feature properly.

    The only real trick is having the discipline to throw away the junk, but that’s a pretty big trick.

    Reply
    1. pauldyson Post author

      Thanks for the comment. Yes, we also do the same. I’m no big fan of wireframes or mockups to get feedback on experimental ideas as they are just as prone to customer/user mis-interpretation as any other abstract model of what will be delivered. So we build very quick prototypes, usually with all the business logic in Javascript, without tests, sometimes not really hooked up to the server, etc. and use these to have the customer try out the new idea.

      Interestingly, one of @RonJeffries comments on twitter was that he can produce a clean high-quality prototype just as fast as a dirty one. I’m not so sure about this (it may work for him, doesn’t mean it will work for us) but it may be worth considering how much extra effort is really required to make a prototype a usable starting point for the ‘real thing’

      Reply
  6. Pingback: What Is Technical Debt? » Absolutely No Machete Juggling

  7. Pingback: An economic model of technical debt? « Paul Dyson’s Blog

    1. pauldyson Post author

      Personally I’d rather leave the spin to the sales people (;)) but also for us techies to get real about what debt/credit is for. Debt != A Bad Thing, it just has to be applied wisely. I’ve tried to quantify this a bit more in the follow up post I wrote on an economic model of technical debt: if we treat it like real debt, can we be more rational about its positive and negative aspects?

      Reply
  8. Pingback: Most interesting links of September « The Holy Java

  9. Pingback: Taking a “Product Debt”? Be Ready to Pay Interest | On SaaS Products

  10. Pingback: Explained: technical debt, and when to pay it off

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s