Category Archives: Not Agile

Technical Debt and the Lean Startup

A conversation on twitter between @david_harvey,  @rachelcdavies@RonJeffries and myself made me to think it was about time to post on my view of technical debt in a lean startup and how that view has changed since the days of running XP-based projects for established companies.

There’s a lot of information about Technical Debt out there on the web (including Ward’s Video, a clear summary by Martin Fowler, and an interesting, albeit somewhat flawed, article by Rod Hilton) and I’m not going to get into a detailed description here, but at its simplest level it says that making some quick-and-dirty choices today may lead to pain later on, and that pain is like a debt that you’re going to have to pay back at some point.

Its that ‘at some point’ which has led me to believe there is a difference between technical debt as applied to a lean startup vs. that applied to a project for an established business. In a project you are trying to deliver something which meets some kind of business case. You will know roughly who the customer is, roughly what their objectives are, roughly what you can spend (time and money) in order to achieve them, and so on. There’s going to be a lot of stuff that isn’t known but there are some constraints in place. When I used to use XP to deliver these kinds of project we had a very simple view of technical debt: don’t incur it if you can avoid it and, if you can’t avoid it, pay it down at the earliest possible time. Why? Because in these projects sustainability is a primary objective: you know you’ll have to pay the debt down at some point so why let it grow uncontrollably?

In a lean startup not only do you not know what your customers want, you don’t even know who they are, and you usually have less time and money to spend on finding out than most established companies would try and run a project with. Your primary objective is finding those customers and learning what they want, not sustainability. Which isn’t to say sustainability is unimportant, just that its not your primary objective; once you have a business, then you worry about how to make it sustainable. Its a High Quality Problem.

Here are some examples of technical debt we incurred at various stages of our search for customers and what they wanted:

  • Areas of the system understood by only one person. I’m a pair-programming fanatic and would love for everything in our system to be developed by pairs. But when you have only one developer in the company, that isn’t possible. When you have two developers and one has to go support a sales meeting, that isn’t possible. In fact, our experience is that until you have 5+ full-time developers there are going to be times when it isn’t possible to have everything paired. And do you know what? That’s okay. Code can be developed by a single developer but the fact that they built it on their own is a form of technical debt: until others have worked on the code, it is more widely understood, and has had the benefit of many eyes on it, you are in debt to the system.
  • Large scale refactorings left undone. We use a number of underlying platforms and tools. If the platform or tool doesn’t do what we need we develop custom code. It isn’t uncommon for the tool or platform provider to then release something that does what our custom code does at a later date and, until we refactor to use the new platform or tool feature, we have some debt to the system. But that’s okay – the custom code still works, its just not as simple as it could be nor as future-proof if the provider develops the feature further.
  • Limitations left in place. Sometimes a truly scalable solution is much harder than a simple but limited one. We had a situation where using in-memory collections led to a hard limit on the number of objects we could handle but the in-memory version of the code took a couple of hours to produce. We knew how to get around the limit – by adding in a system of creating permanent records and then processing them asynchronously using scheduled jobs – but that was several days work. The limit was okay – the function still worked, just for small collections of objects, and we had a debt to the system as and when our customers couldn’t live with the size limit.
  • Sub-optimal design approaches. Sometimes it is easier to deliver a simplistic design than a well-crafted one. For example we delivered a quick-and-dirty reporting function that was simple but slow because we were making it work, then making it right but hadn’t yet got to making it fast. But that was okay, the information was available to the users and correct, just not very quickly delivered.

But why would we live with such limitations? Aren’t we just delivering a shoddy system?

The answer to the first question is that we’re prioritising learning what makes our business work over some abstract notion of quality. If we develop a feature and it turns out that customers don’t want to pay for it, does it matter if it is only understood by one developer, that it could be refactored, that it is limited or sub-optimal? We develop the feature to get it into customers’ hands as quick as possible, to understand whether it is something valuable or not, to understand how it should work. Only if that features is valuable is it worth paying down any debt we have incurred building it. In fact, never mind the feature, the same applies to the whole system: unless the system is sufficiently valuable to customers we have no business and if we have no business the whole question of quality is moot. Customers pay for features and benefits, not fully-paired, beautifully factored, limitation-free, optimised code.

The answer to the second question is an emphatic ‘no’. We produce production quality, tested code. The testing is something we don’t compromise on simply because it is the mechanism by which we will pay down the debt if and when the time comes to do so. I can live with the fact that a design is sub-optimal if we have tests which will help us optimise it when we have to. I can live with code that need refactoring if we have tests which will help us refactor it when we have to. I can even live with something written by one person if … well you get the picture.

In a startup, technical debt is something to be managed, not minimised. We make sure we understand how much debt we have and which bits of the system it affects. We make sure we have the ability to pay down that debt as and when we need to. And we make sure we work the time and money required to pay down debt into any timsescales or budgets we agree.

Anything less would be irresponsible. Anything more is prioritising some notion of ‘code quality’ over learning what makes our business work and equally irresponsible.

I’m not Agile, just a little eXtreme

The moment of epiphany came at OOPSLA ’97 when I went to Ward and Kent’s Pair Programming BoF. Here I saw in practice the things I’d heard Ward, Kent and others talk about: TDD, Pairing, Refactoring, Doing the Simplest Thing, User Stories, System Metaphor and more. I went back to work, full of fire, and managed to convince a sceptical manager that there was a better way of delivering software than what we were currently doing. Since then I’ve been an eXtreme Programmer.

In 1999 I started my first consultancy business and XP was at the heart of our development process and some of our business practices too. But we didn’t talk about it … we couldn’t because the “extreme” word worried people and the mention of such outlandish practices as Programming in Pairs and Writing the Tests First merely confirmed their view we were mad. In other words we did’t sell XP, we sold the outcome of an eXtreme approach to software delivery.

In 2001 the announcement of the Agile Manifesto left me bemused. By this point we were talking about XP and using it with big teams on big projects. Even though there were still customers who thought what we said we could do was “impossible”, that Pairing was “two people doing one person’s job” and so on we had enough of a track record to point to, as well as the experience of others, to overcome the scepticism. I could see where the Agile Manifesto was coming from, and had no doubts about the good intentions of the authors, I just couldn’t really relate the set of value statements to what we were doing: selling the delivery of software in a way that out-performed other projects by every measure. And to be clear, we were selling the delivery of software, not training people, not offering qualifications, not consulting, delivering software.

By 2004 the “Agile” word was in common usage. It was useful to short-cut a lot of the explanation about what we did by talking about an “Agile process”. It was easier to gain acceptance of what we did because this nice fuzzy word was a synonym for good software delivery. But this was also the time that the bigger consultancies started to rebrand what they did as “Agile”, that I started to meet people who described themselves as “coaches”, that I heard about the idea of being certified as a practitioner (I knew people who thought I was certifiable as a practitioner but that was a different thing altogether).

Now in 2010 I can’t help but hope this whole Agile thing has run its course. It was always a somewhat meaningless word anyway (as Mark Stringer once tweeted, “… ask people if their company’s Agile, nobody’s gonna say – oh no, we’re lethargic and arthritic”) but now the people who seem to be talking about it most are those that make money by selling the process, not the outcome. They sell coaching, process improvement, training, certification, tools, people, ‘value’, blah, blah, blah, but very rarely do they actually sell software delivery. You know, as in “we’ll deliver this software” as opposed to “we’ll help you deliver software”.

And that’s okay. I have nothing against people peddling Agile as a way of making money (and, mea culpa, for a brief period in 2008 I did a bit of this too). It’s just that it no longer has anything to do with what I do. I recently sat in front of a customer’s project manager – a very smart and reasonable person – and accidentally used the A-word when describing how we were going to deliver our product and required customisations to them, and they sneered.

They actually snorted in disgust.

When I then explained we would get them live and using the base product quickly, followed by weekly incremental improvements with regular reviews and plenty of opportunity for rework they were very happy.

But they didn’t see any connection between the two things.

Many years ago I tried to explain the concept of small-a architecture in software to a conference in Manchester. This was the idea (from Bruce Anderson) that every software system has an architecture than needs to be understood and cared for. Which is very different from what was considered Software Architecture at the time: people who hadn’t written a line of code in many years drawing lots of boxes and connecting them up in ininteresting ways to hand over to the poor schmucks who actually have to implement the thing.

It’s tempting to champion the concept of small-a agile: achieving agility rather than being Agile. But, do you know what? I can’t be arsed. Because its not about being agile/Agile or achieving agility, or being lean/Lean and efficient , its about delivering software. And I figure the best way to champion that is actually just to get better at doing it.

So, with a nod to Mr Stringer, I’d like to say “I’m not Agile” … I just deliver software.