“Talented, motivated and keen to learn” Yeah? You and everyone else

Singletrack is hiring again. Generally I love the hiring process and I’m looking forward to adding some fresh new talent to our growing development team in the new year. Personal contacts and word-of-mouth referrals only take you so far and we, like every growing business, have to deal with recruitment agents and advertising. I don’t mind: a good agent is worth their fee and ads can bring in unexpected delights.

But what the fuck has happened to CVs recently?

I rarely receive any CVs longer than two pages and the cover letters are bland variations on a theme: “I’m a talented and professional programmer seeking a new challenge. I’m hard-working, motivated and keen to learn.”

Yeah? You and everyone else.

How am I supposed to tell anything useful from a paint-by-numbers cover letter and a two-page resume where the first half of the page is an alphabet soup of programming languages, technologies and methodologies. Where the remaining page-and-a-half is a line or two about the projects you’ve worked on with almost everything about the project and nothing about your work?

How can I get any kind of feeling about you when your CV is devoid of all personal information apart from your name? Worried about discrimination? Aren’t we all but I, as I’d imagine most tech employers, don’t think that way and finding out you’re 50 and have cross-trained to programming after a career as an English teacher is something I’m going to find out pretty quickly anyway. In my case something like that is likely to make you more interesting to me, not less.

Here are my top five tips for getting a job with Singletrack or the many other employers looking for great people to join their development teams:

1. Write a specific cover letter. Given you’re applying I know you want A job … I want to know why you want THIS job. It just has to be short and specific letting me know you’ve read and understood the job description, that you know a little about what we do and that you think you’re the person we need. 15 minutes of your time is all that’s required and if you can’t spare 15 minutes finding out a little about a company you might end up joining you’re definitely not the person for us.
2. Write an adequate length CV. This ‘make everything fit on two pages’ approach is bullshit. We generally look for people with significant experience and there’s no way you can adequately tell me about the work you’ve done, the skills you’ve got and the person you are on two sides. Guess what? I’ve been hiring people for well over a decade and I know how to skim-read a six-page CV and spot good stuff, especially if the covering letter has already caught my eye. I don’t want chapter-and-verse but a short, pithy precis of everything you think I need to know about you if we’re going to work together.
3. Tell me what you’ve done. Okay, I need to know a little bit about what a company or project was about to get some context on the work you’ve done. But its just background. This is your CV so don’t just tell me what the company does or what the project delivered, tell me about your role in it, what you took responsibility for and what you achieved.
4. Tell me what you’ve learned. I can’t believe how few CVs I read that contain phrases like “On this project/at this company what I learned was …”. The thing about what you’ve done is its generally a factor of time: you can only do so much in a period of time. The thing about learning is its a factor of your capacity, openness and willingness to engage. Between a candidate that has done much and learned little and a candidate that has done little but learned much there’s only one winner.
5. Tell me about you. Show me the person as well as the employee. Single mum who’s taught herself programming at nightschool? Now that’s hard-working, motivated and keen to learn. Lover of philosophy? Someone who knows about how to think. Fan of 80’s electro music? We’ll no-one’s perfect but I won’t hold it against you; it didn’t stop me founding a company with someone.  Fanatic Mountain Biker? There’s a reason we’re called Singletrack Systems.

The first three are really the basics: don’t do these and you’re just letting yourself down. The last two can make a stand-out difference. We’re looking for people to come and spend days, weeks, months and years with us so finding out about the person, not their skills and experience, is the number one priority.

And it all starts with the CV.

Advertisements

“Magic, suggestion, psychology, misdirection and showmanship”

“Magic, suggestion, psychology, misdirection and showmanship” is how Derren Brown describes what he does. I’ve long been a fan of Derren’s – I managed to wangle front-row seats to one of his shows years ago and was utterly blown away – but of that list its not the psychology or suggestion that intrigues me, its the magic and, particularly, the showmanship.

The longer I spend doing what I’m doing the more I’m convinced that magic and showmanship are fundamental to the success of a technology product company.

Arthur C. Clarke said “any sufficiently advanced technology is indistinguishable from magic” and whilst I wouldn’t claim that what Singletrack does is so advanced that our customers and users think we’re a group of warlocks and witches, I certainly think there are magical elements to it. Fundamentally I believe software development is close to magic and I’ve written before about how amazing it is to me that we can integrate diverse cloud services to add incredible levels of features and benefits to Singletrack’s product with disproportionately little effort.

But what is magic without showmanship?

Imagine a magician who said “well I suppose I can show you a trick if you want but you won’t like it”. Who said “yeah, but to be honest anyone can do that with a bit a practice”. Who went through their routine in surly silence, sighing at the boredom of it all. Actually that’s not a magician at all its just someone with a bit of dexterity and some specialist knowledge, like a hairdresser or a masseuse*.

Building a product and a product company has taught me to think differently. I no longer see “a simple integration with Amazon S3 using their web api” I see “the addition of an unlimited capacity, edge-cached, versioned document store”. Its not “a bit of AJAX and a few queries returning some data” its “a live data dashboard showing key management information”.

I’m guessing that there is a bunch of people I know who, if they’ve decided to read this and have made it this far, are rolling their eyes and thinking “great, he’s discovered marketing hype”. But that really isn’t what I mean by showmanship. I’m not talking about lying or making inflated claims. I’m not talking about conning the customer or bullshitting the user.

I’m talking about communicating the excitement and the wonder of what you deliver to the people you deliver it to.

I’m talking about making the magic you do, magical.

I love the fact that, over the last 15 years, software development has become so much more open, so much more honest, so much more about delivering quality and value rather than simply working to plans. I love the constant striving for improvement in what we do and how we do it. But I also hope that as we take the mysticism and myth our of our development teams and processes we don’t accidentally remove the wonder and beauty of what they deliver.

That we leave a little room for magic … and showmanship.

* Please note that I have nothing against hairdressers or masseuses, but I wouldn’t pay for front-row tickets to watch them perform for two hours.

In reply to Stop Chasing Exit Strategies …

Jason Gorman has written an interesting article: “Stop Chasing Exit Strategies And Start Chasing Great Software“. @robbowley tweeted enthusiastically about it and I enjoy a lot of what Rob reads and writes so I went and had a look. The whole thing left me with a great big feeling of “Hmmm”. Normally I’d leave a comment but Jason’s blog doesn’t allow them so after a bit of consideration I decided to write this. I hope Jason reads it and takes the opportunity to respond … I’m not really a fan of blog posts that deconstruct other posts but if I can’t comment there, I can only really comment here.

Let’s start with the detail and then think about the overall message. First off the title. Very eye-catching and if it were “Stop Chasing Exit Strategies” I’d have to agree; I’ve long advocated an approach to entrepreneurship that emphasises building a solid, sustainable business on the basis that if you do, exit opportunities will come (should you want them). Hyping the hell out of a business and then cashing in your chips just before the whole thing implodes is a strategy that has worked for some and failed for many others. I’m not saying its wrong but it’s not for me.

But what do exit strategy, or lack thereof, and software quality actually have to do with one another?

Reading on a bit further the impetus for the article is revealed:

Today it was announced … that police state-friendly social networking site Facebook is acquiring pointless image filter service Instagram for $1 billion.

You don’t have to be a psychoanalyst to determine that Jason isn’t a fan of either Facebook or Instagram and I understand that as neither am I. But it does reveal a prejudice. I know a number of smart, savvy people – many of them top-class developers – who think Instagram is fantastic. I also read this interesting piece by Om Malik where he says (at the bottom):

People love Instagram. It is my single most-used app. I spend an hour a day on Instagram. I have made friends based on photos they share. I know how they feel, and how they see the world. Facebook lacks soul. Instagram is all soul and emotion

From Jason’s article:

My goal is to create better software (and, more recently, to try and help other people create better software). Most important to me is what value software brings to the people who use it.

By Jason’s own definition it sounds like Instagram has hit the nail on the head with the quality of their software in that their users get a great deal from their service. What’s more its not clear to me that Instagram have been chasing an exit in the sense of hyping the hell out of it before it implodes; they’ve grown very quickly and taken on a lot of VC money, and I can’t say whether they are really worth $1B to Facebook or not. But, from what I’ve seen, they’ve taken an idea that people like, have developed it in the face of real user feedback, have scaled their operation and so have attracted an exit opportunity. What’s the problem there?

Okay, enough of the nit-picking. Perhaps Jason just picked a bad example in Instagram. What about the general concept that chasing great software will lead to a good business?

Er, well, no. What defines a ‘good’ technology business, or any business for that matter, is its ability to make money and we all know of many successful businesses, including ones that have been around for many years, who get away with pedalling shit software through weight of market share, iron-fisted defence of patents, or just lack of credible competition. But like Jason I’m personally not interesting in pedalling shit software just because it makes money. However I am interested in making money with my business because that’s what pays our salaries which enable us to keep developing software. And here’s where things get tricky. You see quality software does not necessarily equate to software that users get value from. And users getting value from software does not necessarily equate to a good business model.

But a good business model is necessary for a good business, whether your own or one you work for.

Returning to Jason’s article:

A classic example of this kind of thinking is the very damaging advice being propogated among the tech start-up community that the software that powers your new business only needs to last until you find a buyer.

Jason doesn’t name Lean Startup or Customer Development so maybe he is talking about something else. However, if he is, then I think he’s got the wrong end of the stick. Both Lean Startup and Customer Development – of which I am a passionate advocate – are about understanding what it is your customers want and how valuable it is to them without spending lots of time and effort (yours and theirs) that turns out to be wasted. Its about learning how your business can work. Jason has it absolutely right when he says

The game’s afoot when we start getting feedback from real users. That’s when we really start to learn what works for them and what doesn’t.

Spot on. But let’s not confuse good software that users like with software that customers will buy, or confuse sellable software with a good business (with our without exit opportunities). Personally I think you need to figure what you want from a given situation and work to make that happen. If you want to play high-stakes all-or-nothing exit strategy chasing, be my guest but count me out. If you want to produce software that users love without reference to whether the business makes any money or not, ditto. If you want to produce, and keep producing, valuable software then you need to constantly balance, and rebalance, the many opposing forces in software development, software delivery, sales and marketing, customer acquisition, customer retention, investment, and so on. You need to listen to your users, your customers, the market, your peers, your competitors, current and potential investors and to that little voice which tells you what’s right despite all the evidence to the contrary. You need to work out what makes your software valuable, not in the ‘oh, gee, I really love this” sense but in the “I’m prepared to pay $XYZ for the opportunity to use/advertise through/invest in/acquire” sense.

Fair enough: Stop Chasing Exit Strategies. But don’t Start Chasing Great Software if you’re aiming to build business value. Instead: Chase a Great Business.

Stopping Starting-up

I love start-up. The early ‘wouldn’t it be great if …’ conversations, the commitment to going on a journey into the unknown, the first days when absolutely nothing is in place and things like name, logo, and website are the focus of endless creative conversation, the first customer, the first hire …

Its a hell of a ride.

There’s a huge amount of start-up advice around for the would-be entrepreneur. May favourites are the writings of Steve Blank and Eric Reis; Customer Development and Lean Startup have more to say about how to go from a vague idea to a functioning startup than any other business approach I’ve seen. There’s an increasing amount of support for start-ups, in the UK at least, from various tax breaks for would-be investors to the well-intentioned but somewhat ill-executed Startup Britain. There’s also a lot of very early stage investment money washing around from Angels and Micro-VCs; let’s face it if you’re going to invest in a hare-brained, high-risk scheme dreamt up by a bunch wide-eyed idealists you’re probably better off taking a punt on someone’s idea for a new type of social network than buying Euro-zone bonds.

Start-ups will save us from the global economic crisis!*

But once you’ve started your startup, you’ve done some customer development, you’ve perhaps pivoted your idea, you’ve reached a core of a product and you have some paying customers, then what? Its time to stop being a start-up and establish your business. Steve Blank calls this phase ‘Company Growth’ in Four Steps to the Epiphany.

In my experience** there are a few things you need to stop being a start-up:

A sustainable business model

On day one of your start-up you are concerned with the necessary detail of the company: what are we called, what do we do, where do we work? On day two you should be out there discovering your customers and understanding your potential market. On day three you should be trying to show those potential customers why they should become actual customers. And so on. You shouldn’t be worrying about sustainability, about what you need to keep being successful in 12-18 months time because unless you focus on finding your customers and your product-market fit you won’t be around in 12-18 months time.

But at some point you’re going to know roughly who your customers are and your product will have demonstrated a reasonable fit with the market. At this point you need to be worrying about sustainability because things like cash-flow, excessive overheads, technical debt and the like might just prove fatal if you don’t deal with them.

A sustainable business model is simply one where the revenue exceeds the cost (and beware hidden cost; many lean start-ups rely on the early joiners living with a reduced or waived salary in return for a stake in the long-term success of the company, which is an inherently unsustainable situation). It doesn’t necessarily have to be by a lot and it doesn’t necessarily have to make month-on-month profits, but it does need to support itself. But whilst it is defined by financial security, sustainability is about more than just about the money: a sustainable company will retain its employees, improve its processes, and learn from its internal influences as well as its external ones.

Less Product-Market Fit, more Market-Product Fit

A good product, one that meets the needs of its customers and has established itself in the market, should start to distort that market, no matter how niche (“Find your niche and dominate it”† was one of the excellent pieces of advice we were given on starting Singletrack). Disproportionately successful products like the iPad and Facebook demonstrate this in extremis; the newly-perceived tablet market is actually an iPad market, many times greater in size than the tablet market ever was, and early social networks simply never imagined the market was as big as Facebook has demonstrated it to be.

But all successful products demonstrate this effect to a degree. The current dominant player in our market does one thing well and many other things poorly (according to their customers we’ve talked to). They have distorted the market to be all about their core strength and we are actively seeking to displace them by redefining that market. Some of their customers will buy into us, some won’t, but we hope that in the next couple of years, when people talk about the market we’re in, the conversation will be much more diverse than it is today.

What I’m not saying is that Customer Development is a short-term process, or that an established company no longer listens to or learns from its customers. What I am saying is that in the early days a lean startup will do almost anything (within boundaries of ability, desire and possibly legality) that customers commit to paying for. But as time goes on you have a lot more data to go on and a lot more experience in your market and there will come a time when leading your customers on a market-defining journey is more valuable to you and them than focusing on fitting your product to the market.

Weaning off the founders

In the early days the founders are the company. As the company grows this effect lessens but it will be quite some time until the company is immune to the loss of some or all of its founders. But that is precisely what an established business needs to be. In the early days people will buy into the founders of the business as much as they buy into the product.

In fact I’d go so far as to say that believing in the founder has far more to do with an early customer making a commitment to buy something that doesn’t actually exist yet as whatever it is the product is portrayed as shortly to be doing.

But as time goes on the founders’ need to replace themselves with others who do the jobs they’ve been doing better than they can. They need to reduce their day-to-day involvement and ensure they are steering the company in the right direction. This doesn’t mean they need to make themselves redundant or irrelevant, just that the company should continue on if and when the founders decide to leave.††

In Conclusion

These are the things I think you need to stop being a start-up and get established. But I’m sure there are more and am interested in other people’s experience. It seems to me that with all the buzz about start-up around at the moment, a good body of experience and knowledge about how to successfully stop being a start-up is going to be increasingly important.

*Actually start-ups won’t save us from the global economic crisis but they might just create a few much-needed jobs, create a bit of excitement and confidence, and instil more entrepreneurial spirit in this country.

** Background: I’ve started three companies. The first never got out of being a start-up and died midway through its second year. The second was modestly successful and is still around after 10 years. The third is currently making the transition from start-up to established business.

† Parker Harris, co-founder of Salesforce.com

†† I’ll know Singletrack is doing okay when the team start telling me to shut up and let them do their jobs. I’m looking forward to that day.

Optimising Custom Applications on Force.com

One of the greatest challenges of developing on someone else’s PaaS offering is performance optimisation. When I built large-scale enterprise web systems from the ground up there were so many levers we could pull to improve performance without changing a single line of code: more/better CPUs, more memory, more/better use of indexing/caching/threading, and so on. And if we wanted to optimise code we could choose where to apply our efforts, from the lowest of the low-level infrastructure code to the client code running in the browser.

But when you build code that runs on someone else’s platform you have only one thing you can optimise: your own code.

One of the things that amazes me about building on force.com is how infrequently we need to do performance optimisation. Create a simple custom page with a record or two’s worth of data and a smattering of dynamic client-side functionality and an end user will be hard pushed to tell that its not a native force.com tab. Even more complex pages, with more than a couple of records and more than just a smattering of client-side functionality render pretty damn quickly and are perfectly usable. But the Singletrack system also has a few pages that are really very complex, cover a few hundred records and provide a lot of client-side tools. This post covers the specific topic of how we optimised a custom force.com page that took ~40s to render in its first version and got this down to < 2s.

The problem

Deliver information about a list of contacts, usually around 150-300 in length. Which contacts are returned may be manually set or may be dynamically chosen using a set of non-trivial criteria. What information is delivered is configurable on a per-user basis but typically consists of ~20 fields covering the Contact, its Account, and recent Activity History. Add in a number of tools for manipulating and interacting with the information on the list. If it helps, think of it as a Salesforce list view on steroids.

The first solution

Work out what information the user wants to see about each contact (from Custom Settings). Work out the criteria for selecting the contacts (stored on the record underpinning the view). Dynamically construct the query string and execute. Render as a table using Visualforce (think JSP/ASP/ERB for none-force.com’ers) along with embedded Javascript for all the client-side functionality. The result: ~40s from request to availability in Chrome.

The first optimisation … and some lessons learned

Rule #1 of optimisation; profile the shit out of the system and work from facts not opinions. But profiling isn’t well supported in force.com (basically, debug statements with timestamps are required) so we made some guesses as to where we thought the problem was likely to be in order to focus our instrumentation efforts. Given we were still quite new to force.com at the time we were probably a bit too influenced by our fears and immediately set about instrumenting all the querying. Waste of time, even increasing the number of contacts tenfold the querying accounted for less than 300ms of the request. And in general the server processing was really very fast.

Lesson #1 of optimising in the cloud: profile the shit out of the system and work from facts not opinions.

Instead we turned our attention to the page rendering and this turned up a surprising result. We needed two <apex:repeat /> loops to construct the table; one for the rows and one for the columns. Rendering a table of 3000 rows (requiring 2000 iterations in one loop) was pretty fast, rendering a table of 400 rows with 5 columns (also requiring 2000 iterations but two loops) was not. In fact it was 10 times slower and rendering 200 rows with 10 columns – our most typical use case – was much slower still.

This is when lesson #2 of optimising in the cloud really hit home: you can either do less or you can do differently, you don’t have the option of adding more of a vital resource. We could remove the ability of users to choose which columns they saw (making the column set fixed and removing the need for the nested loop) or we could change the way we rendered the table. In the end we decided to do differently and return all the results as JSON data and construct the table in Javascript on the client. Our first version of this approach gave us a 100% improvement in performance: 20s from request to availability.

However we also quickly worked out that our JSON based solution (this was before force.com released native support for JSON) was still pretty slow due to using ‘+’ for String concatenation in creating the JSON string. Replacing this with extensive use of String.format() gave us another 100+% performance improvement: 8.5s.

The second optimisation … and more lessons

We lived with this for a while. 10s wasn’t great but it was no slower than opening up an Excel spreadsheet (what users had been doing before they used our system) and general consensus was 10s was okay. Of course, what seems acceptable on day one rapidly becomes irritatingly slow and within a couple of months there was a lot of grumbling about performance especially as some people were reporting that the page ‘regularly’ took 20+s to load. This turned out to be rooted in some environmental issues: i) browser caches were being cleared out every time the browser was closed – a not uncommon admin setting in our customers’ domain and ii) the ISP routing for salesforce.com subdomains (used for accessing custom code in managed packages) turned out to be less than optimal – sometimes adding 6-8s to a request. The latter was a real eye-opener and we still haven’t got to the bottom of why that was the case but switching to a back-up ISP and resolving the browser cache issue ensured the customers’ were getting consistent 10s response times.

Once we’d resolved these problems we noticed that the latency in requesting Static Resources from force.com could be pretty high: ~1.5-2s in some cases (as is commonly the case all our Javascript and CSS files were packaged up as a zip file and deployed to force.com as a Static Resource). By moving our Javascript and CSS outside the package and delivering it via Amazon Cloudfront we won’t improve performance in the common circumstance of someone accessing the system via a browser with a fully populated cache but, we can shave a second or two off the overall response time for a browser with an empty cache as well as isolating ourselves from whatever the circumstances that cause force.com to update the timestamps for Static Resource caching (it seemed that every new force.com release required all browsers to completely repopulate their caches causing a rash of performance complaints directly after a major Salesforce upgrade).

Lesson #3 of optimising in the cloud: there’s not much you can optimise outside your own code, but there is more than nothing.

This round of work got us looking at our implementation again. One thing that really stood out was the amount of time we had a ‘blank page’ before any of our data was being downloaded, even ignoring client-side processing of it: 4s. Create a custom Visualforce page with just some text in it and it will have sub-second response time. Given we knew that querying and processing the data took only ~300ms it seemed surprising that it was taking ~3s to download the data. Some investigation here turned up a very surprising result – that the viewstate for the page was huge even though there wasn’t much real state within the page. By ensuring that the scope of the <apex:form/> tags was as narrow as possible (just encompassing the fields and actions within the form) and judicious use of transient variables we were able to significantly reduce the size of the viewstate and this brought the ‘blank page’ time down to below 2s and overall response times of ~4.5s. Additionally, the psychological effect of not staring at a blank page for a few seconds meant that people felt the page was a lot faster than you might expect from a 60% improvement.

Something else we looked at here, after realising that Static Resources can be a bit slow, was how long it took for force.com to download its own Javascript and Stylesheets. By removing the Salesforce header and sidebar, and opting not to load standard style sheets we took another 0.5s off the page load.

Lesson #4 of optimising in the cloud: a bit of understanding about how the platform works goes a long way to spotting areas for optimisation even when you think your actual code doesn’t have a lot of room for improvement.

With the use of the new native support for JSON (worth ~1.5s over our homegrown implementation, and resulting in vastly simpler code) and a few other minor tweaks we’re now down to a steady 2s for page load; a whopping 2000% improvement over that first, not-too-naive implementation.

Summary

Far from just having whatever performance you get out of force.com on your first effort, there is plenty of opportunity for optimisation if you find you need to do it. However, don’t look too much to the platform for bottlenecks (apart from a few specific cases mentioned) the answer lies in your use of the platform and in your custom code. String concatenation and viewstate in particular seem to be areas where you can gain some quite significant improvements with relatively little effort and, with the new support for JSON, shipping compact JSON strings that you then render on the client rather than having Visualforce do all your rendering on the server side for you is definitely a good option even if you don’t need nested <apex:repeats />. And if you see inconsistent performance in your customers’ environments there are certain things to go looking for there that can make a dramatic improvement.

Why we don’t Continuously Deploy

Continuous Deployment is very much en vogue; its seen as the natural extension of Continuous Integration and is a part of the lean startup philosophy. But we don’t do it and I thought it might be interesting to explain why.

1. Our users like improvement but they don’t like change. We build a business system for business users and our users each spend 2-8 hours a day using our system. They want to see the system improve but having a consistent, recognisable user experience is more important than having one that is constantly changing, even if that change brings improvement. So we gather all their feedback as they use the system and package that into a release. When that release is ready we do demos and distribute documentation to make sure everyone is aware of the changes coming. Bundling up all improvements into one big change allows us to make sure everyone understands what is coming, is happy about what’s new, and there are no surprises when they log in.
2. We do QA as well as testing. To me QA is a separate activity to writing and running automated tests. My experience of developer-written tests (unit, functional, integration and so on) is they tend to take a rational and logical view of the functionality under test: what happens when used correctly, what happens when expected parameters aren’t supplied, what happens when parameters are supplied with unexpected values? But users aren’t necessarily ‘rational’ or ‘logical’ as developers see it and having humans bash away at the system, hitting the back button randomly, pasting parameter values from surprising sources, leaving the browser open for an hour while they have lunch, and so on helps us to spot the gaps in the automated testing and the assumptions we made whilst writing them. But manual QA takes time and having continuous pressure to have it done as quickly as possible to support continuous deployment will be counter-productive. So we bundle our changes up into weekly QA releases which means QA can be done at a pace that suits the assurance of quality rather than speed of deployment.
3. We’re still learning. To me there’s a bit of a paradox in Continuous Deployment as applied to lean startup. On the one hand, multi-variate testing and continuous deployment allow you to gather and act on lots of feedback very quickly. On the other hand, a constantly changing user experience makes it harder to make sense of that feedback. In our system there is a piece of core functionality that users use for ~3 hours each day. If that changed on a daily basis they’d quickly get pissed off with it and wouldn’t use it until it was ‘ready’ or ‘done’. We’re now on our third major re-working of that functionality and the latest version is vastly improved, based on real user feedback. But each new version has been introduced as a whole, with all the change management mentioned above, and so we’ve kept our users onside whilst making quite radical changes to the product.
4. ‘Continuous’ is a relative, not an absolute, term. Sounds like a stupid statement but its a lesson I learned a while back. In a previous job I was doing some work with a company and their supplier to improve their responsiveness to change. We were pushing for weekly releases but quickly realised that, to a pair of companies used to 18-24 month release cycles, quarterly releases would be a big improvement and much more achievable than weekly. Were we being unambitious? Perhaps from our own point of view as eXtremists, but in terms of the goals our clients had, quarterly would take them along way to achieving what they wanted. At Singletrack we do releases every 6-10 weeks which is so much more continuous than other systems our users use (upgrades every 12-24 months is more the norm) but avoids the disruption that ‘true’ Continuous Deployment would cause.

So we currently do: Continuous Integration in development; Daily Builds with full runs of automated tests (takes about 3 hours otherwise we’d do this more frequently); Weekly QA releases; End-user releases every 6-10 weeks. A lot of this stems from our business: we sell a complex, powerful SaaS product to sophisticated and demanding business users. If we were running a B2C website with a host of irregular and infrequent users (from the point of view of our system, a user that doesn’t log in for hours every day is infrequent) then we would certainly go for a much more continuous process than the one we have now.

An economic model of technical debt?

A lot of what I’ve read about technical debt assumes that it is generally a bad thing. A lot of these articles also use credit card analogies to compare technical debt to personal finance and I think this is missing a trick. Businesses take a different view of debt to people and I wonder if using a more mature model of technical debt  would allow us to take a more nuanced view? [Caveat, I’m not an economist or an accountant, I’ve just run SME businesses for a few years and so have a little knowledge, possibly just enough to get this badly wrong].

Start-ups like ours run on credit. Whether its founders taking no salary, people working for sweat equity, friends and family or Angel funding, buying kit and services on personal credit cards, whatever – there’s going to have to be a bit of debt incurred to get from having nothing but an idea to having enough paying customers to sustain you. When we incur technical debt in our start-up it is in very much the same spirit: if things don’t pan out the debt isn’t going to have to be paid back. That doesn’t mean you take on as much credit as is being offered because loading the business with debt may quickly become one of the causes of its failure. But you take sensible risks, take the credit you need and work bloody hard to pay it off before it becomes a burden.

As a business starts to establish itself things like cashflow become more important. A business might use credit (e.g. order factoring or  bridging loans) to smooth the flow and to reduce their risk of running out of money. Many projects I’ve run experience a similar ebb-and-flow of requirements and I wonder if taking a longer-term view of the ‘requirements flow’ of a project might allow a more structured approach to technical debt; when there are lots of requirements to deliver in a short period, be prepared to take on a bit more technical debt. When there is less pressure to deliver requirements, pay the debt down. If the ‘less pressure’ phase isn’t looking like coming, take extraordinary action if it looks like the TD might get out of hand.

Once a business is established and looking to grow, it often takes on debt in order to achieve strategic objectives. What would be the equivalent in software delivery?

The problem with all the above is that it is still very much an analogy and I’m no great fan of analogies in software development. But what struck me about the conversation leading up to and following my previous post on Technical Debt  is that technical debt isn’t really a metaphor as such. It’s not like monetary debt – where taking a bit of extra risk and incurring a greater overall cost in the long term allows you to achieve important things in the short term – it is monetary debt. By accepting technical debt you are (presumably, or why else are you doing it) incurring a lower cost, or enjoying a higher income, today but incurring a higher cost in the long run.

So what if we stopped treating technical debt as metaphor and started considering it as monetary debt? Could we quantify the costs we save by not doing something today (e.g. pairing, refactoring, resolving limits issues, optimising, or even just applying good old YAGNI) that may cause us pain at a later date? Could we quantify the value gained by doing something valuable sooner? Could we quantify how much more it will cost at that later date when we have to deal with the pain? Could we use these numbers to base decisions about whether or not to incur technical debt?

Here’s a simple example from our start-up. Two customers wanted broadly similar features adding to our product but had different timescales and some differences in detailed requirements. We took the decision to build two separate versions of the feature, one for each customer, even though we knew that this would give us two broadly similar sets of code to maintain and, if we wanted to sell the same feature to other customers, we would not only need to refactor them into one code set, we’d also have to do some additional work to migrate the customer’s data from their individual versions to the new unified model.

In this instance it wasn’t about cost saving as such. Both customers were willing to pay for the feature to be added if we could do it to their timeframe. So we got some money up front, lets say $100 (I’ll preserve the ratios but I’m not prepared to reveal the real sums). The cost of building it twice was a bit more than building it just the once, let’s say $20 instead of $15. We then had an additional cost of maintaining two code sets for  awhile, lets say $3 instead of $2, and we then had the cost of the rework and migration: $12. All in, the gross profit for this was $100 – $20 – $3 – $12 = $65.

Now suppose we’d built it once for one customer, and then evolved it for the second customer. Immediate income is $50. Gross profit is $50 – $15 – $2 = $33. If we then sell to the second customer the profit goes up but probably not to $83, lets say $80 because there’s bound to be some migration or re-work required to keep customer 1 in alignment with what customer 2 wants.

In this economic model it is $15 more profitable not to incur the technical debt. But, and its a big but, in any business and particularly in a startup cash is king. $100 dollars this month is way better than $50 this month with the expectation of another $50 three months later. And that assumes customer 2 still wants to pay in three months time. By then they may have bought or built an alternative. If they don’t buy you’ve only made $33, not $65; by any conventional business model, a certain $65 profit is vastly preferable to a certain $33 profit with the possibility of an additional $47.

But whether or not you agree with the decision to go for the $100 now and incur the technical debt isn’t the point. The question isn’t whether we made the right choice in this instance, its whether quantifying like this makes it easier to surface the benefits and liabilities of technical debt? A lot of the blog articles about technical debt say its hard to quantify but making rough guesses about this stuff isn’t that hard and rough guesses should be all you need (let’s face it, most successful business are run on far rougher guesses about predicted revenue, predicted costs, etc.). And it seems to me that if, as techies, we could get better at using terms, equations and numbers the business understand we could get much better at communicating why we should or should not incur technical debt, what our current debt level is, what it will cost to pay down, what the financial, reputational or other forms of impact might be if the risks inherent in the debt don’t pan out, and so on.

Perhaps we could stop treating technical debt as a metaphor and start using it as a real tool for planning and delivering our products and projects.