Category Archives: Uncategorized

Collaboratively Choreographed Chaos (with Common Cause)

I recently discovered that the best way to spend some time in Venice is to buy an all day ticket for the Vaporetto (ferry bus), find a seat at the front, and just take it all in. Over time I found my attention drawn to how the various users of waterways picked their way through the Canal Grande.

Elegant Gondole present tourists with a facsimile of romance and a personal tour of the city’s heart.


Luxurious Taxi d’Acqua provide both the fastest way of getting from A to B and a comfortable way of touring.


(c) Ted McGrath

Utility barges bring wine and beer (and presumably other goods) into the city, and take the garbage away.

Lumbering Vaporetti move large numbers of tourists and workers en masse through the city.

And so on, all sharing the same small area of water.


Given the vast differences in speed, size and agility of all these craft you might expect a series of strict rules and controls to govern all this, but it appears not. There are no traffic lights or aquatic roundabouts, there is no requirement to stick to one side of the canal or other and no clear right-of-way precedence. What there is is a series of calls and signals that indicate the intentions of the pilots, taxi-drivers and gondoliers to each other. There appear to be a number of common-sense conventions based around the needs and abilities of each craft which are applied, or not, as each situation demands. And the men and women that navigate the canals on a daily basis are highly skilled at what they do.

It is a chaotic system in the physics or maths sense of the word: a complex system whose behaviour is so unpredictable as to appear random. But it is not the randomness of total mayhem, rather the collaborative choreography of a diverse group united in common cause: to service, directly or indirectly, the tourists who are the lifeblood of the city.

Returning to work I was struck by some parallels between the seemingly random and yet ultimately highly effective Venetian waterways and our development process at Singletrack. As a team we have lots of different demands on our time: core product development, support, customer-specific customisations, new customer roll-outs, support of sales efforts and so on. All of these move at different speeds and have different, sometimes competing, characteristics and constraints.  Our development process is a mild variation of late-90s XP: a variant because we’re self organising and most of the team wasn’t doing development in the late 90s, and mild because those core practices and principles still work. Its very lightweight, everybody understands it, and its very effective.

I’ve recently begun to worry that this lack of evolution from how I learned to do things nearly two decades ago demonstrates an unwillingness or inability to change. To keep up with the state of the art. I spend time looking at new and different processes and techniques but somehow they fail to engage me and, other than the occasional cherry-picked idea, I go back to what I know.

I realise that it can’t be that Scrum or Kanban or Modern Agile are sham approaches that deliver nothing. It can’t be that NoEstimates and NoProjects have nothing worthwhile to say (even if they do seem to be determined to say it in a particularly negative way). MobProgramming must be good for some people somewhere. But I’ve found them to be limited in value to me.

What my Venetian experience helped me understand is what’s important is that you have a set a of calls, signals and conventions that allow you to collaboratively choreograph your movements, not what that those calls, signals and conventions actually are. The ones that are called XP work for me, maybe the ones called Kanban and MobProgramming work for you. What’s important is that your approach actively works towards the common cause of your organisation not that its the one that gets the most hashtag traffic on Twitter.

When I thought about writing this blog post I did a bit of research on Venice and its waterways. It turns out that what rules and restrictions are imposed had remained unchanged for nearly 200 years until the death of a tourist in 2013 ( The response to this tragedy wasn’t to impose a totally new regime of strict controls but simply to update the conventions slightly to reflect Venice in the modern era. Barges have had the hours they can enter the city limited, and illegal docks and piers have been removed, but the fundamentals of the collaborative choreography to suit the common cause remain intact.

So don’t look for the latest or the ‘best’. Look for the calls, signals and conventions that work for you. Find the collaborative choreography that marshals all the disparate people with their disparate needs and disparate talents towards your common cause. Make it work and then keep it working.





Getting the best out of Tech at Sales meetings

Like most enterprise software there is usually a point in our sales cycle where the prospect’s technical team want to do a bit of technical fact-finding or due diligence on us. As we grow one of the common internal complaints is Sales find it hard to secure time from our tech team to attend these prospect meetings. So I wrote a short guide to getting the best out of tech at Sales meetings which I realised is not particularly specific to Singletrack:

Tech at Sales Meetings

Its primarily aimed at Sales but its also partly about what a tech team should be able to support.

A Paean to Speccy

“Remember the ZX Spectrum?” the Guardian asked a few days ago. Remember it? That fragile black-and-grey box with the jaunty flash of rainbow colours and precariously attached ‘RAM Pack’ set me on a path that has come to define my entire life.

I remember hooking it up to the battered second-hand black and white portable that served as our family TV in the early 80s and loading in a copied game from tape. Brrrrr, chi. Brrrrr, CHEEEEE etc.

I remember getting it to print ‘Hello World’ 10 times using BASIC, and then modifying that to print ‘Hello Paul’.

I remember the hours of typing Hexadecimal from cheap-paper magazines smudged with sweaty fingerprints. “Is that an E or an F”. Like alchemists we toiled over our steaming cauldron of a computer trying to turn the base metal of those listings into the gold of a simple game. We even succeeded once or twice.

I remember the animated Valentine’s card hand-crafted in Z80A for my first girlfriend. Diagonal scrolling, jaunty tune, teenage poetry. She dumped me the next day for reasons I still haven’t fully grasped.

Getting a chord out of a one-channel sound chip. Creating a spinning wireframe cube. Ghhhhheestttbushhhhhterssshh. High Speed Dubbing tape-to-tape ‘sharing’. And Elite. Oh My God Elite.

Eventually, of course, like my Star Wars wallpaper and Superman duvet cover I grew out of the Speccy and replaced it with a ‘proper’ computer – an Atari 520ST.

But i still miss that little box of magic. And that wallpaper.

“Magic, suggestion, psychology, misdirection and showmanship”

“Magic, suggestion, psychology, misdirection and showmanship” is how Derren Brown describes what he does. I’ve long been a fan of Derren’s – I managed to wangle front-row seats to one of his shows years ago and was utterly blown away – but of that list its not the psychology or suggestion that intrigues me, its the magic and, particularly, the showmanship.

The longer I spend doing what I’m doing the more I’m convinced that magic and showmanship are fundamental to the success of a technology product company.

Arthur C. Clarke said “any sufficiently advanced technology is indistinguishable from magic” and whilst I wouldn’t claim that what Singletrack does is so advanced that our customers and users think we’re a group of warlocks and witches, I certainly think there are magical elements to it. Fundamentally I believe software development is close to magic and I’ve written before about how amazing it is to me that we can integrate diverse cloud services to add incredible levels of features and benefits to Singletrack’s product with disproportionately little effort.

But what is magic without showmanship?

Imagine a magician who said “well I suppose I can show you a trick if you want but you won’t like it”. Who said “yeah, but to be honest anyone can do that with a bit a practice”. Who went through their routine in surly silence, sighing at the boredom of it all. Actually that’s not a magician at all its just someone with a bit of dexterity and some specialist knowledge, like a hairdresser or a masseuse*.

Building a product and a product company has taught me to think differently. I no longer see “a simple integration with Amazon S3 using their web api” I see “the addition of an unlimited capacity, edge-cached, versioned document store”. Its not “a bit of AJAX and a few queries returning some data” its “a live data dashboard showing key management information”.

I’m guessing that there is a bunch of people I know who, if they’ve decided to read this and have made it this far, are rolling their eyes and thinking “great, he’s discovered marketing hype”. But that really isn’t what I mean by showmanship. I’m not talking about lying or making inflated claims. I’m not talking about conning the customer or bullshitting the user.

I’m talking about communicating the excitement and the wonder of what you deliver to the people you deliver it to.

I’m talking about making the magic you do, magical.

I love the fact that, over the last 15 years, software development has become so much more open, so much more honest, so much more about delivering quality and value rather than simply working to plans. I love the constant striving for improvement in what we do and how we do it. But I also hope that as we take the mysticism and myth our of our development teams and processes we don’t accidentally remove the wonder and beauty of what they deliver.

That we leave a little room for magic … and showmanship.

* Please note that I have nothing against hairdressers or masseuses, but I wouldn’t pay for front-row tickets to watch them perform for two hours.

Why we don’t Continuously Deploy

Continuous Deployment is very much en vogue; its seen as the natural extension of Continuous Integration and is a part of the lean startup philosophy. But we don’t do it and I thought it might be interesting to explain why.

  1. Our users like improvement but they don’t like change. We build a business system for business users and our users each spend 2-8 hours a day using our system. They want to see the system improve but having a consistent, recognisable user experience is more important than having one that is constantly changing, even if that change brings improvement. So we gather all their feedback as they use the system and package that into a release. When that release is ready we do demos and distribute documentation to make sure everyone is aware of the changes coming. Bundling up all improvements into one big change allows us to make sure everyone understands what is coming, is happy about what’s new, and there are no surprises when they log in.
  2. We do QA as well as testing. To me QA is a separate activity to writing and running automated tests. My experience of developer-written tests (unit, functional, integration and so on) is they tend to take a rational and logical view of the functionality under test: what happens when used correctly, what happens when expected parameters aren’t supplied, what happens when parameters are supplied with unexpected values? But users aren’t necessarily ‘rational’ or ‘logical’ as developers see it and having humans bash away at the system, hitting the back button randomly, pasting parameter values from surprising sources, leaving the browser open for an hour while they have lunch, and so on helps us to spot the gaps in the automated testing and the assumptions we made whilst writing them. But manual QA takes time and having continuous pressure to have it done as quickly as possible to support continuous deployment will be counter-productive. So we bundle our changes up into weekly QA releases which means QA can be done at a pace that suits the assurance of quality rather than speed of deployment.
  3. We’re still learning. To me there’s a bit of a paradox in Continuous Deployment as applied to lean startup. On the one hand, multi-variate testing and continuous deployment allow you to gather and act on lots of feedback very quickly. On the other hand, a constantly changing user experience makes it harder to make sense of that feedback. In our system there is a piece of core functionality that users use for ~3 hours each day. If that changed on a daily basis they’d quickly get pissed off with it and wouldn’t use it until it was ‘ready’ or ‘done’. We’re now on our third major re-working of that functionality and the latest version is vastly improved, based on real user feedback. But each new version has been introduced as a whole, with all the change management mentioned above, and so we’ve kept our users onside whilst making quite radical changes to the product.
  4. ‘Continuous’ is a relative, not an absolute, term. Sounds like a stupid statement but its a lesson I learned a while back. In a previous job I was doing some work with a company and their supplier to improve their responsiveness to change. We were pushing for weekly releases but quickly realised that, to a pair of companies used to 18-24 month release cycles, quarterly releases would be a big improvement and much more achievable than weekly. Were we being unambitious? Perhaps from our own point of view as eXtremists, but in terms of the goals our clients had, quarterly would take them along way to achieving what they wanted. At Singletrack we do releases every 6-10 weeks which is so much more continuous than other systems our users use (upgrades every 12-24 months is more the norm) but avoids the disruption that ‘true’ Continuous Deployment would cause.
So we currently do: Continuous Integration in development; Daily Builds with full runs of automated tests (takes about 3 hours otherwise we’d do this more frequently); Weekly QA releases; End-user releases every 6-10 weeks. A lot of this stems from our business: we sell a complex, powerful SaaS product to sophisticated and demanding business users. If we were running a B2C website with a host of irregular and infrequent users (from the point of view of our system, a user that doesn’t log in for hours every day is infrequent) then we would certainly go for a much more continuous process than the one we have now.

An economic model of technical debt?

A lot of what I’ve read about technical debt assumes that it is generally a bad thing. A lot of these articles also use credit card analogies to compare technical debt to personal finance and I think this is missing a trick. Businesses take a different view of debt to people and I wonder if using a more mature model of technical debt  would allow us to take a more nuanced view? [Caveat, I’m not an economist or an accountant, I’ve just run SME businesses for a few years and so have a little knowledge, possibly just enough to get this badly wrong].

Start-ups like ours run on credit. Whether its founders taking no salary, people working for sweat equity, friends and family or Angel funding, buying kit and services on personal credit cards, whatever – there’s going to have to be a bit of debt incurred to get from having nothing but an idea to having enough paying customers to sustain you. When we incur technical debt in our start-up it is in very much the same spirit: if things don’t pan out the debt isn’t going to have to be paid back. That doesn’t mean you take on as much credit as is being offered because loading the business with debt may quickly become one of the causes of its failure. But you take sensible risks, take the credit you need and work bloody hard to pay it off before it becomes a burden.

As a business starts to establish itself things like cashflow become more important. A business might use credit (e.g. order factoring or  bridging loans) to smooth the flow and to reduce their risk of running out of money. Many projects I’ve run experience a similar ebb-and-flow of requirements and I wonder if taking a longer-term view of the ‘requirements flow’ of a project might allow a more structured approach to technical debt; when there are lots of requirements to deliver in a short period, be prepared to take on a bit more technical debt. When there is less pressure to deliver requirements, pay the debt down. If the ‘less pressure’ phase isn’t looking like coming, take extraordinary action if it looks like the TD might get out of hand.

Once a business is established and looking to grow, it often takes on debt in order to achieve strategic objectives. What would be the equivalent in software delivery?

The problem with all the above is that it is still very much an analogy and I’m no great fan of analogies in software development. But what struck me about the conversation leading up to and following my previous post on Technical Debt  is that technical debt isn’t really a metaphor as such. It’s not like monetary debt – where taking a bit of extra risk and incurring a greater overall cost in the long term allows you to achieve important things in the short term – it is monetary debt. By accepting technical debt you are (presumably, or why else are you doing it) incurring a lower cost, or enjoying a higher income, today but incurring a higher cost in the long run.

So what if we stopped treating technical debt as metaphor and started considering it as monetary debt? Could we quantify the costs we save by not doing something today (e.g. pairing, refactoring, resolving limits issues, optimising, or even just applying good old YAGNI) that may cause us pain at a later date? Could we quantify the value gained by doing something valuable sooner? Could we quantify how much more it will cost at that later date when we have to deal with the pain? Could we use these numbers to base decisions about whether or not to incur technical debt?

Here’s a simple example from our start-up. Two customers wanted broadly similar features adding to our product but had different timescales and some differences in detailed requirements. We took the decision to build two separate versions of the feature, one for each customer, even though we knew that this would give us two broadly similar sets of code to maintain and, if we wanted to sell the same feature to other customers, we would not only need to refactor them into one code set, we’d also have to do some additional work to migrate the customer’s data from their individual versions to the new unified model.

In this instance it wasn’t about cost saving as such. Both customers were willing to pay for the feature to be added if we could do it to their timeframe. So we got some money up front, lets say $100 (I’ll preserve the ratios but I’m not prepared to reveal the real sums). The cost of building it twice was a bit more than building it just the once, let’s say $20 instead of $15. We then had an additional cost of maintaining two code sets for  awhile, lets say $3 instead of $2, and we then had the cost of the rework and migration: $12. All in, the gross profit for this was $100 – $20 – $3 – $12 = $65.

Now suppose we’d built it once for one customer, and then evolved it for the second customer. Immediate income is $50. Gross profit is $50 – $15 – $2 = $33. If we then sell to the second customer the profit goes up but probably not to $83, lets say $80 because there’s bound to be some migration or re-work required to keep customer 1 in alignment with what customer 2 wants.

In this economic model it is $15 more profitable not to incur the technical debt. But, and its a big but, in any business and particularly in a startup cash is king. $100 dollars this month is way better than $50 this month with the expectation of another $50 three months later. And that assumes customer 2 still wants to pay in three months time. By then they may have bought or built an alternative. If they don’t buy you’ve only made $33, not $65; by any conventional business model, a certain $65 profit is vastly preferable to a certain $33 profit with the possibility of an additional $47.

But whether or not you agree with the decision to go for the $100 now and incur the technical debt isn’t the point. The question isn’t whether we made the right choice in this instance, its whether quantifying like this makes it easier to surface the benefits and liabilities of technical debt? A lot of the blog articles about technical debt say its hard to quantify but making rough guesses about this stuff isn’t that hard and rough guesses should be all you need (let’s face it, most successful business are run on far rougher guesses about predicted revenue, predicted costs, etc.). And it seems to me that if, as techies, we could get better at using terms, equations and numbers the business understand we could get much better at communicating why we should or should not incur technical debt, what our current debt level is, what it will cost to pay down, what the financial, reputational or other forms of impact might be if the risks inherent in the debt don’t pan out, and so on.

Perhaps we could stop treating technical debt as a metaphor and start using it as a real tool for planning and delivering our products and projects.

What is the cloud? Three little words: As-A-Service

I’ve been talking to a few people recently about what this cloud stuff I’ve been doing is really all about. Isn’t it just some marketing hype? If I set up some servers have I just created a cloud? Isn’t ‘cloud’ just another word for ‘Internet’?

Clearly it is a marketing term; witness Microsoft’s embarrassing and somewhat desperate attempt to brand nearly everything they do and sell as ‘cloud’. And there are many ways to set up a ‘private’ or ‘hybrid’ cloud on your own servers, some of which have been around for years and have been re-badged to appease the marketeers.

But what I see, what excites me about what we do, is so much more than the hype. To me the cloud is a computing platform defined by those three little words: as-a-service.

Having spent a professional lifetime building large-scale enterprise systems from the ground up it is mind-boggling to me that I can integrate an apparently infinite amount of versioned, replicated, backed up and edge-cached storage into our system with nothing more than a credit card and a surprisingly small amount of coding. And Amazon S3/Cloudfront is the least sophisticated of the services we utilise.

As-a-service I can deploy to a powerful application platform, complete with user management, sophisticated security, reporting, and more ( Or I can deploy web applications built on Ruby on Rails (Herokusoon to support Clojure for FP fans) or Java (Amazon Elastic Beanstalk or Google App Engine). I can store and manipulate data, send bulk email, instrument and report on my services, the list is growing daily.

As-a-service: as I need it, paying for only what I use, with scalability, availability, security and performance all built in.

And it’s this as-a-service nature that I believe makes ‘the cloud’ a truly different and fundamentally better place to build software. The vision of Service Oriented Architecture is finally being realised but not in terms of the truly dreadful SOAP and WSDL that were bound too-tightly into the concept, or  auto-discoverable service white-pages, or centrally-controlled service architecture that simply re-packaged much of the distributed objects thinking into distributed services.

The as-a-service nature of the cloud creates a free market where service providers compete to capture our attention and take our money. To do this the emphasis isn’t on standardisation or ‘discoverability’ but on simplicity, effectiveness (compare the wonderful JSON with SOAP) and value. The ‘mash-up’ approach to service integration pioneered by web developers is an increasingly viable approach for building large-scale, enterprise-strength systems.