Coming to the end of my first full year developing systems built exclusively on and for cloud platforms (primarily force.com) I figured this was a good time to reflect on some of the lessons learned and some old lessons unlearned.
How I learned to stop worrying and love the platform
[With apologies, Stanley Kubrik]
For 13 years or so prior to committing to the cloud I was a professional worrier. I had no doubt the various teams I worked in could deliver features and good code but part of my job was to worry about the non-functional characteristics of the system. How would it perform, how would it scale, would it be available, would it be secure, was it manageable, was it maintainable, etc. and so on? I like to think I got pretty good at making sure our systems did what they needed to in all these areas by building up a toolkit of common issues and their solutions.
Building on force.com and S3, there’s just no point worrying about most of these. I have no control whatsoever over the scalability or availability of the system. The security is what it is:secure to a degree that no project I’ve run ever had a budget to achieve; and we’d have to be trying pretty hard to leave security holes in our product. Manageability is built into the force.com platform and following the conventions means we get a high degree of manageability in our application ‘for free’. Maintainability is still under our control but this goes back to good old-fashioned code-quality: make sure the code is understandable, well factored, and covered in tests and you’ll be okay. Even performance, to a large degree out of our hands, requires a new way of thinking if we’re going to do anything about it.
So lesson learned/un-learned number one: take all the time and energy you used to spend worrying about the architecture and non-functional characteristics of the system you’re building and divert this into delivering great products and services. You need to understand what the platform offers in terms of security, etc. but there’s not much you can actually do to change it so you might as well learn to love it.
Don’t upset the Algorithm, baby
[With anguished apologies to The Noisettes]
On a walk through the Scottish hills with @johnsnolan he mourned the death of the algorithm. John’s view – which I share – is that many developers no longer understand the wide variety of algorithmic approaches available to them, nor is there enough discussion of algorithm as a subject. As John said: “Not everything can be solved with fucking Map-Reduce!”. Design patterns, object-orientation, frameworks, libraries and so on have lead to a homogenisation of development approach. Perhaps the current slightly hysterical vogue for functional programming languages is in part due to the frustration that, as developers, we spend more time assembling other people’s clever solutions than we do writing our own?
But this homogenisation is not a wholly bad thing. In teams of any significant size, having clear code understandable by the whole team is often better than having incredibly clever or efficient but somewhat obscure code. And you can have your cake and eat it: if your simple code is a bit slow or memory inefficient you can always spec a bigger server, throw a few new database indexes into the mix, up the thread pool size, up the cache size and so on.
But not on someone else’s platform.
On someone else’s platform algorithm is about the only tool you have at your disposal if you want to speed things up or make them more efficient (and on force.com, as on Google’s App Engine, there are some hard and fast limits on things like heap size to ensure your application plays nice on their servers which makes a degree of efficiency imperative). Force.com is a extremely well-optimised platform but its still possible to screw up end-user performance by doing too much calculation or returning too much data for any given request.
Lesson number two: spending time crafting efficient algorithms and designing a UX that supports these (by, for example, loading large data sets in pages rather than all in one go) is the best and possibly only option for ensuring consistently good performance.
It’s a Mashable, Mashable, Mashable, Mashable World
[No, I’m not apologising again]
Probably the nicest lesson to learn, and one that I addressed in my previous post, is that the cloud isn’t about one platform or one language but about creating and consuming services written and integrated in a variety of different languages running on a variety of different technologies. I hate the term ‘Mash-Up’ but there is something enduringly wonderful about the idea that you can take all of these services and platforms and splurge them together in different and interesting way. Its like the anti-SOA … a primordial mass of open and accessible stuff with no-one curating, standardising or organising it.
Lesson number three: developing applications for the cloud is about having some knowledge of a broad range of platforms, technologies and languages at your disposal, and an appreciation what each of them can do for you, rather than understanding one technology or language inside-out.
So my New Year’s resolution is to worry less, reaquaint myself with Djikstra, and learn a bit of Ruby and Heroku and Node.js and …