Between 1998 and 2001 I introduced XP into a number of fairly large and un-agile businesses. In some cases this radical new approach to delivering software was welcomed as potential saviour from the the failing processes that were in place, in others it was treated with suspicion and even fear. In all cases we ran into a dissonance between the goals, agenda and timescales of ‘the business’ and the goals, agenda and timescales of the development process. We were great at repeatedly delivering working software and responding to change at a detailed level. We were not so good at committing to business deadlines, reporting summary status information and responding to change at an organisational level.
Inspired by Brand’s writing on Shearing Layers I began to think of software projects and the organisations they’re embedded in as layers of differing rates of change with different agendas and goals rather than management hierarchies or org charts. I was particularly interested in the idea that, at the highest level of some of these business, being able to change strategy or re-prioritise objectives on a three or six-month timescale was a radical improvement over their current two or three-year plans, where as the development teams implementing the software to support the business was working on daily builds and weekly releases.
Over time I developed a model of Shearing Layers in software delivery. Unlike Duffy and Brand, I’m not interested in proposing a universal model that works for everyone but rather a ‘reasonable model’ that has worked for me over the past twelve years and may work for you. You may need to strip a level out, add some in, fiddle with the boundaries of the layers, or possibly just identify the different rates of change in your organisation and work with those.
What’s important to recognise about this is that it is not an organisational hierarchy. Very senior people can and do operate at all levels, as do very junior people. Individual roles may be played across multiple layers or may be confined to just one; line management structures are totally orthogonal. It is also not a series of communication layers with single points of contact or gatekeepers ‘between the layers’ – there is no reason ‘the business’ can’t talk directly to the software delivery team (and, in an XP process, they should be doing so).
What it is is a recognition that there are different roles, agendas and rates of change at work in any software project. Communication between layers needs to respect these differences and work with them: it would be absurd for a business manager to come and talk to a developer about their worries over the long-term profitability of the project the developer was part of, just as it would be for the developer to seek out the manager for some help with debugging.
In later posts I’ll talk about how different roles, especially those of customer and user, sit within the layers as well as add a bit more detail about the timescales they operate on and the practices that introduce much-needed ‘slippage’ between the layers. For now I’ll just describe the layers in the model.
I’ve founded and run three SME businesses since 1999 so, to some extent or other, I am a ‘businessman’. But I have to say that the practices and principles that guide the boards and management of the large corporations my SME businesses supplied with software delivery skills are a total mystery to me. That’s okay, I don’t need to know how they come up with the various controls and constraints that they apply to software delivery projects to know how to work within them. And I think its important for any software professional to recognise that, whether they’re right or wrong, whether they’re well applied or ill, these constraints are real and exist for a reason. It pisses me off when I hear agile practitioners say things like “we deliver working, valuable software every two weeks; you can’t ask us to commit to a deadline in nine months time” or “if we were a bit more agile as a business, this development process would work a lot more smoothly”. The fact is that most corporations (even fairly small ones) work on timescales measured in years and plan accordingly. Until this changes, and its entirely possible that it never will, us software guys have to learn to work with it rather than rail against it.
Decisions tools: KPIs, Budgets, Strategies, ???
Outputs to Project Management: Goals, Objectives, Deadlines, Budgets, Business Plans
Timescales for key events: Two- or Three-Yearly, Yearly, Quarterly
Project management is about delivering a software project or programme to meet the needs of the business, within the constraints laid down by the business. I’ve written before about the role of the Project Manager in an agile project but the key thing is that project management is largely a facilitating job. The constraints and objectives set by the business often have to be translated into a different form such that they can guide product management and the development team. Equally the successes and otherwise of the people actually involved with building the software need to be described in a way that is comprehensible and useful to ‘the business’. Project Management also has to take a longer-term view than Product Management or Software Delivery team: assuming that the next three months will work roughly as planned, what changes to the team, the project or the inputs from the business might be needed after that?
Inputs: Goals, Objectives, Deadlines, Budgets
Decision tools: (Financial) Burn Rate, Project Velocity, Personnel Reviews,
Outputs to Business Management: Budgets, Hiring & Firing Plans, Risks and Issues, Status Reports
Outputs to Product Management: Project Definition, Budgets, The Delivery Team, Deadlines, Milestones
Timescales for key events: Yearly, Quarterly, Monthly, Weekly/Fortnightly
When I first started running XP projects that actively needed practices to be aligned to these Shearing Layers, Scrum was not well-known and there was no agile movement or Manifesto. So we expanded some of the XP practices to deal with the concerns of Project Management and Business Management and added a few of our own. However now I think it is easier just to think of Product Management as consisting of the Scrum practices of Product Backlog, Sprint Planning, and some additional practices for release planning and customer engagement.
Inputs: Requirements (in some form), Priorities, Team Capacity, Project Velocity
Decision Tools: Planning Game/Planning Meeting
Outputs (to both Project Management and Software Delivery): Sprint Plan, Release Plan, Product Backlog
Timescales for key events: Monthly, Fortnightly, Weekly
I used to call this layer ‘Software Development’ but, even today, I still meet people that think ‘Software Development’ begins with some requirements of some form and ends with the code written by the development team. So I chose ‘Software Delivery’ to try and be clear that, whilst it may being with some requirements of some form, it doesn’t end until some working software is in front of a – hopefully – pleased and grateful user (and, of course, it doesn’t really end there unless there are simply no more releases to produce).
In my world Software Delivery pretty much means XP, with the four key practices at its heart. Your particular poison may be different although I’ve yet to see a credible alternative to XP that tells coders how to go about implementing and delivering high-quality software in an agile manner.
Inputs: Sprint Plan
Decision Tools: Planning Game, Task Board, The Code
Outputs: Daily Build, Product Releases, Documentation
Timescales for key events: Weekly, Daily, Hourly