I have been involved in very many successful agile projects and have got involved at different stages in their evolution. In some of my earlier projects at companies such as Parallax I was pioneering agile techniques alongside people such as Dave Farley, who went on to co-author the seminal book Continuous Delivery. Later with organisations such as Commerce Decisions Ltd I continued to work on successful projects with people who were developing the ideas around automated deployment and test driven development.
What characterised these early forays was the feeling that we were doing something new and bold we had all worked in more traditional waterfall processes before and had found them lacking, believing there must be a better way, with the enthusiasm and courage of pioneers we drove forward into the brave new world which others made were also looking at and gave us the Agile manifesto.
More recently I worked with Dave again on his flagship project at LMAX this was a project that he, Martin Thompson and others where they were able to pose the question “What if we did all this agile stuff right from the start, where would we end up?” I’m still in touch with many of the excellent people there and I believe the evidence speaks for itself, LMAX were #1 in the Sunday Times Tech Track 100 in 2014 and I am sure that the excellent tech such as the disruptor that they developed and expertise using continuous delivery techniques have been a huge contributor to their success. Beyond this unlike many companies, they continue to get faster and better at delivering software all the time as demonstrated by being named best overall testing project in testa 2013.
So this agile stuff works and we know it works, to me it seems that software process is now largely a “solved problem” so why is there still resistance to uptake, clinging to upfront design, planning and other ideas that have been shown to fail again and again?
Walking the tightrope
I think that one of the key barriers to adopting agile is fear, different individuals carry different understandable levels of fear with them. I find that the young and bold often lack the knowledge and experience of the pain when software projects go bad. These people quick to embrace the challenge and excitement of getting to production code quickly, whilst older more conservative types show more caution, fearful of letting go of the supports of careful upfront design and planning. We should not be surprised at this; for them letting go of these practices is like suggesting they cross a raging river by slinging a line across and walking the tight rope, with no safety net and no practice.
(By George Barker – from stereographic image, Public Domain, https://commons.wikimedia.org/w/index.php?curid=9032429)
This is understandably a terrifying idea to people who have been more accustomed to carefully designing a bridge whilst doing their best to predict every future risk and possibility with great attention to detail.
So should they just toughen up?
Of course not, the truth is they are absolutely right to be cautious! Quickly slinging a line across a ravine and leaping onto it with no kind of safety net or training is not bold, it is reckless. I have seen many people being burnt by doing exactly this, among my peers we have often referred to this as having been “Scrummed”. This is because of how scrum, and by association agile, is often implemented with an over emphasis placed on process and lack of emphasis on the development techniques that eXtreme Programming and Continuous Delivery demand. Moreover there is a perception that agile is all about process when the reality is that agile is a collection of techniques that enable a different approach to software design and development. It is precisely these techniques that provide the essential safety net that makes the walk on the agile tight-rope both exciting and rewarding, but ultimately safe. Practices that include;
– The build pipeline – “Continuous integration on steroids”
– Acceptance Test Driven Development(ATDD) – to ensure a full regression suite
– Automated Deployment – to ensure we can actually deploy and crucially upgrade the thing,
– Performance testing – so we know it meets memory and processing requirements
– Merciless re-factoring – to ensure we change the software to reflect our evolving understanding of the world rather than ram our requirements into a model that no longer works.
– Test Driven Development (TDD) – to help us build software that has high cohesion, low coupling, small methods and classes all those good design principles that enable easy re-factoring (Yes that’s right TDD is not really about testing at al you heard here first)
– You aren’t going to need it (YAGNI) – so we don’t loose time maintaining the inventory of code that nobody is using – Keeping the build always releasable – the discipline to ensure we are always integrated and always green forces us to consider quality on every commit. To name but a few.
Be careful out there kids!
Simply dropping upfront planning and design without first gaining and applying these essential skills is, I am convinced, the reason many peoples first experience with “agile” ends with them crashing onto the rocks below. I think we need to be careful how we introduce people to agile, we need to emphasise the safety aspects, spend time building in testing (especially ATDD) and learning how to do good design with TDD.
Beyond this we should also remember that we are now trying to bring agile to the next wave of settlers, not the pioneers and that they, perhaps, are not quite so bold as some early adopters. Moreover simply putting the safety net in place does not in itself enable everyone to show courage, as Martin Fowler discussed, its a balance we need to let people gain height gradually and learn to trust the new practices that protect them and enable them to build better software better and faster.
After all if you have never walked on a tightrope, no matter how good the safety net looked, I suspect most of us would be apprehensive (to say the least) about striking out across Niagra Falls on a slack line.