Category Archives: Agile

Can Continuous Delivery succeed where Agile has failed before?

It’s my hypothesis that Continuous Delivery through its focus on the actual mechanics of the production of software provides us a step change in how we deliver value through software to the business. I also argue that the failure of what is perhaps the most popular Agile approach in use today, Scrum, to put this at the heart of the development effort has been a key contributor to the disappointing results of many agile transformation programs.

The leader on scrum.org on “What is Scrum?” says;

“Scrum is a management and control process that cuts through complexity to focus on building software that meets business needs. Management and teams are able to get their hands around the requirements and technologies, never let go, and deliver working software, incrementally and empirically.”

From https://www.scrum.org/Resources/What-is-Scrum

So guess how many times the Scrum Guide, also available from that site, mentions the words  “Software” or “code”? Quite a lot you’d think yes? But no, the actual number of times is zero, not once, zip, zilch that’s right not one single mention of the very thing its supposedly focussed on building. In fact I’d argue that if you read this guide and did not already know that Scrum was about building software, there is no way you could ascertain that from the text.

Don’t get me wrong, there are many good things in Scrum, but it’s focus exclusively on the organisational patterns and processes is simply not enough, on its own, to ensure the delivery of working software that meets business needs.  This much was recognised by Jeff Sutherland when he wrote about the very first Scrum project;

“…a key to entering a hyperproductive state was not just the Scrum
organizational pattern. We did constant component testing of topic areas, integration of packages, refactoring of selected parts of the system, and multiple builds per day.”

From AGILE DEVELOPMENT: LESSONS LEARNED FROM THE FIRST SCRUM By Dr. Jeff Sutherland October 2004
https://www.scrumalliance.org/resources/35

Actually talk of code is very much at the fore in this early six page document on Scrum; it mentions code 12 times and software 36 times. Somewhere along the line however it seems that Scrum has forgotten to emphasise this central element of a successful product, one that to my mind is far more fundamental than organisational patterns. Unfortunately I’d argue that it is these elements of eXtreme Programming that were much more crucial to the success of a software project than the organisational concerns that Scrum majors on.

Ironically this lack of talk about code, I think, is one of the reasons using Scrum as the route into agile has been a popular choice as the route in for non-technical management. It talks about things they understand and inputs they can influence without worrying them about the tricky technical stuff.

Because we fail to ensure the basic building blocks of continuously delivering working software are in place we have repeatedly undervalued the importance of key technical skills. Its my experience that introducing agile organisational practices without the fundamental technical practices in place is not only unlikely to improve matters but can be downright dangerous – trying to deliver working software every couple of weeks without test automation, or clean code inevitably slows to a point of failure. Often this will result in cargo cult ceremonies instead of actual improvements. You may have seen the BBC Red Dwarf episode where Rimmer spends most of his six week period of revision fine tuning his revision timetable, leaving him with no time to actually do any revision. It seems to me that the focus on organisation patterns and process we see so often in agile leads to us failing to spend any time on actually writing better software.

Enter continuous delivery;  in contrast to the  Scrum Guide Continuous Delivery mentions software 588 times and code 405 times, now I accept that this is a bigger document but at 463 pages that’s still over twice per page on average for the two words combined. There is no doubt what this book is about – its about automating the software delivery process. It focuses on the desired outcome – how to get working software reliably and predictably into the hands of users.

Continuous Delivery, by building on XP  and Lean, provides us,  I think, with a step change in software delivery. By bringing the basic engineering practices of software, automation, building in quality, TDD, ATDD, clean code and code craftsmanship to the fore we put the code where it should be, right at the heart of our development efforts. All of the really effective development teams I have seen have these engineering practices at the core of how they deliver software. In fact by focussing on automating the mechanics of the software process they have, perhaps paradoxically, been able to consider the softer organisational issues that may be even more challenging.

Start with Continuous Delivery other good stuff can follow.

The fundamental building block’s of automation and good engineering practice must be seen as at least as important as the introduction of organisational patterns like Scrum. Scrum and CD practices can be used together successfully, but whilst great benefit will almost certainly be gained from CD alone, using Scrum in isolation is, I believe, doomed from the start.

As a final thought, if you are a non-technical manager reading this article you may be assuming your technical team are learned in these core technical practices, sadly the state of our industry is not so great. Anecdotal evidence suggests only around 1/3 of developers have or apply these core engineering skills on a regular basis, a quick search on a popular job search site today threw up the following numbers (within 100 miles of London);

2,218 jobs for agile
779 jobs for (BDD or ATDD or TDD or “clean code” or craftsmanship)

Not very scientific, I admit, but its consistent with other anecdotal evidence from show of hands at conferences etc. and it gives an indication of the perceived relative importance of these skills. I say again I have never seen Agile Software development be a success without these things at the core – Like Rimmer we need to stop worrying about our revision time table and focus on the actual revision.

See my talk on how implementing continuous delivery turned around our development efforts here.

If you want the benefits of pair programming – Start by sorting the furniture out.

Lets say you’ve read the various justifications for pair programming and you have decided to give it a trial or perhaps you’ve already tried it and not been successful. What things can you do to give pairing the best chance of success and make it a pleasant experience for all?

Lots has been said about the day to day practice of pairing but here I want to look at some of the more practical physical office considerations that can, in my view make or break a team.

Sort the furniture outPairing station vs

A poorly laid out office can destroy your chances of a successful pairing experience before you have even begun. If your desks are arranged like those on the left above you are not going to have a fun time. Not only this you could even be breaking regulations on providing a safe working environment for office workers. The pairing station on the right shows the set up at transFICC.com

Get rid of those L shaped desks and dividers if you have them Straight desks with no obstructions underneath provide a much more comfortable place to sit. Aligning PC ‘s lengthways at the back of the desk can free up leg room and don’t have under-desk pedestals – put them at the end of desk rows if you must have them.

This is the single most important piece of advice in this article if you do nothing else – do this.

Provide breakout areas

Lots of drawing on walls Gavin Tapp CC BY 2.0 via flickr.com

“lots of drawing on walls” Gavin Tapp CC BY 2.0  via http://www.flickr.com

Its not all typing. Its really important not to spend all the time coding at the workstation, we often need to go and sketch out ideas and discuss them away from the screens. Pairs should be encouraged to and given somewhere to breakout and have design discussions. Seating areas with tables and whiteboards everywhere should be the norm for any agile team. Painting the walls with whiteboard paint is really effective.

Provide standardised machine configuration

My laptop stickers came Doug Belshaw CC BY 2.0 via fickr.com

“My laptop stickers came” Doug Belshaw CC BY 2.0 via fickr.com

How someone has their machine set up can be a very personal thing, keyboard short-cuts, themes etc. This can make it very difficult to enable a sense of shared ownership and collaboration. So don’t just pair by bringing along one of the developers laptop’s. Get  pairing stations set up with two keyboards, two mice and two screens. Configure these in a standard way ideally with automation such as puppet or chef.  This enables you to clean install the station every night and will ensure that any developer can work on any workstation and will immediately be in a familiar environment. It also encourages the frequent small check-ins approach used in XP & Continuous Delivery.  You should be checking in your IDE configuration with your source code to ensure “Everything is in Source Control”.

I recommend the team agrees on a single IDE (or text editor if you must, if you are still having vim/emacs wars maybe this is an opportunity to move on a decade) for pairing.

Tip have a USB hub on the desk with spare slots – this allows people to plug in personal mice/keyboards if they have particular styles they like or want to for hygiene reasons, it also provides somewhere to charge your phone 🙂

This is an opportunity by the way to get a much higher specification machine for development – you need only enough machines for one between two.

Hot desking

Pairing station

“Pairing Station” by Andrew CC BY SA 2.0 via http://www.flickr.com

Pairing with the same person for day after day not only gets monotonous it also misses out on the knowledge sharing and team building that pairing brings. You need to be rotating pairs at least daily, for this reason I don’t recommend trial pairing with just a single pair you are unlikely to see any real benefits. If you are doing this, hot desking is really a pre-requisite as people are, by definition, moving around every day. But you can think about a couple of things to make this run better.

Make sure every desk is equipped with a fully set up pairing workstation as mentioned above with dual keyboards and screens.

Have a bank of desks dedicated to each team but encourage full hot desking within that area.

Many people want to keep using the same chair so just get some labels and stick them on its a small price to pay for comfort.

Don’t forget to have white boards and the team board near by.

Clean desk policy

office-932648_640

 CC 0 Public Domain via Pixabay.com

Clean desk policy can be controversial but it ensures that desks are truly hot.

Have someone responsible every evening (Cleaner or office admin perhaps) for clearing all the papers and books into a document tray

Also have them replenish every desk with a supply of post-it notes, sharpies, pens, note paper and whiteboard markers. When we started doing this almost no-one complained any more about the clean desk policy, it seems people value having tools to hand over an untidy desk.

Final thoughts

Although this article is written with pairing in mind, even if you are not pairing regularly many of these tips still apply and will in themselves encourage greater team collaboration. In particular sorting out the furniture, providing breakout areas standard machine setups with dual keyboards and screens will allow ad hoc pairing to occur on an informal basis. The “Could you just help me with this a minute” situation.

So before you embark on your pairing trial I would strongly recommend these measures, I consider the furniture and standardised workstations pretty much essential.

  • Sort the furniture out
  • Provide Breakout areas
  • Provide standardised machine configuration
  • Hot desk
  • Clean desk

 

Further reading

Pair Programming – The most Extreme XP Practice? – Dave Farley

Why pair programming is as much about business continuity as it is about code quality – Dave Hounslow

 

 

 

 

Tagged ,

How Branching Affects Team Culture – Talk at Pipeline Conf 2016

At the Pipeline Continuous Delivery Conference 2016 I was given the opportunity to talk about our experiences introducing continuous delivery at http://www.your-loop.com

“Great story of agile & XP”

“Great arguments, nicely supported with actual examples from practice”

“Great examples and very engaging”

Covering themes I explore elsewhere on this blog including Pairing and Automated Testing, This was my first conference talk and I am delighted to have received overwhelmingly positive (91%) feed back, thanks everyone.

And thanks to all at the pipeline team for organising a great event!

Slides from the talk are available here.

Dave Hounslow – You Are What You Eat – How Branching Affects Team Culture – PIPELINE Conference 2016

Tagged ,

5 ways to avoid the Technical Debt Trap

Technical debt is toxic debt, much like a payday loan you take on technical debt often in a state of panic when that feature just has to be out there right now. Further just like a payday loan if not paid off quickly it can rapidly become a millstone around your neck where you find yourself forever in a paralysing technical debt trap.

So what can be done to avoid the pain of Technical Debt in the first place?

#1  Acceptance Test Driven Development

Code that has no tests cannot be refactored safely, code that cannot be refactored is code that will rot, the second you write code without a regression test you have taken on some debt.

For me ATDD is even more important than TDD in this respect, once you have worked on a system with close to 100% acceptance test coverage you will wonder how you ever made changes safely without it. I have had to go back to working on systems that lack good acceptance test coverage having worked on some with close to 100% coverage and it is paralysing. More than this it is shocking to see the different attitude of developers to refactoring in such environments, without full regression suites (and if you don’t use ATDD you probably haven’t got a full regression suite) the attitude becomes “If it ain’t broke don’t fix it” rather than constant improvement. By using ATDD and treating the test as the source of truth you enable the freedom to change how code does stuff without changing what it does.

 

#2 Don’t feature branch – work on trunk

Feature branches discourage refactoring. I hadn’t really appreciated this benefit of trunk based development until recently when a colleague of mine rasied it. He had previously always worked with teams who used feature branching, when he moved to our team which was using trunk based development he was blown away by the freedom it gave him.  He enthusiastically declared

“It’s fantastic I can do fundamental refactoring’s knowing I won’t have people breathing down my neck at some point in the future because they have merge hell”.

Sound familiar? With trunk based development the worse case scenario is maybe ½ a days worth of code to merge in.

I find it ironic that feature branching is often championed as allowing developers to work independently on features when the truth is the opposite.
 

#3 Test Driven Development

TDD encourages good design by nudging us in the direction of loose coupling, strong cohesion and clean encapsulation. It doesn’t of course guarantee any of these things by my experience is that coders who have and do practice TDD tend to write cleaner more maintainable code. Code that is well written is easier change our efforts upfront reduce the debt to the future.

 

#4 Refactor all the time

Good citizen’s pick litter up and don’t add to it carelessly so always leave code a little better than you found it. Personally I find a little bit of refactoring is a great way to understand how the code does what it does. When faced with a method or a class that does more than one thing if we start breaking it up into smaller methods teasing apart layers and introducing abstractions its purpose becomes clearer. This has benefits both for us and for future pairs who work on the code.

 

#5 Limit your work in progress

Control your work in progress, working at a sustainable pace is key to maintaining velocity. If you are constantly firefighting technical debt will keep piling up. Techniques such as swarming can help you here.

Finally

Remember that ultimately it usually costs a lot more to pay back than what was saved by doing it well in the first place. Left to fester you end up taking on another piece of debt to cover the debt you already took out, then another and another until the debt becomes crippling. It’s much better to save up and invest in quality upfront than borrow from your future productivity.

Tagged , , , ,

Is this a feature request or a bug?

How does a bug differ from a feature request?

I’m sure like me you have been in many a heated discussion on whether something is a bug or a feature. In my view a great deal of time is wasted in these arguments that could be spent more productively.  A long time ago I concluded that, at best, it wasn’t a valuable distinction, at worst that to distinguish them is in fact damaging.

Consider the following possible definitions of bugs and stories


Feature Request

A proposed change to how your software currently behaves in the hands of users to deliver greater business value.


Bug

A proposed change to how your software currently behaves in the hands of users to deliver greater business value, which you want to blame the development team for doing wrong in the first place.


Lets think about two important points in these statements

  • “Currently behaves In the hands of users”
  • “which you want to blame the development team for”

“Currently behaves In the hands of users”

If no one can see this behaviour its not done yet. So during story development we should simply adjust and fix things as we find them – we want to get to done not half done, if you need to raise and track bugs at this stage you probably have one or more of the following problems or something else that needs to be addressed as the root cause

  • Stories too big – taking too long – think about how you can break down the delivery of value
  • Working in silos, with lack of integration of QA/BA/Developers/Product Owner’s
  • No definition of done – “In the hands of users” is a pretty good place to start in my book.

“Which you want to blame the development team for”

This really seems to me to be the nub of the issue. When we call something a bug there is an implicit message that the development team did a bad job. My experience is that development teams generally set themselves very high standards on quality that this attitude can damage morale and ultimately lead to lower quality and rate of delivery. Calling things bugs can lead to wrong expectations such as;

  • the cost of fixing it will not impact other work – devs did bad they can make up the time
  • its automatically the most important thing to do which it may or may not be

The “No Bugs” messaging that has been around can also lead to prioritisation of bugs over stories when the bug fix may have minor business value compared to other changes in play.

Given these two definitions is it a useful distinction?

It is my contention that distinguishing bugs from stories adds no value. I have had good success with treating bugs and stories the same once the software is in the hands of users. They both have some business value associated with them and they both need to be prioritised based on that value. So having a single backlog makes much more sense to me. Moreover separating these out into separate tracking systems or even labelling them bugs or stories in the same system is a bad idea stop doing it now!

But people like raising bugs I’ll never be able to get rid of <insert favourite bug tracking software here>?

Depending on your company culture you will find it easier either to call everything a feature or everything a bug. I don’t think it actually matters much, although it can be fun to start a new project with a bug along the lines of “Unable to see login page” before a single line of code has been written. Whichever way you go my recommendation is to pick one and treat it as a single backlog.

A word of caution, if you call everything a bug remember ONLY the product owner should control the priority of the backlog that means sysadmins, customer service etc don’t get to set the priority – sure they get a say but the product owner has to balance all the business priorities and make the call.

Tagged ,

How to succeed at agile – remember the safety net.

I have been involved in very many successful agile projects and have got involved at different stages in their evolution. In some of my earlier projects at companies such as Parallax I was pioneering agile techniques alongside people such as Dave Farley, who went on to co-author the seminal book Continuous Delivery. Later with organisations such as Commerce Decisions Ltd  I continued to work on successful projects with people who were developing the ideas around automated deployment and test driven development.

What characterised these early forays was the feeling that we were doing something new and bold we had all worked in more traditional waterfall processes before and had found them lacking, believing there must be a better way, with the enthusiasm and courage of pioneers we drove forward into the brave new world which others made were also looking at and gave us the Agile manifesto.

More recently I worked with Dave again on his flagship project at LMAX this was a project that he, Martin Thompson and others where they were able to pose the question “What if we did all this agile stuff right from the start, where would we end up?” I’m still in touch with many of the excellent people there and I believe the evidence speaks for itself, LMAX were #1 in the Sunday Times Tech Track 100 in 2014 and I am sure that the excellent tech such as the disruptor that they developed and expertise using continuous delivery techniques have been a huge contributor to their success. Beyond this unlike many companies, they continue to get faster and better at delivering software all the time as demonstrated by being named best overall testing project in testa 2013.

So this agile stuff works and we know it works, to me it seems that software process is now largely a “solved problem” so why is there still resistance to uptake, clinging to upfront design, planning and other ideas that have been shown to fail again and again?

Walking the tightrope

I think that one of the key barriers to adopting agile is fear, different individuals carry different understandable levels of fear with them. I find that the young and bold often lack the knowledge and experience of the pain when software projects go bad. These people quick to embrace the challenge and excitement of getting to production code quickly, whilst older more conservative types show more caution, fearful of letting go of the supports of careful upfront design and planning. We should not be surprised at this; for them letting go of these practices is like suggesting they cross a raging river by slinging a line across and walking the tight rope, with no safety net and no practice.

Samuel_Dixon_Niagara
(By George Barker – from stereographic image, Public Domain, https://commons.wikimedia.org/w/index.php?curid=9032429)

This is understandably a terrifying idea to people who have been more accustomed to carefully designing a bridge whilst doing their best to predict every future risk and possibility with great attention to detail.

So should they just toughen up?

Of course not, the truth is they are absolutely right to be cautious! Quickly slinging a line across a ravine and leaping onto it with no kind of safety net or training is not bold, it is reckless. I have seen many people being burnt by doing exactly this, among my peers we have often referred to this as having been “Scrummed”. This is because of how scrum, and by association agile, is often implemented with an over emphasis placed on process and lack of emphasis on the development techniques that eXtreme Programming and Continuous Delivery demand. Moreover there is a perception that agile is all about process when the reality is that agile  is a collection of techniques that enable a different approach to software design and development. It is precisely these techniques that provide the essential safety net that makes the walk on the agile tight-rope both exciting and rewarding, but ultimately safe. Practices that include;

The build pipeline – “Continuous integration on steroids”

Acceptance Test Driven Development(ATDD) – to ensure a full regression suite

Automated Deployment – to ensure we can actually deploy and crucially upgrade the thing,

Performance testing – so we know it meets memory and processing requirements

Merciless re-factoring – to ensure we change the software to reflect our evolving understanding of the world rather than ram our requirements into a model that no longer works.

Test Driven Development (TDD) – to help us build software that has high cohesion, low coupling, small methods and classes all those good design principles that enable easy re-factoring (Yes that’s right TDD is not really about testing at al you heard here first)

You aren’t going to need it (YAGNI) – so we don’t loose time maintaining the inventory of code that nobody is using – Keeping the build always releasable – the discipline to ensure we are always integrated and always green forces us to consider quality on every commit. To name but a few.

Be careful out there kids!

Simply dropping upfront planning and design without first gaining and applying these essential skills is, I am convinced, the reason many peoples first experience with “agile” ends with them crashing onto the rocks below. I think we need to be careful how we introduce people to agile, we need to emphasise the safety aspects, spend time building in testing (especially ATDD) and learning how to do good design with TDD.

Beyond this we should also remember that we are now trying to bring agile to the next wave of settlers, not the pioneers and that they, perhaps, are not quite so bold as some early adopters. Moreover simply putting the safety net in place does not in itself enable everyone to show courage, as Martin Fowler discussed, its a balance we need to let people gain height gradually and learn to trust the new practices that protect them and enable them to build better software better and faster.

After all if you have never walked on a tightrope, no matter how good the safety net looked, I suspect most of us would be apprehensive (to say the least) about striking out across Niagra Falls on a slack line.

Tagged

The thinkfoo test for Continuous Delivery – What does good look like?

Dave Farley makes an excellent point in his recent post that many people in our industry have never worked on an efficient project team and wouldn’t know good if they saw it. Indeed many are probably blissfully unaware of just how inefficient they are comforted in the knowledge that they seem to be no worse than anyone else. So how do you know if you are good?

Some fourteen years ago Joel Spolsky wrote an often quoted piece he called the Joel Test.

This was great stuff back in 2000 but we have moved on a long way since then and I propose an updated test that reflects modern development practices founded in the principles of Continuous Delivery (CD).

I’ve worked with some incredibly productive teams (and some not so productive ones) I prefer to work with productive ones and I have over the years formed some strong opinions of exactly what good looks like. I realised I have been informally using a set of criteria to evaluate teams for some time and thought I’d write it down and share it.

So here is my “thinkfoo test”; efficient development teams will be able to answer yes to most if not all of these questions, if your team can’t and you think you have an efficient team with a good process maybe its time to take the red pill and think again?

1. Do your developers push changes to a single trunk branch at least twice a day?

2. Do you practice ATDD and write automated acceptance tests for every feature?

3. Do you practice TDD?

4. Do you run your full set of acceptance tests, unit tests and performance tests on every commit to trunk?

5. Do you have automated single step repeatable builds and deployments?

6. Does your team release to production at least every two weeks?

7. Do you practice pairing as the normal way of working?

8. Do your teams self organise?

9. Is your team in a noisy collaborative environment?

10. Do you have cross functional teams of BA’s, developers, QA and operations responsible for end to end delivery of working software?

11. Do you have a single product backlog (with bug’s and features together)?

12. Do you use minimal stories as specifications rather than detailed up front designs?

13. Do you have regular retrospectives to continually improve your process?

Lets look at each of these in a little more detail

1. Do your developers push changes to a single trunk branch at least twice a day?

  • feature branching is evil, long lived feature branches doubly so. Trunk development is, for good reason, a key requirement for CD.
  • trunk should be in an always releasable state, every commit should be a release candidate.
  • To be continuous your integration must be frequent; committing to mainline twice a day should be considered a minimum.

2. Do you practice ATDD and write automated acceptance tests for every feature?

  • these should be built in advance and alongside the feature and treated as the source of truth or living specification. The feature isn’t done until all the acceptance tests pass. By doing this and keeping work in progress to a minimum the need for bug tracking is diminished.

3. Do you practice TDD?

  • many people are confused about what level to set their tests at that’s why I specifically want to know if you have both ATDD (are you building the right thing) and TDD (are you building it right).
  • The “red, green, refactor” cycle is critical to producing good clean code
  • Your whole team needs to practice TDD, I wince every time a development lead says “We don’t enforce it but sure you can work that way if you want”.

4. Do you run your full set of acceptance tests, unit tests and performance tests on every commit to trunk?

  • #1 is a prerequisite of this if you are not using a single trunk your continuous integration isn’t, end of.
  • performance tests are a critical part of your build pipeline many products fall under the pressure of their own success.

5. Do you have automated single step repeatable builds and deployments?

  • if you can’t do repeatable automated deployments there’s not much point in doing #1 through #4 as you won’t know that what you deployed is what you tested.

6. Does your team release to production at least every two weeks?

  • if you do iteration after iteration without release I don’t really buy that you are agile, this is usually a waterfall project dressed in agile clothing.

7. Do you practice pairing as the normal way of working?

  • see my post on pairing for my views on why this is essential to building quality software assets.

8. Do your teams self organise?

  • teams that do top down allocation of tasks in my experience always end up optimising resource usage over feature delivery, it discourages collaboration, common code ownership and pair rotation don’t do it!

9. Is your team in a noisy collaborative environment?

  • the opposite of Joel here, no single developer can be productive enough to jeopardise whole team productivity don’t allow it and don’t hire people who insist on working this way. Great software is built by teams not individuals beware the hero developer!

10. Do you have cross functional teams of BA’s, developers, QA and operations responsible for end to end delivery of working software?

  • and the roles should be blurred rather than designated, BA’s should write some tests, QA’s should write some code, developers should write deployment scripts and talk to customers, ops guys should be coding etc etc.
  • Your developers should be on first line support for your production systems.

11. Do you have a single product backlog (with bug’s and features together)?

  • do the most important thing first and optimise for flow of delivery over resource utilisation (see managing your backlog for more on this)

12. Do you use minimal stories as specifications rather than detailed up front designs?

  • stories are about enabling conversation not detailed specification see https://leanpub.com/50quickideas for some great thoughts on user stories.
  • Joel believes in detailed up front specifications sorry but he is just wrong on this, in any project lasting more than half an hour the specification will change, living specifications in the form of automated acceptance tests are the only way to keep specs current.

13. Do you have regular retrospectives to continually improve your process?

  • nothing is perfect; core to the principles of Continuous Delivery is continuous improvement retrospectives are an essential part of that learning and improving.
Tagged ,

TDD teaches you how to code well

You may have seen the recent   Kent Beck, David Heinemeier Hansson, Martin Fowler discussion on Is TDD Dead? where David Heinemeier Hansson contests that TDD is “poisoning” good designs. My take on this can be summarised with the below diagram;

TDDvsGood-2

Not all Test Driven Code is good and not all Good Code is test driven. But a lot of code (probably most) is neither good or test driven. I contest that to be considered good, code must be both well designed and well tested. Now I could have labelled the above “tested” or even “testable” but I have seen very little code that is written without TDD which gets anywhere near approaching “well tested” or even “testable” no matter how well intentioned the developers. Moreover I believe that by practising TDD you have much better chance of getting to good than by not doing so.

I say this because I have found that people who consistently write good clean code have almost invariably been people who have mastered TDD. Why could this be?

I think the reasons for this is are that in applying TDD you learn that keeping your tests manageable is at least as hard as writing the underlying code. To avoid brittle difficult to understand tests you want tests that test only one thing and you have to balance this with code that is itself manageable and does one thing well. Consequently;

TDD leads you to code that has small interfaces and low cyclomatic complexity.

TDD tells you things about your code; Tests that require a lot of dependant objects to be constructed or mocked are telling you your code has high coupling. Tests that test more than one thing can hint at poor cohesion – classes under test are doing too many things.

Learning about techniques like mocking and stubbing teach you about coding to interfaces rather than concretions.

TDD teaches you about best ways to handle error conditions and exceptions

TDD teaches you about separation of concerns, why for example its beneficial to separate threading from business logic.

As you get practice writing tests you learn that testing multiple layers of abstraction at the same time is hard and intuitively start to understand what layers of abstraction really mean.

TDD also teaches you to understand what it is to be a user of your code and code for the users rather than the implementers benefit.

In becoming a master practitioner of TDD you provide yourself with all these learning opportunities. Some people having done this feel that TDD can occasionally be overkill (Dan North talks about Spike and Stabilise and opportunity cost of TDD here) but it is the very expertise they have gained from TDD that gives them the ability to make this judgement and to proceed in a way that is safe to retrofit tests to. It’s worth pointing out though that most people I know that have have become competent at TDD never really want to code without it (myself included).

So finally if you are wondering if your code is good or bad and haven’t yet mastered TDD I would suggest it probably has considerable room for improvement.  Check out Steve Freeman and Nat Pryce’s Growing Object-Oriented Software Guided by Tests and Kent Beck’s Test Driven Development.

Tagged ,

Why pair programing is as much about business continuity as it is about code quality.

I heard a statement recently, it’s a common problem and is probably familiar to many of you.

It goes something like this ;

“We will need to wait until next week when Max is back from holiday to do that work, that module is his baby ”

When I heard this my initial reaction was confusion, why was no one else able to work on this code? It was in the same language, the same repository, was built in CI, had tests it made no sense. Then it dawned on me no one else in the team had EVER worked on this module, they had no clue what it did it how it did it, confusion turned to horror, did anyone know what this guy’s code was like? What if he left for good, got hit by a bus? Was this module going to rot forever or need rewriting?

I realised I’d heard this in a different form some time previously in another organisation from a new CTO who had not had extensive experience in a pairing environment;

“So Paul is leaving, what do we need to do for handover? ”

It left us old hands a bit stumped sure we were losing a very talented developer but there was nothing unique about his knowledge of our code that needed “handing over”.

The common negative response to pair programming from management is “I can’t afford to use two people to do one persons job” . As developers our common reaction to this is to explain how pairing can be faster, result in less bugs etc. This is understandable as these are the benefits most obvious to developer. But I would argue that it is business continuity that could be the killer argument for pairing.

Done correctly pairing addresses business continuity in a number of ways;

Knowledge sharing – many people have actually worked on the code this is far superior to a code review; think of pairing as having co-written the book and code review as being told the story by someone who has seen the film.

Code becomes easier to understand on each pair rotation (pairs should rotate or swap partners at least once a day to reap the benefits of pairing) as this rotation occurs small improvements in code clarity are made as the new person gets their head around what’s been done so far. This makes it a much more valuable and flexible piece of software six months down the line when some new requirement is needed of it.

No ego’s: the code is collectively written and ergo is collectively owned – no need to wait for Max to get back from holidays if it needs changing today it can be worked on today without upsetting anyone.

It’s a truism to say that code takes on the form of the team or teams that write it. If only one developer is working on a system the benefits of breaking down the code into modules are diminished a single person can only work on a single piece of code at a time. This is one reason why it’s common to see software written by a team of single developers be one big blob of software. I have seen pairing address this in this way: as pair’s rotate, realising that a piece of work can be done in parallel if the code is structured in a particular way they do so and your most important feature gets into your customers hands faster.

The software a company builds should be considered a company asset, not some individuals baby. I have seen pair programming to be an extremely powerful way to ensure this is the case. If your organisation is developing software the question you should be asking is not can you afford to have developers pairing but can you afford not to?

dave speaking

 

Watch my talk at the pipeline conference where I discuss introducing pairing and other XP practices to a team.

 

 

 

Tagged , ,

Using story swarming to reduce WIP

Some years ago I was on a training course on lean thinking and one particular exercise has stuck in my mind ever since; Teams of six or so were given a piece of A4 paper and were told to arrange themselves so that none of them were in contact with the floor. The solution most people took was simply to get all of the team to stand on the paper at the same time. The tutor then folded the paper in half and told us to repeat the exercise. It was still possible on one foot for the team to stand on the paper. But the paper was folded again and again until people were resorting to climbing on one another or declaring the task impossible. Finally the tutor took the piece of paper away entirely. Suddenly one team realised that the paper was completely surplus to requirements and the goal could be achieved even with no paper at all by having the whole team jump on the count of three. The point of this exercise was to demonstrate “Muda” or waste and how the presence of a tool or resource tends to make people use it even if in practice it is not needed at all.

One of the tenets of lean production, which as you will know is one of the foundations of the approaches to software development collectively know as agile, is the reduction of waste in all its forms. Inventory that is not being used and work in progress (which is product that is not yet ready) is an example of one of seven “wastes”. In software development code is inventory, if this code is not usefully in production it is my conjecture that this is waste and we need to find ways to minimise it.

One way of addressing this waste will be familiar to any practitioner of agile development; namely the use of an iterative approach, one of the benefits this gives is that we deliver useful features into the production code every iteration hence reducing the wasteful inventory of code that is work in progress. We also strive to prioritise our work so that the most important feature is delivered first and we do not put wasteful effort into less important features and to ensure that we do not “over deliver” excess inventory that is not yet needed..

You’ll hear it stated frequently that having a single product backlog that is ordered as a single queue is essential to a successful agile project. The idea being that work is done in strict priority order. In practice what often then happens is that each pair will grab the top available card in the backlog and begin work on it, the next pair will grab the second and so on. What this means is that in a team with six developers you suddenly have three cards running in parallel typically all placing conflicting demands on your QA’s, BA’s and the customer at the same points in an iteration which then leads to features being held up or bugs being found late in the day. Worse still it could be your top priority card gets held up because of some problem with a lower priority card that conflicts with (either through necessity or some error in the solution).

A consequence of which is that some or all (and not necessarily the lower priority ones) end up being incomplete at the time of cutting a release. Subsequently juggling of features which are ready or perhaps not wanted yet has to occur. Methods to tackle this include “feature branching”, “branch-by-feature” and “feature switches”. These allow us to switch on or off particular features either by clever use of version control systems or with a code switch to “turn off” code that is not ready. You will hear proponents of both approaches. But it seems to me that whichever approach is used the wrong problem is being addressed in that they are all ways of dealing with excess code inventory be it code that is not ready to put into production at the end of an iteration or code that delivers a feature that is not wanted yet.– there are great tools to help with this to be sure but my point is that if this is needed at all;

(a) There is excess inventory either in the form of unfinished code or in the form of features delivered ahead of when they are wanted or needed

(b) A process has been introduced purely to deal with this excess inventory, which in itself is a form of waste.

Swarming

So what else can we do to deal with this excess inventory? Well the thing here is to eliminate the excess in the first place.

An alternative approach to the pair by pair grabbing of stories, and one which is getting some attention, is to have your entire team “swarm” on a story until its done. This basically means that the whole cross functional team (Devs/QA/BA/Customer) works on the top priority item together breaking down the whole feature into small manageable tasks (often each of these takes less than a couple of hours for a pair to complete)

We have been swarming on stories for quite a while now and I would like to share my observations on some of the benefits we have seen;

  1. Our top priority story gets all the attention this means it gets delivered first and sooner than it would if the team was spread out on other stories we have found that even unusually large stories that we would have previously expected to take most of a two week iteration often get completed in a few days.

  2. The team delivers testable functionality early on in an iteration keeping QA’s busy and revealing bugs quickly

  3. The customer is focused on the item most important to them and has a clear view of the progress being made and what the team is working on

  4. More robust to absence whether planned or unplanned as all of your team knows about the feature being developed we have found for example that there is better pair rotation and BA’s and QA’s get included more frequently in those pairs.

  5. The code delivered ends up coherent as the context switching that can occur with pair rotation and multiple stories in play does not occur.

  6. And of course it encourages small frequent commits and CI

  7. We have found that we rarely need to “turn off” features with this approach at the worst case scenario you have a single story in play at the point you want to release and in practice this has not often occurred but where it has a relatively simple approach with feature toggling has sufficed – its only a single feature switched off and it won’t be in production switched off for long after all as its our top priority story for the next release.

In summary I think if you are finding the need to use advanced version control tools to manage feature branching, or regularly using coded feature toggles to manage multiple “in progress” features you are probably suffering from Muda in your process and I would highly recommend that you give swarming a try after all do you really need that piece of paper or can you just jump?

Tagged , , ,
%d bloggers like this: