Monthly Archives: April 2016

Can Continuous Delivery succeed where Agile has failed before?

It’s my hypothesis that Continuous Delivery through its focus on the actual mechanics of the production of software provides us a step change in how we deliver value through software to the business. I also argue that the failure of what is perhaps the most popular Agile approach in use today, Scrum, to put this at the heart of the development effort has been a key contributor to the disappointing results of many agile transformation programs.

The leader on scrum.org on “What is Scrum?” says;

“Scrum is a management and control process that cuts through complexity to focus on building software that meets business needs. Management and teams are able to get their hands around the requirements and technologies, never let go, and deliver working software, incrementally and empirically.”

From https://www.scrum.org/Resources/What-is-Scrum

So guess how many times the Scrum Guide, also available from that site, mentions the words  “Software” or “code”? Quite a lot you’d think yes? But no, the actual number of times is zero, not once, zip, zilch that’s right not one single mention of the very thing its supposedly focussed on building. In fact I’d argue that if you read this guide and did not already know that Scrum was about building software, there is no way you could ascertain that from the text.

Don’t get me wrong, there are many good things in Scrum, but it’s focus exclusively on the organisational patterns and processes is simply not enough, on its own, to ensure the delivery of working software that meets business needs.  This much was recognised by Jeff Sutherland when he wrote about the very first Scrum project;

“…a key to entering a hyperproductive state was not just the Scrum
organizational pattern. We did constant component testing of topic areas, integration of packages, refactoring of selected parts of the system, and multiple builds per day.”

From AGILE DEVELOPMENT: LESSONS LEARNED FROM THE FIRST SCRUM By Dr. Jeff Sutherland October 2004
https://www.scrumalliance.org/resources/35

Actually talk of code is very much at the fore in this early six page document on Scrum; it mentions code 12 times and software 36 times. Somewhere along the line however it seems that Scrum has forgotten to emphasise this central element of a successful product, one that to my mind is far more fundamental than organisational patterns. Unfortunately I’d argue that it is these elements of eXtreme Programming that were much more crucial to the success of a software project than the organisational concerns that Scrum majors on.

Ironically this lack of talk about code, I think, is one of the reasons using Scrum as the route into agile has been a popular choice as the route in for non-technical management. It talks about things they understand and inputs they can influence without worrying them about the tricky technical stuff.

Because we fail to ensure the basic building blocks of continuously delivering working software are in place we have repeatedly undervalued the importance of key technical skills. Its my experience that introducing agile organisational practices without the fundamental technical practices in place is not only unlikely to improve matters but can be downright dangerous – trying to deliver working software every couple of weeks without test automation, or clean code inevitably slows to a point of failure. Often this will result in cargo cult ceremonies instead of actual improvements. You may have seen the BBC Red Dwarf episode where Rimmer spends most of his six week period of revision fine tuning his revision timetable, leaving him with no time to actually do any revision. It seems to me that the focus on organisation patterns and process we see so often in agile leads to us failing to spend any time on actually writing better software.

Enter continuous delivery;  in contrast to the  Scrum Guide Continuous Delivery mentions software 588 times and code 405 times, now I accept that this is a bigger document but at 463 pages that’s still over twice per page on average for the two words combined. There is no doubt what this book is about – its about automating the software delivery process. It focuses on the desired outcome – how to get working software reliably and predictably into the hands of users.

Continuous Delivery, by building on XP  and Lean, provides us,  I think, with a step change in software delivery. By bringing the basic engineering practices of software, automation, building in quality, TDD, ATDD, clean code and code craftsmanship to the fore we put the code where it should be, right at the heart of our development efforts. All of the really effective development teams I have seen have these engineering practices at the core of how they deliver software. In fact by focussing on automating the mechanics of the software process they have, perhaps paradoxically, been able to consider the softer organisational issues that may be even more challenging.

Start with Continuous Delivery other good stuff can follow.

The fundamental building block’s of automation and good engineering practice must be seen as at least as important as the introduction of organisational patterns like Scrum. Scrum and CD practices can be used together successfully, but whilst great benefit will almost certainly be gained from CD alone, using Scrum in isolation is, I believe, doomed from the start.

As a final thought, if you are a non-technical manager reading this article you may be assuming your technical team are learned in these core technical practices, sadly the state of our industry is not so great. Anecdotal evidence suggests only around 1/3 of developers have or apply these core engineering skills on a regular basis, a quick search on a popular job search site today threw up the following numbers (within 100 miles of London);

2,218 jobs for agile
779 jobs for (BDD or ATDD or TDD or “clean code” or craftsmanship)

Not very scientific, I admit, but its consistent with other anecdotal evidence from show of hands at conferences etc. and it gives an indication of the perceived relative importance of these skills. I say again I have never seen Agile Software development be a success without these things at the core – Like Rimmer we need to stop worrying about our revision time table and focus on the actual revision.

See my talk on how implementing continuous delivery turned around our development efforts here.

Advertisements

Roulette Kata – exploring the affect of Time and Randomness in TDD

I did this Kata in response to a challenge set by @jjefries1 here
attempted by @RonJeffries here

For ease of reference the summary of the kata is repeated here;

A roulette wheel has numbers from 0 to 36. A spin of the wheel takes 20 seconds and leaves the ball on  a random number.

Design and implement a roulette wheel using TDD. Think about how using time and random numbers affects your design. Consider how much functionality each test covers and what responsibilities you are testing.

I wanted to drive this in a way where I did not need to mock out clocks or random number generators. To do this I thought that a pattern I have seen used extensively before would be suitable; where the generation of tick events was the responsibility of one class, whilst another had the job of responding to those events. Secondly I planned to  going with the model of “Object’s send messages to share data” so I knew straight away that I would be looking at a tell don’t ask approach.

The first notes I made on the kata were:

This is event based so avoid “World” stuff in test/codelets and to try for public methods are have void return types.

That about constituted the amount of “up front design I did before hitting the keyboard.

My very first failing, as opposed to not compiling,  test looked like this;

Note that at this point I am not taking any interest in time at all. Only that the observer is notified that the wheel has stopped after the spin.

I questioned where to go next and in writing my summary from the kata on cyber-dojo I think I went the harder route to start with. I took on verifying random number outputs, this took several goes round the red/green/refactor cycle and the final test I ended up with was a bit of a behemoth;

To push the design forward my next test looked like this;

This as you can see led to some changes in the interface. Which I didn’t jump straight to but my RouletteWheel class was starting to take shape;

There were quite a few more evolutions of this but the next interesting bit of work to talk about on a TickProvider class, something to generate those tick events

Again we are getting the basic communication right without worrying about timing.

to get to that we eventually ended up with;

Finally we got to the point where we needed to wire up an assembly of the RouletteWheel and the TickProvider. I did this by testing in TTDAIYMI (TDD as if you meant it) style writing the code directly in the test before extracting to a static method on RouletteWheel

The TDD approach has led my final code I think to show good separation of concerns between the generator of tick events (TickProvider), the consumer of those events (RouletteWheel) and the consumer of the stopped event that gives the final location of the ball (WheelObserver). This last one I particularly like as it totally separates presentation from the model, its also easy to envisage how the code could be adapted to publish intermediate events whilst the wheel was “spinning”.

No doubt someone will point out some horrendous threading (or other) bug, please note this is a Kata or practice and I didn’t pair on it. But if you do spot one can I suggest you respond by proposing a test that would expose it? Remember TDD is not about passing when software works, its about improving your design and as a bonus you get some tests that should fail when it doesn’t behave as you intended. I hope I showed that here.

For the full gist including test classes see below;

gist https://gist.github.com/daveneedstoknow/4cbb6a7d707d8efe4cb5fff1ee748923

You can also see my working on the excellent cyber-dojo here I hope no-one will make further changes at the time of writing my work goes up to revison 106. Note that revisions after that may not be mine.

You’ll see (as is my typical experience of TDD and coding in general) that progress is not quite as linear as some explanations will have led you to believe, including here 😉

The total time I spent on this was around the 3 hour mark so higher end of allotted time (I took a break in the middle to teach my nephew some TDD 🙂

I enjoyed the Kata, thanks to @jjefries1 for suggesting it. I tackles some of the more challenging things in TDD but I think it also shows that if TDD is followed it does give you that nudge to a better final design.

Tagged ,

If you want the benefits of pair programming – Start by sorting the furniture out.

Lets say you’ve read the various justifications for pair programming and you have decided to give it a trial or perhaps you’ve already tried it and not been successful. What things can you do to give pairing the best chance of success and make it a pleasant experience for all?

Lots has been said about the day to day practice of pairing but here I want to look at some of the more practical physical office considerations that can, in my view make or break a team.

Sort the furniture outPairing station vs

A poorly laid out office can destroy your chances of a successful pairing experience before you have even begun. If your desks are arranged like those on the left above you are not going to have a fun time. Not only this you could even be breaking regulations on providing a safe working environment for office workers. The pairing station on the right shows the set up at transFICC.com

Get rid of those L shaped desks and dividers if you have them Straight desks with no obstructions underneath provide a much more comfortable place to sit. Aligning PC ‘s lengthways at the back of the desk can free up leg room and don’t have under-desk pedestals – put them at the end of desk rows if you must have them.

This is the single most important piece of advice in this article if you do nothing else – do this.

Provide breakout areas

Lots of drawing on walls Gavin Tapp CC BY 2.0 via flickr.com

“lots of drawing on walls” Gavin Tapp CC BY 2.0  via http://www.flickr.com

Its not all typing. Its really important not to spend all the time coding at the workstation, we often need to go and sketch out ideas and discuss them away from the screens. Pairs should be encouraged to and given somewhere to breakout and have design discussions. Seating areas with tables and whiteboards everywhere should be the norm for any agile team. Painting the walls with whiteboard paint is really effective.

Provide standardised machine configuration

My laptop stickers came Doug Belshaw CC BY 2.0 via fickr.com

“My laptop stickers came” Doug Belshaw CC BY 2.0 via fickr.com

How someone has their machine set up can be a very personal thing, keyboard short-cuts, themes etc. This can make it very difficult to enable a sense of shared ownership and collaboration. So don’t just pair by bringing along one of the developers laptop’s. Get  pairing stations set up with two keyboards, two mice and two screens. Configure these in a standard way ideally with automation such as puppet or chef.  This enables you to clean install the station every night and will ensure that any developer can work on any workstation and will immediately be in a familiar environment. It also encourages the frequent small check-ins approach used in XP & Continuous Delivery.  You should be checking in your IDE configuration with your source code to ensure “Everything is in Source Control”.

I recommend the team agrees on a single IDE (or text editor if you must, if you are still having vim/emacs wars maybe this is an opportunity to move on a decade) for pairing.

Tip have a USB hub on the desk with spare slots – this allows people to plug in personal mice/keyboards if they have particular styles they like or want to for hygiene reasons, it also provides somewhere to charge your phone 🙂

This is an opportunity by the way to get a much higher specification machine for development – you need only enough machines for one between two.

Hot desking

Pairing station

“Pairing Station” by Andrew CC BY SA 2.0 via http://www.flickr.com

Pairing with the same person for day after day not only gets monotonous it also misses out on the knowledge sharing and team building that pairing brings. You need to be rotating pairs at least daily, for this reason I don’t recommend trial pairing with just a single pair you are unlikely to see any real benefits. If you are doing this, hot desking is really a pre-requisite as people are, by definition, moving around every day. But you can think about a couple of things to make this run better.

Make sure every desk is equipped with a fully set up pairing workstation as mentioned above with dual keyboards and screens.

Have a bank of desks dedicated to each team but encourage full hot desking within that area.

Many people want to keep using the same chair so just get some labels and stick them on its a small price to pay for comfort.

Don’t forget to have white boards and the team board near by.

Clean desk policy

office-932648_640

 CC 0 Public Domain via Pixabay.com

Clean desk policy can be controversial but it ensures that desks are truly hot.

Have someone responsible every evening (Cleaner or office admin perhaps) for clearing all the papers and books into a document tray

Also have them replenish every desk with a supply of post-it notes, sharpies, pens, note paper and whiteboard markers. When we started doing this almost no-one complained any more about the clean desk policy, it seems people value having tools to hand over an untidy desk.

Final thoughts

Although this article is written with pairing in mind, even if you are not pairing regularly many of these tips still apply and will in themselves encourage greater team collaboration. In particular sorting out the furniture, providing breakout areas standard machine setups with dual keyboards and screens will allow ad hoc pairing to occur on an informal basis. The “Could you just help me with this a minute” situation.

So before you embark on your pairing trial I would strongly recommend these measures, I consider the furniture and standardised workstations pretty much essential.

  • Sort the furniture out
  • Provide Breakout areas
  • Provide standardised machine configuration
  • Hot desk
  • Clean desk

 

Further reading

Pair Programming – The most Extreme XP Practice? – Dave Farley

Why pair programming is as much about business continuity as it is about code quality – Dave Hounslow

 

 

 

 

Tagged ,

How Branching Affects Team Culture – Talk at Pipeline Conf 2016

At the Pipeline Continuous Delivery Conference 2016  I was given the opportunity to talk about our experiences introducing continuous delivery at http://www.your-loop.com

“Great story of agile & XP”

“Great arguments, nicely supported with actual examples from practice”

“Great examples and very engaging”

Covering themes I explore elsewhere on this blog including Pairing and Automated Testing,  This was my first conference talk and I am delighted to have received overwhelmingly positive (91%) feed back, thanks everyone.

And thanks to all at the pipeline team for organising a great event!

Slides from the talk are available here.

Dave Hounslow – You Are What You Eat – How Branching Affects Team Culture – PIPELINE Conference 2016 from Software Engineering Practice on Vimeo.

 

Tagged ,

Why I’m fed up with people claiming TDD is Snakeoil

Clark_Stanley's_Snake_Oil_Liniment

I just read Itamar Syn-Hershko’s  response to Uncle Bob’s Giving Up on TDD?  (itself  a response to Ian Sommerville’s blog). In his article Itamar proposes an approach to software development that he calls “Spaghetti Driven Development”. To be honest the opening remarks in his post were so maddening that I had to read the rest of the article just to be sure he wasn’t trolling (I’m fairly convinced he wasn’t but if he was, well done, you got me).

I’m not going to defend TDD itself here, as this has been done admirably by many before (eg. http://www.davefarley.net/?p=203) . No, my beef here is the seemingly never ending accusations of “Preaching” TDD from people who don’t know how to use the technique or even bother to try to learn it. I think quite the opposite is in fact the case and it is often they not the TDDist’s who are the preachers.

On to the post that made me write this rant in the first place…

In his article Itamar states upfront :- “I never really did TDD and probably never will.” – Well, at least we know his position on the topic is one held from a self-declared position of ignorance in it.

“I never really did TDD and probably never will.” – Itamar Syn-Hershko

He then goes on to explain an alternative approach to software development giving it the name “Spaghetti Driven Development”.  His approach as far as I can tell is in essence;

  • Write terrible spaghetti code
  • Refactor into something more understandable
  • Then and only then write tests

What the author is perhaps unaware of is that many of us, who are proponents for TDD, have been on a journey through software development and on that journey we have used approaches like this (among others) for many years. But, unlike the author, we found ourselves dissatisfied with the results. Ironically then, it is through our extensive experience of this type of approach, rather than ignorance of it, and through a continued desire for lifelong learning that we have chosen to look at alternatives such as TDD.

From a personal viewpoint I would summarise my ongoing journey in my understanding of software development as follows. Its worth saying that in all these approaches I have been privileged to have worked with many excellent developers and we delivered commercially successful products which I believe were to some of the highest standards achievable with the tools and techniques of the time.

Early 80’s

Write some spaghetti code and refactor it until it was tidy enough I could understand it. No automated tests but who did back then? This was when I was still at school and just beginning to learn how to code.

1980s-1990s

First professional job as an Analyst/Programmer. Small up front design, code and refactor manual testing – some of us were still arguing if “goto” was a good thing back then and COBOL wasn’t just legacy.

1990s-2000s

Big upfront design, starting to use some automated testing but only after development and often by a separate test team (as an aside; it seems this is about the time that somehow we moved away from the analyst/programmer who did everything from requirements capture, through deployment and support to user training and into more segregated roles of DBA/SysAdmin/BA – I’m glad to see the DevOps movement finally starting to bring this back again.  The use of patterns and OO started around here for me, along with a brief foray into Functional Programming in ML.

Early 2000s

Small upfront design, code and refactor, lots of automated testing after the code was written – this is the period where what I practised was closest to the approach Itamar seems to be championing.

Mid 2000s

Present day almost exclusively ATDD/BDD and TDD in line with what’s generally recommended for Continuous Delivery.
Even today I find that too often we have to refactor spaghetti code, almost invariably when encountering a code base which wasn’t test driven. Fortunately for those of us who have seriously undertaken to learn and apply TDD, our understanding of what good and testable looks like has been massively advanced by having done so, but it still hurts.

“Personally, right now, I would not consider hiring anyone into any senior development role unless they had considerable experience with TDD.” – @thinkfoo

The very nature of software development makes comparative studies difficult but we are increasingly seeing actual evidence that TDD and other Lean/Agile approaches to development such as continuous delivery are demonstrably better than the current known alternatives (see Puppet Labs 2015 State of Devops Report for example).

Personally, right now, I would not consider hiring anyone into any senior development role unless they had considerable experience with TDD. Whatever your position on it, TDD has been around long enough and is widely enough practised that, any senior developer, should, in my opinion, have taken the time to learn and properly understand the technique, even if they then choose not to apply it on a daily basis.
Now I don’t necessarily believe  TDD is the end of the journey and perhaps it will be superseded. Indeed if someone has extensively practised TDD and subsequently has ideas and arguments for a better alternative; Fantastic, lets hear about it!  If what you have is genuinely new and valuable, all us “TDD Preachers” would, I am certain, be very interested to listen and learn. But to convince us, you first will need to have used TDD full-time, for a prolonged period, with other skilled practitioners. But unless you have that authority and vision, please don’t risk peddling your own snake oil, I believe it is only serving to confuse the issue.

Tagged ,
%d bloggers like this: