Tag Archives: agile

5 ways to avoid the Technical Debt Trap

Technical debt is toxic debt, much like a payday loan you take on technical debt often in a state of panic when that feature just has to be out there right now. Further just like a payday loan if not paid off quickly it can rapidly become a millstone around your neck where you find yourself forever in a paralysing technical debt trap.

So what can be done to avoid the pain of Technical Debt in the first place?

#1  Acceptance Test Driven Development

Code that has no tests cannot be refactored safely, code that cannot be refactored is code that will rot, the second you write code without a regression test you have taken on some debt.

For me ATDD is even more important than TDD in this respect, once you have worked on a system with close to 100% acceptance test coverage you will wonder how you ever made changes safely without it. I have had to go back to working on systems that lack good acceptance test coverage having worked on some with close to 100% coverage and it is paralysing. More than this it is shocking to see the different attitude of developers to refactoring in such environments, without full regression suites (and if you don’t use ATDD you probably haven’t got a full regression suite) the attitude becomes “If it ain’t broke don’t fix it” rather than constant improvement. By using ATDD and treating the test as the source of truth you enable the freedom to change how code does stuff without changing what it does.

 

#2 Don’t feature branch – work on trunk

Feature branches discourage refactoring. I hadn’t really appreciated this benefit of trunk based development until recently when a colleague of mine rasied it. He had previously always worked with teams who used feature branching, when he moved to our team which was using trunk based development he was blown away by the freedom it gave him.  He enthusiastically declared

“It’s fantastic I can do fundamental refactoring’s knowing I won’t have people breathing down my neck at some point in the future because they have merge hell”.

Sound familiar? With trunk based development the worse case scenario is maybe ½ a days worth of code to merge in.

I find it ironic that feature branching is often championed as allowing developers to work independently on features when the truth is the opposite.
 

#3 Test Driven Development

TDD encourages good design by nudging us in the direction of loose coupling, strong cohesion and clean encapsulation. It doesn’t of course guarantee any of these things by my experience is that coders who have and do practice TDD tend to write cleaner more maintainable code. Code that is well written is easier change our efforts upfront reduce the debt to the future.

 

#4 Refactor all the time

Good citizen’s pick litter up and don’t add to it carelessly so always leave code a little better than you found it. Personally I find a little bit of refactoring is a great way to understand how the code does what it does. When faced with a method or a class that does more than one thing if we start breaking it up into smaller methods teasing apart layers and introducing abstractions its purpose becomes clearer. This has benefits both for us and for future pairs who work on the code.

 

#5 Limit your work in progress

Control your work in progress, working at a sustainable pace is key to maintaining velocity. If you are constantly firefighting technical debt will keep piling up. Techniques such as swarming can help you here.

Finally

Remember that ultimately it usually costs a lot more to pay back than what was saved by doing it well in the first place. Left to fester you end up taking on another piece of debt to cover the debt you already took out, then another and another until the debt becomes crippling. It’s much better to save up and invest in quality upfront than borrow from your future productivity.

Tagged , , , ,

Is this a feature request or a bug?

How does a bug differ from a feature request?

I’m sure like me you have been in many a heated discussion on whether something is a bug or a feature. In my view a great deal of time is wasted in these arguments that could be spent more productively.  A long time ago I concluded that, at best, it wasn’t a valuable distinction, at worst that to distinguish them is in fact damaging.

Consider the following possible definitions of bugs and stories


Feature Request

A proposed change to how your software currently behaves in the hands of users to deliver greater business value.


Bug

A proposed change to how your software currently behaves in the hands of users to deliver greater business value, which you want to blame the development team for doing wrong in the first place.


Lets think about two important points in these statements

  • “Currently behaves In the hands of users”
  • “which you want to blame the development team for”

“Currently behaves In the hands of users”

If no one can see this behaviour its not done yet. So during story development we should simply adjust and fix things as we find them – we want to get to done not half done, if you need to raise and track bugs at this stage you probably have one or more of the following problems or something else that needs to be addressed as the root cause

  • Stories too big – taking too long – think about how you can break down the delivery of value
  • Working in silos, with lack of integration of QA/BA/Developers/Product Owner’s
  • No definition of done – “In the hands of users” is a pretty good place to start in my book.

“Which you want to blame the development team for”

This really seems to me to be the nub of the issue. When we call something a bug there is an implicit message that the development team did a bad job. My experience is that development teams generally set themselves very high standards on quality that this attitude can damage morale and ultimately lead to lower quality and rate of delivery. Calling things bugs can lead to wrong expectations such as;

  • the cost of fixing it will not impact other work – devs did bad they can make up the time
  • its automatically the most important thing to do which it may or may not be

The “No Bugs” messaging that has been around can also lead to prioritisation of bugs over stories when the bug fix may have minor business value compared to other changes in play.

Given these two definitions is it a useful distinction?

It is my contention that distinguishing bugs from stories adds no value. I have had good success with treating bugs and stories the same once the software is in the hands of users. They both have some business value associated with them and they both need to be prioritised based on that value. So having a single backlog makes much more sense to me. Moreover separating these out into separate tracking systems or even labelling them bugs or stories in the same system is a bad idea stop doing it now!

But people like raising bugs I’ll never be able to get rid of <insert favourite bug tracking software here>?

Depending on your company culture you will find it easier either to call everything a feature or everything a bug. I don’t think it actually matters much, although it can be fun to start a new project with a bug along the lines of “Unable to see login page” before a single line of code has been written. Whichever way you go my recommendation is to pick one and treat it as a single backlog.

A word of caution, if you call everything a bug remember ONLY the product owner should control the priority of the backlog that means sysadmins, customer service etc don’t get to set the priority – sure they get a say but the product owner has to balance all the business priorities and make the call.

Tagged ,

Simple alternatives to feature branching – #1 Build alongside

With Continuous Delivery (CD) every commit is a potential release candidate and all pushes are to a single central trunk. For people new to CD it an be very hard to reconcile frequent commits with this concept; How can I commit twice a day when this feature is going to take several days to be in a state that I want it to be public?

A good example we had of this recently was a full reworking the registration pages for the https://www.your-loop.com energy monitoring website.The registration process requires the installation of hardware to do meter readings, ensuring that radio communications are working and serial numbers etc. have to be entered and the flow can vary depending on the type of meter a user has. The rework involved changes of page flow and content. The nature of the changes meant we didn’t see a clear way to break it down into smaller sizes that made sense to the user. It was also clear this is not something that could be completed in a single commit.

Why commit to trunk?

The key things we are trying to ensure by committing frequently to trunk are;

– New automated acceptance tests are developed as we go and run on the new feature in every CI build .

– Existing acceptance tests continue to run on the existing code in every CI build.

– Ease of manual testing so the changes can be viewed (without redeployment to switch for example) .

– Continuous integration and merging into all developers work .

Feature branching – just say no.

Its common for people to revert to what they know when presented with a challenge and some of the team were immediately reverting to suggesting a feature branch. The problem with this of course is that feature branches make all of the above desired objectives difficult, expensive or simply impossible. Every time you feature branch, a release fairy dies,  just say no!

Feature Toggles were not ideal.

One option discussed was to use a feature toggle. The problem with toggles is how to toggle them on and off. Feature toggles often take the form of a singleton property, perhaps in a configuration file or dependency injection that switches between different implementations at build time. The problem with this approach is test isolation, I want all my regression tests to run whether the feature is currently on or off, and this is an awkward thing to do in your CI environment unless you run tests sequentially, even then it requires duplicated toggles in test code that you must keep in synch.

There are other approaches to feature toggling that can happen at runtime which I will talk about in another post but this wouldn’t have been much help here as we needed a global new/old switch.

Build alongside

The approach we used is to “build alongside” by building the new registration pages with a different URL which is not linked to from the site;

So if the existing registration page entry point is;

http://www.example.com/registraion

We build

http://www.example.com/newregistration

Whilst the new feature is being developed all buttons/links etc. point to the existing registration pages entry point. We can write selenium tests that drive the new registration pages and leave all of the existing tests in place. QA and the product owner can access the new pages without any configuration (by simply going direct to the URL), but they are sufficiently hidden from the public to not get seen. In practice we called it a slightly more obscure name and there was really no risk beyond seeing a “work in progress” should a customer have guessed the URL which was unlikely.

Once all are happy with the new pages the new URL is renamed to the existing one and the old code is deleted.

It’s a simple approach that everyone understood and met all of the desired objectives regarding frequent commits, CI, manual testing and frequent merging with everyone’s code.

Tagged , ,

The thinkfoo test for Continuous Delivery – What does good look like?

Dave Farley makes an excellent point in his recent post that many people in our industry have never worked on an efficient project team and wouldn’t know good if they saw it. Indeed many are probably blissfully unaware of just how inefficient they are comforted in the knowledge that they seem to be no worse than anyone else. So how do you know if you are good?

Some fourteen years ago Joel Spolsky wrote an often quoted piece he called the Joel Test.

This was great stuff back in 2000 but we have moved on a long way since then and I propose an updated test that reflects modern development practices founded in the principles of Continuous Delivery (CD).

I’ve worked with some incredibly productive teams (and some not so productive ones) I prefer to work with productive ones and I have over the years formed some strong opinions of exactly what good looks like. I realised I have been informally using a set of criteria to evaluate teams for some time and thought I’d write it down and share it.

So here is my “thinkfoo test”; efficient development teams will be able to answer yes to most if not all of these questions, if your team can’t and you think you have an efficient team with a good process maybe its time to take the red pill and think again?

1. Do your developers push changes to a single trunk branch at least twice a day?

2. Do you practice ATDD and write automated acceptance tests for every feature?

3. Do you practice TDD?

4. Do you run your full set of acceptance tests, unit tests and performance tests on every commit to trunk?

5. Do you have automated single step repeatable builds and deployments?

6. Does your team release to production at least every two weeks?

7. Do you practice pairing as the normal way of working?

8. Do your teams self organise?

9. Is your team in a noisy collaborative environment?

10. Do you have cross functional teams of BA’s, developers, QA and operations responsible for end to end delivery of working software?

11. Do you have a single product backlog (with bug’s and features together)?

12. Do you use minimal stories as specifications rather than detailed up front designs?

13. Do you have regular retrospectives to continually improve your process?

Lets look at each of these in a little more detail

1. Do your developers push changes to a single trunk branch at least twice a day?

  • feature branching is evil, long lived feature branches doubly so. Trunk development is, for good reason, a key requirement for CD.
  • trunk should be in an always releasable state, every commit should be a release candidate.
  • To be continuous your integration must be frequent; committing to mainline twice a day should be considered a minimum.

2. Do you practice ATDD and write automated acceptance tests for every feature?

  • these should be built in advance and alongside the feature and treated as the source of truth or living specification. The feature isn’t done until all the acceptance tests pass. By doing this and keeping work in progress to a minimum the need for bug tracking is diminished.

3. Do you practice TDD?

  • many people are confused about what level to set their tests at that’s why I specifically want to know if you have both ATDD (are you building the right thing) and TDD (are you building it right).
  • The “red, green, refactor” cycle is critical to producing good clean code
  • Your whole team needs to practice TDD, I wince every time a development lead says “We don’t enforce it but sure you can work that way if you want”.

4. Do you run your full set of acceptance tests, unit tests and performance tests on every commit to trunk?

  • #1 is a prerequisite of this if you are not using a single trunk your continuous integration isn’t, end of.
  • performance tests are a critical part of your build pipeline many products fall under the pressure of their own success.

5. Do you have automated single step repeatable builds and deployments?

  • if you can’t do repeatable automated deployments there’s not much point in doing #1 through #4 as you won’t know that what you deployed is what you tested.

6. Does your team release to production at least every two weeks?

  • if you do iteration after iteration without release I don’t really buy that you are agile, this is usually a waterfall project dressed in agile clothing.

7. Do you practice pairing as the normal way of working?

  • see my post on pairing for my views on why this is essential to building quality software assets.

8. Do your teams self organise?

  • teams that do top down allocation of tasks in my experience always end up optimising resource usage over feature delivery, it discourages collaboration, common code ownership and pair rotation don’t do it!

9. Is your team in a noisy collaborative environment?

  • the opposite of Joel here I agree with uncle bobs views of headphone clad developers with their backs to each other, no single developer can be productive enough to jeopardise whole team productivity don’t allow it and don’t hire people who insist on working this way. Great software is built by teams not individuals beware the hero developer!

10. Do you have cross functional teams of BA’s, developers, QA and operations responsible for end to end delivery of working software?

  • and the roles should be blurred rather than designated, BA’s should write some tests, QA’s should write some code, developers should write deployment scripts and talk to customers, ops guys should be coding etc etc.
  • Your developers should be on first line support for your production systems.

11. Do you have a single product backlog (with bug’s and features together)?

  • do the most important thing first and optimise for flow of delivery over resource utilisation (see managing your backlog for more on this)

12. Do you use minimal stories as specifications rather than detailed up front designs?

  • stories are about enabling conversation not detailed specification see https://leanpub.com/50quickideas for some great thoughts on user stories.
  • Joel believes in detailed up front specifications sorry but he is just wrong on this, in any project lasting more than half an hour the specification will change, living specifications in the form of automated acceptance tests are the only way to keep specs current.

13. Do you have regular retrospectives to continually improve your process?

  • nothing is perfect; core to the principles of Continuous Delivery is continuous improvement retrospectives are an essential part of that learning and improving.
Tagged ,

Why pair programing is as much about business continuity as it is about code quality.

I heard a statement recently, it’s a common problem and is probably familiar to many of you.

It goes something like this ;

“We will need to wait until next week when Max is back from holiday to do that work, that module is his baby ”

When I heard this my initial reaction was confusion, why was no one else able to work on this code? It was in the same language, the same repository, was built in CI, had tests it made no sense. Then it dawned on me no one else in the team had EVER worked on this module, they had no clue what it did it how it did it, confusion turned to horror, did anyone know what this guy’s code was like? What if he left for good, got hit by a bus? Was this module going to rot forever or need rewriting?

I realised I’d heard this in a different form some time previously in another organisation from a new CTO who had not had extensive experience in a pairing environment;

“So Paul is leaving, what do we need to do for handover? ”

It left us old hands a bit stumped sure we were losing a very talented developer but there was nothing unique about his knowledge of our code that needed “handing over”.

The common negative response to pair programming from management is “I can’t afford to use two people to do one persons job” . As developers our common reaction to this is to explain how pairing can be faster, result in less bugs etc. This is understandable as these are the benefits most obvious to developer. But I would argue that it is business continuity that could be the killer argument for pairing.

Done correctly pairing addresses business continuity in a number of ways;

Knowledge sharing – many people have actually worked on the code this is far superior to a code review; think of pairing as having co-written the book and code review as being told the story by someone who has seen the film.

Code becomes easier to understand on each pair rotation (pairs should rotate or swap partners at least once a day to reap the benefits of pairing) as this rotation occurs small improvements in code clarity are made as the new person gets their head around what’s been done so far. This makes it a much more valuable and flexible piece of software six months down the line when some new requirement is needed of it.

No ego’s: the code is collectively written and ergo is collectively owned – no need to wait for Max to get back from holidays if it needs changing today it can be worked on today without upsetting anyone.

It’s a truism to say that code takes on the form of the team or teams that write it. If only one developer is working on a system the benefits of breaking down the code into modules are diminished a single person can only work on a single piece of code at a time. This is one reason why it’s common to see software written by a team of single developers be one big blob of software. I have seen pairing address this in this way: as pair’s rotate, realising that a piece of work can be done in parallel if the code is structured in a particular way they do so and your most important feature gets into your customers hands faster.

The software a company builds should be considered a company asset, not some individuals baby. I have seen pair programming to be an extremely powerful way to ensure this is the case. If your organisation is developing software the question you should be asking is not can you afford to have developers pairing but can you afford not to?

dave speaking

 

Watch my talk at the pipeline conference where I discuss introducing pairing and other XP practices to a team.

 

 

 

Tagged , ,

Using story swarming to reduce WIP

Some years ago I was on a training course on lean thinking and one particular exercise has stuck in my mind ever since; Teams of six or so were given a piece of A4 paper and were told to arrange themselves so that none of them were in contact with the floor. The solution most people took was simply to get all of the team to stand on the paper at the same time. The tutor then folded the paper in half and told us to repeat the exercise. It was still possible on one foot for the team to stand on the paper. But the paper was folded again and again until people were resorting to climbing on one another or declaring the task impossible. Finally the tutor took the piece of paper away entirely. Suddenly one team realised that the paper was completely surplus to requirements and the goal could be achieved even with no paper at all by having the whole team jump on the count of three. The point of this exercise was to demonstrate “Muda” or waste and how the presence of a tool or resource tends to make people use it even if in practice it is not needed at all.

One of the tenets of lean production, which as you will know is one of the foundations of the approaches to software development collectively know as agile, is the reduction of waste in all its forms. Inventory that is not being used and work in progress (which is product that is not yet ready) is an example of one of seven “wastes”. In software development code is inventory, if this code is not usefully in production it is my conjecture that this is waste and we need to find ways to minimise it.

One way of addressing this waste will be familiar to any practitioner of agile development; namely the use of an iterative approach, one of the benefits this gives is that we deliver useful features into the production code every iteration hence reducing the wasteful inventory of code that is work in progress. We also strive to prioritise our work so that the most important feature is delivered first and we do not put wasteful effort into less important features and to ensure that we do not “over deliver” excess inventory that is not yet needed..

You’ll hear it stated frequently that having a single product backlog that is ordered as a single queue is essential to a successful agile project. The idea being that work is done in strict priority order. In practice what often then happens is that each pair will grab the top available card in the backlog and begin work on it, the next pair will grab the second and so on. What this means is that in a team with six developers you suddenly have three cards running in parallel typically all placing conflicting demands on your QA’s, BA’s and the customer at the same points in an iteration which then leads to features being held up or bugs being found late in the day. Worse still it could be your top priority card gets held up because of some problem with a lower priority card that conflicts with (either through necessity or some error in the solution).

A consequence of which is that some or all (and not necessarily the lower priority ones) end up being incomplete at the time of cutting a release. Subsequently juggling of features which are ready or perhaps not wanted yet has to occur. Methods to tackle this include “feature branching”, “branch-by-feature” and “feature switches”. These allow us to switch on or off particular features either by clever use of version control systems or with a code switch to “turn off” code that is not ready. You will hear proponents of both approaches. But it seems to me that whichever approach is used the wrong problem is being addressed in that they are all ways of dealing with excess code inventory be it code that is not ready to put into production at the end of an iteration or code that delivers a feature that is not wanted yet.– there are great tools to help with this to be sure but my point is that if this is needed at all;

(a) There is excess inventory either in the form of unfinished code or in the form of features delivered ahead of when they are wanted or needed

(b) A process has been introduced purely to deal with this excess inventory, which in itself is a form of waste.

Swarming

So what else can we do to deal with this excess inventory? Well the thing here is to eliminate the excess in the first place.

An alternative approach to the pair by pair grabbing of stories, and one which is getting some attention, is to have your entire team “swarm” on a story until its done. This basically means that the whole cross functional team (Devs/QA/BA/Customer) works on the top priority item together breaking down the whole feature into small manageable tasks (often each of these takes less than a couple of hours for a pair to complete)

We have been swarming on stories for quite a while now and I would like to share my observations on some of the benefits we have seen;

  1. Our top priority story gets all the attention this means it gets delivered first and sooner than it would if the team was spread out on other stories we have found that even unusually large stories that we would have previously expected to take most of a two week iteration often get completed in a few days.

  2. The team delivers testable functionality early on in an iteration keeping QA’s busy and revealing bugs quickly

  3. The customer is focused on the item most important to them and has a clear view of the progress being made and what the team is working on

  4. More robust to absence whether planned or unplanned as all of your team knows about the feature being developed we have found for example that there is better pair rotation and BA’s and QA’s get included more frequently in those pairs.

  5. The code delivered ends up coherent as the context switching that can occur with pair rotation and multiple stories in play does not occur.

  6. And of course it encourages small frequent commits and CI

  7. We have found that we rarely need to “turn off” features with this approach at the worst case scenario you have a single story in play at the point you want to release and in practice this has not often occurred but where it has a relatively simple approach with feature toggling has sufficed – its only a single feature switched off and it won’t be in production switched off for long after all as its our top priority story for the next release.

In summary I think if you are finding the need to use advanced version control tools to manage feature branching, or regularly using coded feature toggles to manage multiple “in progress” features you are probably suffering from Muda in your process and I would highly recommend that you give swarming a try after all do you really need that piece of paper or can you just jump?

Tagged , , ,
%d bloggers like this: