Agile estimation: separating effort from risk

By The App Business under Agile 01 June 2016

At The App Business, we put the 12 Principles of Agile at the heart of everything we do. Not only do we believe Agile helps us develop the best software, but it allows us to adapt and positively respond to change with openness and transparency.

While Agile development gives you the tools to respond better to changing circumstances, it also importantly helps to give you an understanding of the building blocks of the systems you are creating.

That isn’t to say Agile is without challenges, and it’s certainly not a magic bullet - in fact, some areas are as challenging and difficult in Agile as they are in more traditional, Waterfall-based methodologies. One of the biggest of these is work estimation.

Estimation issues

Estimating work to be done is no easy feat - in Agile, or in Waterfall methodologies. Despite the unscientific nature of work estimates in both approaches, they are, however, a necessity. Stakeholders need an indication of effort and time from the development team for budgetary and planning reasons - this is unavoidable.

Due to the focused nature of most Agile methodologies, estimating how much work an entire feature list - or list of user stories - can be very difficult and, sometimes, inaccurate. This is especially true when you have a new team with an unknown velocity - or in plain terms, when you don’t yet know how fast or efficiently the team can work because they are untested.

It is worth noting again that both Agile and Waterfall face the same problems when it comes to estimating work. For us, though, the big difference is that Waterfall approaches create an illusion of control and a false sense of confidence. The reason for this illusion is that Waterfall methodologies usually expect you to estimate all the work to be done ahead of starting any actual build work.

This means that you are estimating based on some very big assumptions, and even with an experienced team, this is not very likely to work. Additionally, in Waterfall, the system is designed at a very high level, up front - this leaves little room for unknown unknowns or the need to respond to change. Not only does this make estimating inaccurate, but it also means it’s very easy to miss the nuance and detail in the way the system interacts - and only discover the problem a long way down the line.

While estimating with Agile is just as challenging, in comparison, it gives you far better ways to incorporate and respond to change - addressing and identifying risks early through a short, continuous loop of inspecting, testing and adapting.

The key, then, in Agile is to make these estimates as realistic as possible, and to find a way to make them a true indicator of effort - even if they can’t give you exact numbers.

This is where Agile estimation comes in.

Agile estimation

There are many Agile estimation techniques that use all kinds of relative measurements of size and effort, such as T-shirt sizes, story points, work days, etc. The problem is that these techniques don’t tend to separate effort from technical risk. What that means in real terms is teams over-inflate estimates, giving inaccurate impressions of how much work a certain story or epic really is.

For the purposes of this blog, we will use T-shirt sizing as an example.

T-shirt sizing means the team is asked to give each story on the backlog a size ranging from XS to XL. For example:

Typically - and quite understandably - when you ask a development team to estimate the size of a task where they are less knowledgeable on the technical specifics, they are more likely to inflate their estimates to the larger sizes.

This means that unless the team has a high level of confidence in exactly how to achieve a task, the estimates will be higher than actually needed to complete the task.

You may think that is ok, as it makes up for the time needed to figure out how to accomplish the task. However, it also introduces its own problems.

Separating effort from risk

The main issue revolves around prioritisation. If estimates are inflated, is it because the user story has many unknowns? Or, is it actually relatively straightforward, but just requires a lot of effort? Ultimately, this ambiguity can lead to even more unpredictability in the development timelines.

The best way to resolve this issue and introduce more order to Agile estimates is to separate effort from risk. Think of technical risk as the potential impact of technical ‘unknowns’. Having a separate measure for each can give you clarity on where the issues lie with a story - effort or risk - and help inform your backlog prioritisation.

In order to quantify risk, you ask the development team to ignore any technical uncertainty and assume they know all the little details that go into implementing the story. If the technical uncertainty is so great that they can not know what goes into achieving a task/story at all, then the story needs to be revisited to clarify what needs to be done in more detail.

Most of the time, however, the uncertainty is around technology-specific implementations and data sources, or around unfamiliar interfaces. By separating technical risk from effort, the development team can give you an honest view of how much effort they think a task will be without trying to compensate for the details they currently do not know.

After the effort is sized, you then size up the amount of technical risk. These should also take the form of T-shirt sizes with the following options: S,M,L.

Once you have done this for all the user stories you are trying to estimate for, you will have a high-level view of relative effort vs. relative technical risk for each story, as seen by the development team. This will serve two important purposes.

First, the risk sizes will be turned into a multiplier that will be applied to the lower and upper range of effort values (such as 1-3 for ‘S’).

This is where it is really important to know your team. For a team that is relatively new at the problem domain you are tackling, or a team that you know has a long ramp up time with new techniques, you can set higher risk multipliers (S = 1.2, M = 1.5, L = 2). Similarly, with a team you know has solved similar problems before and are good at learning new techniques, you can set lower risk multipliers ( S= 1, M= 1.2, L =1.5).

Now you need the effort required to achieve a task or story, and adjust it for the risk.

To calculate this, you simply multiply the work day values for effort required (from the effort T-shirt sizes) with the risk multiplier values (from the risk T-shirt sizes).

During elaboration 

The second purpose for decoupling effort and risk is that now you have a good indication of which stories have the most technical risk, and need to be investigated/de-risked first. Armed with this extra information, you can now more effectively prioritise your backlog.

This is especially valuable in investigation/elaboration phases when people need to understand the system they are building - including what needs to be investigated further.

Once any extra investigations or technical spikes are undertaken, the technical risk multiplier can be adjusted, or even entirely eliminated, to reflect this fact. The multiplier gave the team enough of an effort ‘buffer’ to properly investigate the approach to the user story, resulting in a much more accurate estimate.

Importantly, that story should then be re-estimated on effort to reflect the learnings from the technical investigation. Now, both the development team and stakeholders have a better idea of technical risk vs. development effort.

Beginning of construction:

As the team continues to build the product and iterate over multiple sprints, a better understanding of the team’s velocity within this problem domain emerges. This should be mapped against the original estimates and risk multipliers in order to evaluate how accurate they were. Having this data means you can better calibrate the risk multiplier for that team when taking on a similar problem in the future.

It is important to reiterate that this is absolutely not a magic bullet, nor a solution to all of your Agile estimation problems. However, this method can bring a little bit more clarity to the estimation process. Ultimately, it should help you to prioritise and de-risk the biggest technical unknowns up front, and continuously improve your estimates throughout the project lifecycle.

---

Do you find estimations to be a pain, or just plain inaccurate? Get in touch to let us know if this method has helped you and your team.