Don't use story points for prioritisation

Anyone who has worked in an agile team is fully aware of some of the problems with story estimation. While estimations can be useful internally, story points and person-hours estimates are often misused to assess team performance or output from the outside.

“Why did you only complete 33 story points this sprint when you completed 52 in the last sprint?” is a valid question for a team retrospective, but no so great if it’s coming as a challenge from an external stakeholder. This is a sure sign that a team is being assessed purely on output rather than outcomes.

But perhaps my biggest gripe is using estimates as a proxy for cost during backlog prioritisation.

At first this seems reasonable. Take this example: you have two features in the backlog - feature A and feature B. Both are worth an equivalent expected value to the business. The team has estimated feature A at 13 story points and feature B at 5 story points. Any team would pick feature B to work on first, right?

OK, here’s a spanner in the works: feature B will put an extra 30% workload on your already over-worked customer support team. But feature A is a behind-the-scenes improvement that requires no extra support effort. Unfortunately it also requires an approximate 5-10% increase in infrastructure costs.

Now which do you choose?

If you’ve put your business person’s top hat on, the correct answer is: not enough information. You’ll need a better understanding of operational costs and impact to get there.

In the real world, a team also has to collectively decide between features and ongoing internal improvements like paying down tech debt. If a team never gets to work on internal improvements, that’s also a sure sign that informal or formal team success is defined by fancy-named but vacuous vanity output metrics like feature throughput.

A story estimate is only a small part of the true cost of a feature. There are really three main factors:

1. The cost of actually building the feature while in development
2. The cost of actively keeping the feature in production - e.g. computing resources, bug fixes, customer support, etc.
3. The opportunity cost of delaying release due to time spent waiting in the product backlog, in idle states during development and while user awareness builds.

A product team typically estimates some approximation of 1) in order to work out how many stories they can deliver for a sprint. But the other two are rarely considered in detail as part of prioritisation. It’s worth thinking through a few examples to understand how much hidden cost is in 2) and 3).

Of course, in order to prioritise effectively, it’s also important to estimate the upside of having the feature in the product. Like any business decision, both the value and cost of an activity should be at least estimated before proceeding. It’s amazing how little this happens in product development, though.

Unfortunately, determining the value upside of work is often seen as the sole purvue of the product manager, while it is the developers’ responsibility to determine the cost (where ‘cost’ is defined as cost of development only).

In fact, combining the two estimates are vital for creating a prioritisation approach that the whole team can get behind and discuss on equal terms.

This is why tech debt ‘stories’ are impossible to prioritise ahead of low-value features. They just aren’t being compared on equal terms.

Next time we’ll look at how cost of delay can be used to highlight the true value of paying down tech debt. Yes, even that long-neglected spaghetti of a CSS code base you’ve been meaning to improve for months or years.

In the meantime, reply and tell me about a feature you have worked on that had some unexpected costs once it was put in the hands of users.

All the best,

– Jim