Variation in software estimation

Last week I visited Warsaw, Poland for the first edition of Agile By Example.

The organization did a great job setting up this conference.  For instance, they opened a call for solutions on their website, where people could list the topics they are interested in.  This helped to create a balanced program with blocks of two sessions on one topic.  After each block there was a 15 minute slot available for the audience to asks questions to the speakers, which sometimes led to interesting discussions on stage.

Last but not least, they found quite an interesting venue. The Centralny Basen Artystyczny used to be a swimming pool and has been renovated into an event venue.

The audience was actually sitting inside the swimming pool!

I was invited to do a talk on Agile Estimating & Planning.

The session was based on these 4 questions:

  • Why do we need estimating & planning?
  • How do we estimate & plan?
  • Why do Agile projects need a different style of estimating & planning?
  • How do we estimate & plan in Agile projects?

Part of the second question ‘How do we estimate and plan?’, I explained the differences between guessing, counting and measuring by asking the audience to estimate the number of people in the room using the techniques guessing and counting.

I collected the results afterwards and visualized them in this graph.

What are my conclusions from this data?

  1. There’s a lot of variation between people’s estimates.
    While it seems not very hard to estimate the number of people inside the room, knowing that this was not a very big conference, estimates ranged from 400 to 50.
  2. Guessing has a higher variation than counting.
    You can see on the graph that the range of estimates was much narrower when we used counting techniques.
  3. Most of us over-estimated.
    This doesn’t compare to estimating the duration of tasks in software development, where research has shown that we typically under-estimate 20% to 30%.
  4. Measuring gives us a pretty accurate estimate.
    By measuring the number of forms that I collected afterwards, I had a pretty good estimate on the number of people present.  Although I still couldn’t be 100% sure it was accurate since some people might not have participated.

When I think of the reasons why there’s so much variation in these estimates, I realize that I might not have been clear about the scope of what we were estimating.

Estimating the number of people present can be understood in different ways:

  •  Estimate the number of people sitting in the swimming pool
  • Estimate the number of people sitting in the swimming pool & the ones sitting on the sides
  • Estimate the number of people sitting in the swimming pool & the ones sitting on the sides & the people at the registration desk in the back.

This confirms that when you’re estimating, the scope should be clearly understood by all.

We often have these issues in teams when we’re estimating software features & tasks.  But for me,the underlying message stays the same.

Count or measure if you can, only use guessing as your last option since it is often the least accurate.

About Nick Oostvogels

Hi, I'm an independent management consultant. My biggest strengths are located in the fields of teamwork, motivation, leadership and continuous improvement. In the IT industry you find a lot of these values in the agile movement, in which I often act as a project leader, product owner or coach. My interests go a lot further, into other industries where we find these values in lean production. Besides that, I try to broaden my horizon as much as possible, always looking for better ways of doing business.

One comment

  1. Pingback: AgileByExample Feedback Summary – AgileByExample

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: