Agile Estimation: Is estimation about "risk" or "size"?

Note: this was originally written for a single Wikimedia team and a small audience, with a lot of sourcing from all over the internet, so it's likely lifted entire phrases or sentences without attribution. If you notice anything that should be cited with a known source, please feel free to do so. It is posted here because it has often been shared privately and the info is useful to have in a public space.

2017-08-22

Question came up in task refinement (paraphrasing): Is estimation about "risk" or about "size"?

TL;DR: Estimation isn't always simple or easy, and I encourage you to reflect on how it is useful for you. The answer to "is it about size or risk?" also depends on context, including who is involved in asking that question. The answer may, in fact, be "both." The factors may include "complexity" or "size" or "unknowns" or something else. This issue probably resurfaces the ongoing issue of too many responsibilities for this team, but as that has been challenging to engage with, engaging with estimation in and of itself is valuable.

Generally, I recommend not using "time to do something" as a way to estimate because we all view time-to-do-something differently. "Size" and "risk" and "effort" are all appropriate, as long as the right conversations are being had. Often, using multiple factors/descriptors is useful (or necessary) to get beyond barriers, such as language (my "big" doesn't mean the same as your "big," both in terms of how big "big" is, and whether "big" means "large" or "weird" or "full of X" or something else). Finally, consider all the angles of how estimation is useful (such as for prioritization).

MEAT:

Question came up in task refinement (paraphrasing): Is estimation about "risk" or about "size"?

Answer: it depends. This isn't meant to be a cop-out of an answer. It really does depend on what the team finds useful, and how you think about things as you work toward making estimation useful to you. Additionally, I realize that in the meeting my answer was immediately, "It's about risk," and this email is as much about clarifying that as it is about acknowledging that it's the way that I, myself, best process the concepts around estimation (primarily because framing estimation with "risk" has been a successful approach to coaching several teams in my short career). Upon reflection, I could have instead posed the question to the team and worked with you as a group to determine what you think is the most effective framing for you. I was more focused on pushing through the meeting and "having the answer" than I was on what I feel is a good coaching approach.

With all that in mind, here's how I typically understand estimation to be useful to teams that leverage it:

• It helps the team surface unknown or un-talked-about aspects of a task, usually through Planning Poker (hatjitsu etc) that requires team members to say what they think of a task without their perspective being primed by someone else's.
• It helps the team predict, over time (especially long-term), how long a batch of tasks might take ("velocity", which is team-specific). This is more relevant to a traditional Scrum team, but still useful to think about (especially as we discuss "time" as a factor).
• The results of estimation can signal to a team whether or not a task is ready, or can/should be broken down into smaller and more manageable parts. This could be determined, perhaps, because a team of people all agree that it is missing something, or it could be determined because different perspectives arrive at different needs to complete the task.
• Story Points can help a team describe something in a unified language that avoids personal experience (X points rather than Y time-to-complete).

Now, before you go and think, "Well, shit, I've been estimating wrong," ask yourself, "What do I find useful in estimating?" This implicitly questions a team already estimating, "Is the way I do estimation useful to me (or my team)?" For instance, I've encountered a wide variety of estimation styles (Fibonnacci, T-shirt, Zoo Animals, etc) that all do the same basic thing but some folks have strong preference for one over another because of what makes sense to them (language, concept, whatever). Similarly, if you are able to get work done more effectively by thinking about, say, "size" instead of "risk," or "time" instead of "complexity," go for it. The point is to make sure you're getting something out of the activity to assist with that which would otherwise be challenging (for any reason, including diverse perspectives on the team, maturity of the team and it's process, maturity of individuals, communication impediments, distribution challenges, whatever).

Having observed this team for some time, I would argue that estimation has been most useful for this team for determining whether or not a task is ready to work on. That is, a [this team] task often gets a broad spectrum of estimates, and it's because the task is not fleshed out and/or the task is subject to the huge and ancient codebase in addition to the broad and diverse responsibilities of the team and its members, to say nothing of volunteers and the opinion of the masses. Simply put, because of complicating factors that are the reality of this group of people, each task is unique and hard to estimate because they don't exist in a vacuum but rather in context of those complicating factors. Estimating helps make the conversation happen. It's also a reason it's painful at times, because ideally tasks are ready-to-work-on way more often than they are not.

Now, having said all that, my recommendation is to avoid "time" as the motivating factor for estimation. That is, don't think of things in terms of "this takes 20 minutes, that takes 3 hours." There is another form of estimating, Task Hours (vs Story Points), that tackles that. Would that model be useful to this team? Maybe, but I think it's more complicated to introduce than could be gained from it. Digression aside, I similarly find "time" to be more trouble than it's worth.

The reason I caution against using "time" as a motivating factor is because the time it takes one person to do something is not the time it takes another person to do something. The classic example is that the distance ("size") between 2 buildings is traversed by Person A in 5 minutes, but Person B in 10 minutes. Time becomes less useful here. Are there instances where time is useful for estimation? Probably, and I imagine it's on tasks that are consistent, frequent, and not variable based on who is doing them. I don't think those are common, so while "time" is a useful factor, it's not enough on its own in most circumstances. And even if you find a task like that, there are probably better descriptors for it by the time you identify that it is consistent, frequent, variable, etc.

"Complexity" is another factor often considered. Using the classic example, it's probably not more complex to walk between buildings a kilometer apart and buildings 2 kilometers apart (and certainly not "twice" as complex). You might encounter more random issues along the way, simply because statistically if you increase time you also increase the chances of randomness, but in all likelihood (estimation!) it's gonna be the same experience, just one taking twice as long as another. On the other hand, maybe those random experiences are huge issues if encountered, so it's worth weighing their impact against their likelihood. This also raises the value of relative estimation (one task vs another). Also, note that "complexity" may also be variable based on who is working on it (skills, which can be upgraded, etc). If we think of distance as "size" then chalk another win for #teamsize.

Now, maybe there is a building that is as far away as a previously measured distance, but this time there are obvious hazards along the way (let's say a rope bridge over a gorge filled with crocodiles). The distance ("size") is the same, but the complexity is obviously very different. If we were only measuring distance ("size") then we would not capture the whole picture. "Time" is also likely suddenly unknown. The risks are affecting our ability to complete the task of getting from Point A to Point B.

I've heard that you can frame it all as rolled up in "effort." I like that, so long as it doesn't always transliterate to "time to do something" because there is that trap of "time" as the only motivating factor (in addition to "your time" vs "my time"). Sometimes, it does literally mean "this thing is this hard, so it will take this time," or "this this is this big, and very clear, so it will take this time." Sometimes, it's not that simple. And again, "effort," like "time," could be interpreted individual to individual.

OK, so even if that makes sense (and stop me if it doesn't), why estimate with story points at all? Going back to our original question ("Why is this useful?"), I can surmise that it allows the team to have certain conversations that are impeded otherwise, or maybe aren't hard so much as difficult to identify without a motivating tool. And even if the team has matured to the point where the conversations are obvious and easy, there is also the process itself to consider (which serves a diverse group rather than an individual). In the case of the this team, estimation has been useful for

• signaling when work is ready (or at least has been discussed by a few devs)
• signaling if work is worth taking on in the context of other work
• prioritization
• onboarding outsiders, especially new team members
• contributing to evaluating work volume and focus

Estimation is not an exact science (inherently), so treating it as such is just going to end in frustration. It's a tool, not a rule. When using it to leverage knowledge about a task and work in general, think about how it can be useful to actually getting work done, both short and long term. In doing so, try to approach from different angles. It doesn't matter so much if your definition of "risk" is different from another person's, as long as you're all discussing every angle you can collectively identify. When that becomes too challenging to proceed, consider fleshing out the task more. If that isn't the catalyst to moving forward, try comparing the task to previously and recently completed tasks, to get a sense of how the task in question relates. And if it seems like estimating for the task's sake is moot, consider the usefulness of estimation for others, including this author's needs to get real meta sometimes. :)