Communicating Software Estimates

by Krishna on December 2, 2007

In my last post, I had written about the conflict over software estimates between the people running the business and the people writing the software. I mentioned that most projects are likely to be underestimated in the first place and further pressure from the business side only makes things worse.

How can engineers get business people to understand this? Let us first look at some of the dynamics between the two sides:

  1. Business people generally have more power in the company than purely technical people. They have the power to hire, fire, outsource, buy, etc. This power equation is a conscious or sub-conscious aspect of every communication.
  2. Business people are used to trying to get the best deal. They would like to get the most features in the product for the least cost at the earliest. Before an estimate is provided, they may not actually need all those features, but after that, they believe that they are entitled to what they were promised.
  3. Business people seldom have any idea of the complexity of changes. This sounds really obvious, but software developers don’t understand this. They are confused at the irrationality of changes requested when they are already behind on the schedule.

A new developer eager to please business people starts off by giving or agreeing to relatively aggressive estimates. After the project blows through deadlines on its own or with the help of a few “change requests”, the developer becomes more circumspect. However, the answer here is not over-estimation (charitably called “adding a buffer“). Contrary to what many people think, over-estimating a task is not that easy in practice. Here is why:

  1. Over-estimation cannot be so excessive that it belies common sense. For example, normally you cannot say that you need 30 days to change the layout of an existing report.
  2. Over-estimation should not arouse suspicions of deliberate over-estimation. Otherwise, the other party will haggle and negotiate down the estimate, sometimes even below what is acceptable.
  3. If the over-estimation is accepted, one has to show signs of being busy. Otherwise, the next estimate negotiation could bring up that issue.
  4. And most importantly, over-estimation is over-estimating what you know today. It does not account for misunderstanding requirements or change requests.

So what does an engineer do? The right answer, in my opinion, is to start with the statement, “I want to help you, but I don’t know how much it will take without learning more. Can you help me understand the problem better?”

This statement does two things. First, it puts you on the same side as the business people and both looking at the problem. Being on the same side is important to put thoughts of self-benefit, negotiation and compromise away. Secondly, it allocates responsibility to the business people to help you get to the right answer. It establishes a quid pro quo: You need an estimate; you provide the knowledge for the estimate.

The next step should be to arrive at an understanding of how long it will take to understand the project enough to provide an estimate. This could be a single meeting or it could be a few weeks, depending both on the complexity of the project and the current knowledge of the estimator. These sessions will attempt to reduce the mystery around the estimates.

What I am suggesting seems suspiciously like a waterfall model, but that is not my intention. A fixed point estimate in a waterfall model is a loss for both sides because it does not deal with the reality of change management. Any project lasting more than a few weeks will have to undergo change because of changing business needs. A fixed estimate forces the technical team to reject valid business needs or, worse, accept them within the current estimate.

An iterative model that accommodates changing business requirements is a better option. But even in that situation, business people need estimates in one form or another (Sprint backlog, dynamic project plans, etc.) to plan their activities. By collaborating with them to understand before estimating, you both win.

A final question remains: What do you do about the person who says, “I don’t care what you think it takes. I must get this done by date ‘x’. Otherwise the following bad things will happen to the company/you/me.”

After many years in the field, here is my opinion: Even if a negative outcome could affect you, it is not a problem you have created. You do not have to feel obligated to solve it if there is no solution. Even if you put your best effort, it is very unlikely that you will hit the deadline and more likely that you will be blamed in some way. There is nothing in it for you.

I realize that personal circumstances (such as not losing your job just then) may prevent someone from taking such a stand. But absolving yourself of the responsibility and preparing for the worst can relieve you of the mental stress involved in death marches. You are trying your best. It is not going to happen. Let it go…

{ 2 comments }

Kalpesh December 4, 2007 at 12:58 am

Krishna,

I agree with you. I am not sure, why there are 2 sides, when they work for common goal (not against each other, but with help/support to each other)

Sadly, the cultural aspect plays role here. PMs are like bosses & Developers are like resources. Whereas, both are working towards common goal of delivering some value.

There is no semi-scientific way of estimating things. People add buffer to not take the firings from management. Management don’t know, the complexities of software development. S/W development hasn’t reached industrialization stage where input/output can be determined.

Agile shows the problem to some extent. However tools/languages/our knowledge of things needs to be standardized – that might help one come at a predictable delivery.

Krish December 4, 2007 at 10:46 am

Right on, Kalpesh. The biggest problem with software is that we are always working on things that we have never worked before and thus estimates are really a difficult proposition

Comments on this entry are closed.

Previous post:

Next post: