Internal Motivations in Software Estimation

by Krishna on August 5, 2007

Most of us are quite familiar, through software engineering literature, stories or direct experience, about the pressures faced by software teams from marketing and management to change estimates to meet some external deadlines. Since software developers are, by nature, less equipped to handle negotiations, they frequently find their expert opinions about size, effort and cost estimates overruled. The end result is an unrealistic schedule which is bad for everyone.

But software developers also create pressure for themselves when they give in to certain internal motivations:

  1. The need to look intelligent: No developer wants to look dumb and hence software developers seldom over-estimate a project. Here, I want to differentiate between what you (as a technical manager) think is a correct estimate and what the estimating person thinks is an estimate. For example, you may think a particular screen would take only 12 hours and the developer tells you that it would take 24 hours, clearly a 100% difference. But the developer is actually giving you true information about the minimum time that he or she will take.

    The more senior a developer is, the more ambitious and optimistic they get. Without the benefit of metrics, they have a tendency to under-estimate the effort they have put in the past and this affects their current estimates. This means that they will take pride in stating that they can complete the work in an unrealistic duration.

  2. The need to do a particular project: Sometimes a particular piece of work is so exciting and challenging that every developer wants to work in it. They want management (or the customer) to approve the project budget and start working. If the developers are aware of budgetary constraints, this may influence them when preparing estimates — they will try to adjust estimates instead of negotiating features and deadlines.The other problem in this situation is underestimating capability. If the project is a new and highly popular technology or framework, developers may not think that they may be unable to handle the needs of the new project. When the choice for working on the prestigious project is available for only selected people, the ones who project confidence in their skills (regardless of their capability) will be selected.

    You can see the opposite effect when a new project involves legacy technology or maintenance work. You will see developers exhibiting their lack of confidence in tackling the technical needs of the project. So generally speaking, the more confidence a developer exhibits, the lower the estimate because the perceived risks are lower.

  3. The need to do any project: Suppose the development team does estimate work properly and provide that to the internal management or the customer. And suppose the management looks at the estimate and decides not to do the project because the overall cost is too high regardless of the schedule. And let us say that this happens a few times in succession.This really puts the development team in a tough spot. Job security becomes an issue, but even ignoring that, the team feels that they are letting down the stakeholders. After this has happened a few times, the development team tries to come with estimates that look good to the decision makers so that they can agree to do the project. Obviously, if done consciously and without the knowledge of management, this is highly unethical.

Using formal estimation techniques can reduce some of these problems. The Wideband Delphi method uses different developers to estimate anonymously and then converge onto a consensus estimate. Using historical data of the organization can be a sanity check.

However, managers have a key role to play here. They must understand the motivations of the estimating persons. How much or how little do they want to do the project, and is that reflected in the estimates? How much do they feel the need to prove themselves when providing an estimate?

One solution that seems to be present itself is to have an “estimation expert” who will not be part of the project. But this is no silver bullet, either. The expert has no incentive to be right in his or her estimates. If the estimate turns out to be an under-estimate, this person is not part of the development team which has to take up the burden. Also, the development team may feel no need to honor an estimate which they think is wrongly done.

The biggest danger in these situations is to miss what is happening, because as a manager, you are happy that the estimates fit within your plan. If it seems too good to be true, it is usually too good to be true. So be careful.

Comments on this entry are closed.

Previous post:

Next post: