Code Size in Software Projects

In a recent post, Steve Yegge talked about a software game he had written and how the code size (500,000 lines of code) had become too big for him to manage. He had previously written the application in Java, and now has decided to rewrite the application in Rhino to reduce the code size to around 150,000 lines.

For managers, the topic of code size has many implications for software projects in the areas of software quality, resource allocation and effort. Let us look at these:

1. Code is written to create product functionality. However, writing code requires resources and is expensive. The less code that can be written to accomplish the same functionality, the better in terms of cost. A concise language is better than a verbose one.
2. The more lines written by a developer, the greater the potential for bugs. This is particularly true of boiler-plate code where repetition can introduce typos and other mistakes.
3. Fewer lines of code does not always mean less time for overall development. Sometimes, smaller code can be difficult to read and debug, resulting in greater costs for testing, debugging and maintenance.
4. If software developers can reuse code or functionality in tested libraries, they can save a lot of development time. The best choice is the language framework itself, followed closely by open source libraries which have business-friendly licenses.
5. Software developers are most efficient in their own languages and tools. Although another language may be more concise, they may take more time to effectively use it, or make mistakes, resulting in greater costs.

The ideal goal for a technology choice for a software project would be a language known to all developers on the team, and which has the best reusable libraries and expressive syntax. The reduced code size and development effort translates into tangible benefits (less cost, less time, greater quality).

To be most effective, the decision should be made at the start of the project. The project should be staffed with the best people (available to you) in that technology. You should purchase the best tools that you can afford for working with the specific technology.

Code reviews, when done right, are very effective in identifying copy-and-paste or inefficient code. The people participating in the code reviews can suggest different ideas for making the code better, such as redesigning a class. By sharing their ideas, the developers become increasingly knowledgeable in code reduction techniques and their work automatically improves.

Now, what happens if you never paid attention to the size of your code base for a long period, or worse, you have just been handed a large existing project? Before you proceed, the first thing to understand is that your primary goal is not making the code size smaller or trying to understand the code base. Your foremost objective is to enhance the functionality of the application.

With that in mind, the first question is: How much of the code base do you need to be familiar with to enhance the functionality? If you don't need to make any changes to some portions of the code, you could work as if their source code never existed, and just link to them.

Secondly, do you completely know the existing functionality and dependencies of the code you will be modifying? Usually, the existing code base will have some convoluted code written for bug fixes and change requests. Sometimes, a particular line of code may affect other modules.

In this situation, refactoring code makes it much easier to make changes to the code without affecting functionality. Refactoring may result in larger number of lines, but a significant portion of them can be isolated away and never looked at again.

So, when you inherit a large code base, your objective should be to treat as much of the code base as a black box, never to be tinkered with. This will reduce the code that you need to learn, understand and modify. And most importantly, your goal of enhancing the application functionality will be met.

Still, developers will continue to worry about the huge code size that has now been isolated, usually citing performance, memory needs and maintainability. In several instances, there is no actual evidence that there is a negative impact detrimental to the user and it is an assumption by the developer.

However, if any part of the isolated code base does impact on performance, and is causing maintenance issues, then its status should now be upgraded to the "working source code" and become a candidate for refactoring (or in extreme case, rewriting).

So, what about Yegge? I think he is making a huge mistake by committing vast amounts of his productive time to rewriting hundreds of thousands of lines of code. He could be adding more functionality to the application by spending significantly less time to understand those pieces. He could release the application as open source and start work on creating some other application.

A good developer has a penchant for order and organization. Unfortunately, it can be taken to excess at the cost of useful work. Keeping your house clean is a good thing. It just doesn't make sense to tear it down when you cannot remember where you kept your photo albums.

---

Technorati tags: software development, refactoring, rewriting, yegge

2007 – The Best and Worst Books I Read

I read around 65 non-fiction books in 2007. Here are the lists for the best and worst books I read. Some of them may have been published prior to 2007, but the best books seldom get outdated and the worst never get better over time.

Best books I read in 2007

• The No Asshole Rule, by Robert Sutton: The title that resonates with nearly everyone who has ever had a job. The most timely book of the year.
  
  
• The Halo Effect, by Phil Rosenzweig: The theory described in this book invalidates popular ideas about what makes a company great and profitable. My review here.
  
  
• What Got You Here Won't Get You There, by Marshall Goldsmith: How your behavioral failings at the executive level can hurt you, and how to start fixing them.
  
  
• Peopleware, by Tom DeMarco & Timothy Lister: How to manage knowledge workers: A must for every software project manager.
  
  
• Coping with Difficult People, by Robert M. Bramson: This is a much older book from 1988, but is worth the read as it talks about the different difficult behaviors exhibited by people in the workplace and how to cope with them.

Honorable mention goes to Barack Obama's non-political autobiography "Dreams from My Father" (2004). Poetic!

There are a few books I should have read years ago and finally managed to get around to them this year. They include "Code Complete 2" (Steve McConnell), "The Goal" (Eliyahu M. Goldratt and Jeff Cox), and "Freedom at Midnight" (Dominique Lapierre and Larry Collins). All great reads.

My favorite authors for 2007 are Steve McConnell (Code Complete and Software Estimation) and Robert Sutton (The No Asshole Rule and "Hard Facts, Dangerous Half-Truths & Total Nonsense").

The worst books of 2007 are not the ones that are truly bad in the sense that they are devoid of content or the author writes poorly. The following books are selected because they promise much, and deliver little. A number of these authors are among my favorites, but they disappointed badly. They are:

• The 4-Hour Workweek, by Timothy Ferriss: I am surprised that more people, having read his book, have not found many aspects of his advice disturbing and impractical.
  
  
• The Cluetrain Manifesto, by Christopher Locke, Rick Levine, Doc Searls, and David Weinberger: Corporate bashing at its worst. I looked forward to reading this for a long time, but the rants overwhelmed the content.
  
  
• The Dip, by Seth Godin: Short is good and readable, but the topic deserved a lot more treatment. It failed to explore the nuances, exceptions and pitfalls effectively. A rare misstep by a great author.
  
  
• Wikinomics, by Don Tapscott and Anthony D. Williams: I have never read a more incomprehensible book. The authors use laborious examples and specialized jargon to obfuscate any ideas contained in the book.
  
  
• Go Put Your Strengths to Work, by Marcus Buckingham: What a way to fall! Buckingham's "First, Break all the rules" remains one of my favorites, but this book is a true hack and condescending towards the reader.

---

Technorati tags: reading, books, top 5

Delegation and Follow-up

A good manager must learn to delegate. By doing so, he can focus his time on activities which others cannot do as well as he can. Since a manager's time is costly, he should delegate repetitive activities to less expensive resources, and spend more time on tasks that will contribute to better organizational results and profitability.

Delegation is, simply, asking someone else to do a task. You may have to provide essential information at the outset. You may have to supply additional information during the course of the task to fill in the details or correct misunderstandings. Finally, the task is finished.

Successful delegation means that you have accomplished the task with considerably less effort than what you would have spent by doing it yourself. And by the same measure, delegation is a failure if you achieve negligible or negative time savings, the quality of the work done is poor and you are constantly worried about the progress of the task.

In this post, I want to address the last issue. When you delegate, you want to be confident that the task will be taken up, worked upon and finished in a timely manner. However, the following situations can dent your confidence:

1. When the other person does not acknowledge the task: This is most common in email (or any non-verbal) communication. You ask someone to perform a task and you hear nothing back. You are not sure what happened: Did the person receive your email? Has he started working on it? In a face-to-face setting, silence is usually not an issue, because there is no confusion about the person having received the task. But lack of reply to an email has the opposite effect.
2. When the other person does not provide a plan: Without a plan, there is a greater likelihood that the task could be consumed by other priorities. A plan is not necessarily about exact dates and times for deliverables, because the other person needs time to understand the task. But the plan should contain specifics about time allocation and the activities that will get you to the point where the task is completed.
3. When the other person does not provide details of the progress: If you are not updated regularly about the progress of the task, your confidence in the timeliness and quality of the deliverables decreases. That is why incremental deliverables are very important. It can easily show the amount of work done, what is remaining, the quality of the work and whether everyone is on the right track.
4. When there is no mutual acceptance of task closure: The task is really only completed when it has been done to your satisfaction. Every deliverable remains unaccepted until you absolve the other person of any further responsibility and decide to take it from there.

As a manager, you can improve your chances of successful delegation by

1. Following up with the other person for task acknowledgement, and constant progress updates.
2. Implementing organizational or project processes for planning, monitoring and acceptance.
3. Using software for task planning, scheduling, and reminders.

Unfortunately, any activity that you do to improve delegation is time and effort that you have to spend. It is, of course, too optimistic to accept delegation as a black box (give a task and a deadline, and get it back on time with quality). But if you do not gain significant time and cost savings by delegation, and in the process, worry yourself to death's door, then delegation is useless. This is the point where most managers say, "Let me do it myself. It will be easier. Delegating is not worth the hassle."

What I am leading up to is: Pick the right people to delegate to. Some people, by nature, are much better at this than others. The good ones are very organized. When they get a task, they acknowledge it. They learn to prioritize or ask for input for doing so. They provide regular progress updates and intermediate deliverables. They stay on the task until you tell them that it is over.

There is a reason why other people are not the same. Such behavior requires certain personality traits:

1. Communicative: People who are open and communicative are easier to delegate to than people who are reserved or reticent. Some people are also better at communicating clearly and accurately than others who cannot provide a direct answer to anything.
2. Organized: Many disorganized people are very creative and innovative. But when it comes to planning and following up, you need people who have greater self-discipline and organizing skills. Such people can keep track of all the things that they are supposed to do, and arrange their schedules to be effective at doing them.
3. Resilient: Everyone always asks for honest feedback. Few can handle the truth; fewer improve themselves. As human beings, we want acceptance and hate disapproval. When someone makes a mistake, it is safer to hide and try to fix the problem than publicize it and risk negative reactions. And many people consider any suggestion to change their work as a rebuke or a threat. So you want to find and work with people who can handle pressure.
4. Communal: People who have an independent streak may be good performers, but you cannot rely on regular communication from them. They believe that they are doing the right thing and do not like any interference with their work (including your asking if things are going okay). That makes collaboration all the more difficult.

So, find the right people who will reduce your effort, not add to it. You may not always get a perfect match, so for the areas where the other person is deficient, discuss with them and come to a mutually acceptable and light-weight solution.

---

Technorati tags: business management, delegation, planning

DataModel and ViewModel

Recently in a conversation with a friend, he described a problem with an algorithm he was writing for displaying a calendar with appointment data. His original algorithm worked fine, but when faced with appointment conflicts because of multi-booking, he had to employ a 2-pass algorithm over the data so that he could display the information properly in the calendar view. Let us explore this problem further (please also visit the references).

A well-structured architecture separates the presentation of information from the storage of information. The MVC model is the most commonly-used architectural framework with a view representing information display and the model representing the data source. The advantage of this approach is that you can have multiple views for the same data. For example, you can represent sales data in a tabular format, as a bar diagram, or expose it as XML data for consumption by other applications.

However, one model cannot adequately serve the needs of different views without additional overhead. For example, while a view that displays the data as a table can operate with just rows and columns of data, a bar chart view needs additional information (such as number of values and the data value range) to display the data properly.

Some of this is additional metadata about the information that the model could take responsibility for. However, it introduces a complication: For example, if metadata information is calculated as part of a GetAllRecords() operation, it introduces an overhead that simple views may not desire. A different scenario is when you have no control over changing the model. This may be the case where you are working with specialized storage and the vendor provides you an API. Without access to the source code, you cannot change the model interface.

Consider a view that must work with a rigid model. One strategy is to implement a 1-pass algorithm that calculates metadata and changes the view on the fly. Changing the view may be appropriate in some situations (a map that displays cities one by one), but not appropriate, or even practical, in others (a map that changes zoom sizes as it renders its data).

A 2-pass algorithm is better, because it gives you the capability to calculate metadata and then render the view appropriately. However, this re-introduces the original complexity of making the view responsible for handling data.

Hence, you introduce the ViewModel, a model that is meant for use by a specific view. The ViewModel is a bridge between the View and the DataModel (which was our original Model). The DataModel is responsible for data storage and isolates the rest of the application from the implementation details of data access and manipulation. The ViewModel provides a model that the specific view can use to render the presentation more easily.

Now, the performance issue: Clearly, there is some amount of data replication and duplicate data processing. But how much duplication is going on? Clearly, most views operate with a subset of the entire data. A calendar displays (usually) one person's appointments over a small date range. A histogram is a summary of the data points, not the entire data set. However, if your View is processing a large volume of data and has performance issues, you may want to rethink many parts of the entire architecture itself, not just the Model.

Thus, the ViewModel converts the DataModel into a smaller data set that can be conveniently processed by the view. Since the ViewModel is associated with a specific View, its events, methods and properties can be changed to suit the demands of the View. All the time, the original Model remains intact to serve the common needs of all views.

References:

• Model/View/ViewModel: http://chanson.livejournal.com/123094.html
• Introduction to Model/View/ViewModel pattern for building WPF apps: http://blogs.msdn.com/johngossman/archive/2005/10/08/478683.aspx
• WPF Patterns: http://www.orbifold.net/default/?p=550
• WPF Patterns: http://blogs.sqlxml.org/bryantlikes/archive/2006/09/27/WPF-Patterns.aspx
• DataModel-View-ViewModel: http://mathgeekcoder.blogspot.com/2007/12/datamodel-view-viewmodel.html

---

Technorati tags: datamodel, viewmodel, MVC

Pitfalls of Performance-Based Compensation

Performance-based compensation has a simple premise: Show better results, obtain greater rewards. Commission-based professions (like sales) are performance-based. It seems like a good idea to have the same principle applied to all jobs within a company. Unfortunately, there are many disadvantages and risks in such an approach.

First, measuring performance is not an easy task. How do you measure and compare the performance of different individuals? You have to measure and weight many different attributes like quality and productivity. Such weights are subjective and are not constant in time or across jobs. Performance also depends on external factors like environment, vendors, customers, etc. The opportunities afforded to an employee can result in a higher or lower compensation for no fault of the employee.

Second, employees depend on one another. The output of an employee depends on the other members of the team. For example, if a manager does not provide timely information, an engineer may be slow to deliver on his work. In knowledge-based work, one employee's work cannot be separated and measured individually from the output produced by the entire team.

Third, no one thinks that they perform below average. Every employee thinks that they are doing their best. Rating them as "below average" will only result in disappointment and resentment. Many good employees derive great satisfaction from and take pride in their work. Money is only one motivator for them. Throwing money at them for showing better performance demeans them, while it does nothing to increase the capability of weaker employees.

Fourth, market value cannot be ignored. You cannot use the compensation model to pay your employees less what they could get in another company. That would only fuel attrition among the more talented employees who can easily get jobs elsewhere. Also, even less capable employees can get other jobs (think of the employees you fired - are they still unemployed?)

Fifth, a meaningful difference in compensation is difficult. Sometimes an employee performs 5 times better than another employee. Does that mean that the first person should be paid 5 times more? I am guessing probably not. No company can pay more than it earns. So compensation differences between employees are not properly related to the results they produce. This may have the unexpected effect of the high performers reducing their efforts to be more in line with other employees.

Finally, employee comradeship can be affected. Salary disparities can cause friction and jealousy between employees and resentment against the organization. This is particularly true in cultures (like Indian) where people share salary information as part of regular conversation. If salary figures are hidden, heavy consumerism by some employees can create the same effect if your compensation model is performance-based.

Forgoing performance-based compensation does not mean accepting lower performance from your team or setting lower growth targets for your company. You should demand high quality and professionalism from all your employees and pay them all well. Remove anyone who does not perform to the necessary standards. Special sacrifices from team members should be avoided and, if not, compensated through non-monetary means.

However, ditching the performance-based model is not that easy. Some employees want to contribute more and expect more in return - they demand salary differentiation. Your company or project may benefit from such enthusiasm, if properly rewarded. So it depends on what you are trying to do. If your company has some kind of performance-based compensation, it will take effort and time to take a different path. Convincing your organization (even if you are the Man) to accept the change may be harder than you think.

---

Technorati tags: business management, compensation

Riddles and Quiz in Interviews

Try solving the puzzle below. The answer and an explanation of how to solve the generic version of the problem can be found in this article by Jack Wert.

You have 9 coins, one of which is heavier than the rest. You are provided a weighing scale. What is the least number of weighings required to discover the heavier coin?

You may have already known the answer. Or you may not, but after deriving the answer or looking it up, you will feel confident to answer any similar question in future. Someone asking you this puzzle again cannot say whether you knew the answer or found it yourself, unless you tell them.

This is my primary objection against those people and companies who try to filter potential employees by asking them to solve puzzles and problems during interviews. It is impossible to understand if someone is intelligent through such methods, because these puzzles can be learnt.

Once you understand the basic technique for solving the problem, you can handle many similar problems too. And I am not just talking about above-average people. Most people can understand techniques and remember things, especially if doing that would lead to benefits like a job.

Another interviewing error is to convert the interview session into some sort of quiz contest. The interviewer asks obscure details about a particular technology, perhaps under the misguided notion that everyone in the world works in the exact same environment as he does. Many of these questions can be easily looked up through the API documentation, a Google search, or some book.

Such interviews are a waste of time even if you have a lot of time. The reason is that they measure the wrong things:

1. What you can discover is more important than what you can remember. There are more things that you can discover than you can remember. And there are only so many things you can remember because you are a human being and do NOT have a RAID array.
   
   
2. Your ability to learn is more important than your current knowledge. Today's innovation continues to make a mockery of what you have learnt yesterday. Especially in the software industry. If you were an expert 3 years ago and you learnt nothing new since, you are a dodo today.
   
   
3. The big picture is more important than the small details. The latter is not a percentage game. Out of the thousands of technical idiosyncrasies and code snippets that you have accumulated, you may use only a handful ever again. Worse, you may not even remember that you had saved the information and now have to look it up. Focus on the big picture. Get help when you need to know more detail. That is what the Internet is for.
   
   
4. The team is more important than the individual: The individual's ability to contribute to the team and work in a particular role is more vital to its success than the individual's capability. Effective communication, mutual respect, and common goals drive success, not a bunch of talented individuals thrown together in a room.

When you interview, try to understand the "fit" of the individual with your team. Is he a person who can change and grow? Is he interested in learning and self-improvement? Is he interested in the same technical things you are? Does he have the same values as you do? Is he motivated by the same things that you are?

Don't be in too much of a hurry to hire someone who has only passed your technical qualifications. If they don't blend in your environment, all they provide you is temporary relief for a vacant position. In the long run, their inability to grow and adjust will come to haunt you.

P.S. This article is meant for interviewers. If you are an interviewee, don't try to beat the system. Pick up a few puzzle books and learn the answers. Read the commonly asked questions in your industry and be prepared before going for the interview. I don't know why more people don't do this.

---

Technorati tags: business management, interviews, hiring, employment, interview puzzles

The Joy of Setting Up a Java Framework

Joy? Actually, no. Unfortunately, Java is becoming a victim of too much innovation. Anyone doing web development using a Java-based framework is faced with an abundance of choices and very little guidance and help from the innovators. Here, take a look at these lists: Apache projects, web frameworks, persistence solutions, web servers, IDEs, etc. What tools and projects would you choose for your next development?

Here are the challenges:

1. Many projects have no published timelines for compliance with the latest standards or emerging technologies.
2. They rely on external solutions to provide key functionality such as object-relational mapping.
3. They have a very cumbersome installation process that involves tinkering with a multitude of XML files.
4. Documentation is non-existing, elementary or plain wrong. For some reason, the developers are interested in only writing books, not free online documentation.
5. If you are interested in going bald, try combining multiple components to accomplish something useful.

How would you feel (or rather, experience) if you were given the following steps for disposing a bomb over the phone?

First, cut the green wire.
Now, cut the blue wire.
Sorry, before cutting the blue wire, you should cut the red wire.

That is how installation works. When you follow all the steps and try to run your application, you encounter an error. You spend a tremendous amount of time walking through and verifying all the steps again. It doesn't work. Then you read the next chapter and it says, "Hey, by the way, you should also do this." Wonderful, thank you.

A few years back, I remember seeing a website that had a single installation that bundled all the popular Java and open source products and promised to get it all working. I couldn't find that site, but I think that is a business idea that must be revived with respect to a Java framework. Here is what I think it should look like:

1. It goes without saying that all the components should be open source. But there are various types of open source licenses. Hence you may have multiple versions: "Free for any commercial use without restriction", "Free as long as you give away the source code", etc. Yes, that means not forcing people to figure out the difference between GPL, LGPL, Apache and BSD licenses.
2. Again, obviously, there must be different installations for different operating systems. My point in stating this is that each operating system has its own preferred way of installation. If it is a Windows operating system, provide an installer, not a ZIP file.
3. Any installation configuration must be done through a visual interface both at and after installation. The user should not even know how that information is stored.
4. The installation must package a visual IDE. Otherwise it is just a development kit. Eclipse is a popular choice.
5. The installation must package a lightweight J2EE server. When you create a new web application, it will run in that server when you press the Run button. No questions asked.
6. The installation must bundle the latest database drivers for different databases.
7. It must contain searchable user-friendly help and meaningful tutorials that are available locally. It should ideally have the ability to download and index new articles from selected web sites.
8. And the killer feature: It will offer its own choice of frameworks by combining the more popular frameworks to provide end-to-end functionality.

Let me explain the last point with an example. For example, Struts is an MVC framework. You could have a more powerful framework by adding a Java persistence component. So an installation process could let you choose "MVC - yes or no?", and "JPA - yes or no?". As you choose each one, it could offer you choices of the most popular solutions available.

When any of the underlying components change, a patch build should be available to download and install silently. The vendor should do the hard work of making sure that the new version of the component does not affect backward compatibility.

Doing something of this sort is pretty expensive, but not impossible. It would be similar in scope to creating a Linux distribution and certifying it with various hardware devices. Someone interested in pursuing this business idea has to hire testers, technical writers, and usability experts, in addition to developers familiar with different operating systems and environments.

What I have outlined is simply applying the Visual Studio concept to Java. Unfortunately, the companies that have the resources to do this, like IBM or Oracle, are intent on plugging their own Java solutions in the server and IDE space. Maybe someone else will come along. Till then, less joy and more pain.

---

Technorati tags: software development, Java frameworks

Communicating Software Estimates

In my last post, I had written about the conflict over software estimates between the people running the business and the people writing the software. I mentioned that most projects are likely to be underestimated in the first place and further pressure from the business side only makes things worse.

How can engineers get business people to understand this? Let us first look at some of the dynamics between the two sides:

1. Business people generally have more power in the company than purely technical people. They have the power to hire, fire, outsource, buy, etc. This power equation is a conscious or sub-conscious aspect of every communication.
2. Business people are used to trying to get the best deal. They would like to get the most features in the product for the least cost at the earliest. Before an estimate is provided, they may not actually need all those features, but after that, they believe that they are entitled to what they were promised.
3. Business people seldom have any idea of the complexity of changes. This sounds really obvious, but software developers don’t understand this. They are confused at the irrationality of changes requested when they are already behind on the schedule.

A new developer eager to please business people starts off by giving or agreeing to relatively aggressive estimates. After the project blows through deadlines on its own or with the help of a few "change requests", the developer becomes more circumspect. However, the answer here is not over-estimation (charitably called "adding a buffer"). Contrary to what many people think, over-estimating a task is not that easy in practice. Here is why:

1. Over-estimation cannot be so excessive that it belies common sense. For example, normally you cannot say that you need 30 days to change the layout of an existing report.
2. Over-estimation should not arouse suspicions of deliberate over-estimation. Otherwise, the other party will haggle and negotiate down the estimate, sometimes even below what is acceptable.
3. If the over-estimation is accepted, one has to show signs of being busy. Otherwise, the next estimate negotiation could bring up that issue.
4. And most importantly, over-estimation is over-estimating what you know today. It does not account for misunderstanding requirements or change requests.

So what does an engineer do? The right answer, in my opinion, is to start with the statement, "I want to help you, but I don’t know how much it will take without learning more. Can you help me understand the problem better?"

This statement does two things. First, it puts you on the same side as the business people and both looking at the problem. Being on the same side is important to put thoughts of self-benefit, negotiation and compromise away. Secondly, it allocates responsibility to the business people to help you get to the right answer. It establishes a quid pro quo: You need an estimate; you provide the knowledge for the estimate.

The next step should be to arrive at an understanding of how long it will take to understand the project enough to provide an estimate. This could be a single meeting or it could be a few weeks, depending both on the complexity of the project and the current knowledge of the estimator. These sessions will attempt to reduce the mystery around the estimates.

What I am suggesting seems suspiciously like a waterfall model, but that is not my intention. A fixed point estimate in a waterfall model is a loss for both sides because it does not deal with the reality of change management. Any project lasting more than a few weeks will have to undergo change because of changing business needs. A fixed estimate forces the technical team to reject valid business needs or, worse, accept them within the current estimate.

An iterative model that accommodates changing business requirements is a better option. But even in that situation, business people need estimates in one form or another (Sprint backlog, dynamic project plans, etc.) to plan their activities. By collaborating with them to understand before estimating, you both win.

A final question remains: What do you do about the person who says, "I don't care what you think it takes. I must get this done by date 'x'. Otherwise the following bad things will happen to the company/you/me."

After many years in the field, here is my opinion: Even if a negative outcome could affect you, it is not a problem you have created. You do not have to feel obligated to solve it if there is no solution. Even if you put your best effort, it is very unlikely that you will hit the deadline and more likely that you will be blamed in some way. There is nothing in it for you.

I realize that personal circumstances (such as not losing your job just then) may prevent someone from taking such a stand. But absolving yourself of the responsibility and preparing for the worst can relieve you of the mental stress involved in death marches. You are trying your best. It is not going to happen. Let it go...

---

Technorati tags: software development, software estimation