Comments (0)

By Krishna, January 23, 2008

A few days ago, I had a discussion with a programmer about his solution to a particular problem. Since he was having difficulty getting it to work, I suggested a different approach. He said that he had already considered that, but it would be too expensive in terms of performance.

So I asked him how much more expensive the 2nd approach was. Was it a few percentage points (which we could live with) or orders of magnitude. He said that he was not sure, but based on the logic, it would be slower. He had never measured it.

Let me not end the story by saying who was right or wrong. The point I want to illustrate is that in many situations, people tend to make decisions without ever having factual data. Their decision may be right. It may be wrong. But at the time they make it, they base that on pure speculation without any basis in real data.

It is interesting how often this happens in software development decisions. Why was a particular technology, language or methodology chosen? Usually, the decision maker will point to it being “better”, “higher-performing”, “more efficient”, etc. than other choices. Unfortunately, those persons cannot explain why other companies (and perhaps superior-performing competitors) seem to do very well with those sub-standard technologies.

In many cases, a person makes a decision based on one primary factor, and then uses other reasons to justify the decision. For example, a programmer may be more comfortable with Java or C# or Ruby, because of greater experience in and knowledge of the language and tools. It is perfectly okay to say, “This works for me because I know this better”. Instead, we get religious wars suggesting that there is only one pure language and the others are all doomed. Apparently, no one has created working software in those other languages.

Once someone has made a decision, it is very easy to find evidence to support it. Take your pick in any area: politics, religion, technology, arts, etc. Think of one belief you have right now and Google it. You will find many results supporting it. Now, Google the opposite of that belief. Lo and behold, you can find just as many results.

The more important the decision, the more people are prone to guessing. What should the company do for its employees and its customers? How should it respond to market trends and competition? One of the most frequent sentences I hear from people in various companies is, “I think so (or he/she said so) and hence we will go ahead and do this.”

How about saying, “I have no clue. Let us experiment. Let us try this out. Let us get the results. Let us ask people and get feedback. Maybe we are making a mistake. Let’s see if it works. If it doesn’t, let us try something else.”

The key thing that stops people from doing this is the horror of admitting they are wrong. Of course, if they have guessed wrong, they will be proved wrong. But then, they can always blame bad luck and other convenient scapegoats. Some people can never make wrong judgments.

Very often, leaders and managers do not explain the reasons why they ask for something. By doing that, they put themselves in an impossible position. There is no one to correct their reasoning or provide contrary data. And when their decisions start showing signs of failure, they are hesitant to change them because it highlights their original wrong thought process.

I think the solution for this problem is a method of decision-making that incorporates honesty and introspection. What is the real reason behind a particular decision? Sometimes, the honest answer to that question can be pretty ugly. Maybe the decision is made out of fear, or selfishness. Maybe so, but denial will only lead to problems down the road.

Understanding the true reasons behind a decision will help divorce you and your ego from the decision and marry the reasons to the decision. If the reasons change, you can change your decision without thinking that you are at fault.

Comments (4)

By Krishna, January 12, 2008

Unions have their advantages and disadvantages, but one thing I have disliked about them is that they provide a false sense of job security in a fast-changing global environment. They imply that your job will last for ever, your pension will be intact when you retire and technological, economic and political trends can be somehow reversed. The sad part is that people believe them and do nothing to improve their career prospects until it is too late.

In the professional space, particularly the software industry, unions can rarely be found, and fewer people believe in lifelong jobs. In fact, because of greater opportunities, employees change jobs frequently, particularly during an economic boom. We also see migration of professionals from one country to another, a trend that has only increased in recent times.

Yet, despite these trends, professionals possess another kind of old-school thinking. They don’t believe in the eternity of their job, but they believe in the eternity of their profession. Each person believes that companies will always need people belonging to their profession. And they believe that the essential nature of their profession will not change.

The past has shown us that this is not a right assumption. We have seen the slow death of many types of jobs in agriculture, manufacturing, manual labor and other fields.  But past examples of out-of-demand professions do not mean much to people in today’s in-demand professions and so they tend to ignore them and the lessons they teach us:

1. Consumers of any free-market society will drive costs down and make commodities out of products. Computers, that not so long ago sold for thousands of dollars, now sell for a few hundred. Shrink-wrap software is on the same path, with the increasing sophistication of web applications.
2. Consumers cluster around a few market leaders, rendering the rest irrelevant and unprofitable. For example, despite the ease of creation of web applications, if you look at the market today, it is dominated by a few big players (Google, Yahoo!, AOL, Microsoft), some open source initiatives and that is pretty much about it.
3. Business owners will try to reduce costs by every means possible: automation, outsourcing, cheaper labor, better business processes and tools, etc. Such trends have only accelerated in recent times because of greater globalization, privatization, technology, etc.
4. Growth of an industry also signals expanded supply of resources. For example, if programming is a good career, you can expect that more people will learn programming, join the labor pool and put greater pressure on salaries.
5. Innovation tends to taper off at some point because of decreasing benefits. This results in greater standardization of tools and techniques, and consolidation of the labor pool through lack of differentiation.

No profession becomes extinct overnight. There is a sequence of events that reduce the demand and profitability of a profession. The problem is that unlike the past, these events are happening at breakneck speed and there is increasingly less time to understand and respond to the changes around us.

Here are the realities for every software professional today:

1. Advanced development tools today have significantly reduced coding effort for building applications. What is required today is a greater emphasis on product quality and usability.
2. Computing power has made old ideas about performance less relevant. At the same time, there is a greater need for performance efforts on the big (data centers) and the small (mobile phones).
3. What can be outsourced will be outsourced. Hence, software developers must focus on what cannot be outsourced as easily, such as interacting directly with customers and understanding their needs.
4. There is simultaneously consolidation (such as in programming platforms) as well as innovation (such as better technologies on successful platforms). The astute developer must stay ahead of both.
5. There is increasing encroachment by one profession on another’s territory. For example, developers are now able to handle many tasks previously handled by specialized testers, database developers, system administrators, etc. because products are becoming easier to use.

It is difficult to predict what we will be doing in 2015 or 2020, but one thing is for sure: The changes in the next 7-15 years will far outpace what we have seen before, radically transform our jobs and bring many new opportunities and challenges. Expect a wild ride ahead.

Comments (2)

By Krishna, January 12, 2008

The standard HTML drop-down control is convenient for allowing users to easily select from a set of choices. It is preferable to using multiple radio buttons for selection because it helps reduce screen real-estate needs. It is not only a data entry field, but is also frequently used for providing a set of user actions or screen navigation.

However, it has a few shortcomings that make it inadequate for many purposes:

1. As the number of choices increases, the drop-down control becomes more difficult to use. The user has to spend more time and effort scrolling to find the right choice. Ordering the values alphabetically only helps to some extent. When the number of records increases into the thousands, it presents technical difficulties, such as performance issues.
2. Although the user can type a few letters to bring the focus to a particular selection, those letters are not displayed on the screen. If the user wants to cancel their input, they cannot hit the backspace key, because the browser would go to the previous page. Instead the user has to wait for 1-2 seconds and start typing again from the start. Also, not having a regular text input prevents the application from using user input to filter the records in the drop-down instead of simply navigating to the record.
3. The drop-down control cannot easily display a columnar format for multiple fields (say Serial Number, Product Name and Vendor) for a single record. You could resort to pseudo-formatting by appending the fields, padding them with spaces and using a fixed width font, but that would conflict with the fonts on other fields. Even if you manage to show multiple values on a single choice, the user cannot sort them as they could normally by clicking on the header column.
4. Although users can manage drop-down records in an administrative section, they frequently request the ability to do so in any screen where those records appear. Users find it inconvenient to cancel their work on a form because a drop-down value was missing and have to go to another screen to enter the record.The standard drop-down control does not allow any easy way of adding a new value.  One way around this is to provide a link to a modal window that would allow the user to add, update and delete records in the table referenced by the drop-down. However, those changes must then show up back in the drop-down control choices in the parent form.
5. Sometimes, users have the need to “expire” choices in the drop-down control so that they are not available for selection in future data entry. An example would be “East Germany” in a country selection. However, when the user wants to edit a prior record, the country drop-down no longer has the previous value and it cannot be used to display the value in the form. This even if the user may have no intention of editing that field.
6. This last one is more a browser issue: In some versions of Internet Explorer, the drop-down control prevents any DIV controls to be overlaid over them. If you had a floating layer, the drop-down would show through the layer, resulting in an ugly and unusable display. Developers have to come with various code workarounds to solve this problem.

This is not an exhaustive list, but let me stop here. I think there is a real need for programmers to stop using the default HTML drop-down control. The better option would be to design a combo-like control that has the following features:

1. Text box: This can be used to filter the values in the drop-down or directly enter the value. For example, a product id can be entered using a barcode scanner instead of scrolling through a long list. The text box could be used to display “expired” choices that may not be available in the drop-down list.
2. Drop-down list: The display could be configured in different values – a simple list, a hierarchical list, a sort-able table, etc. The values themselves could be directly edited or deleted in the display itself, and new records added. The display could be a static pre-loaded list (if there are only a few records), an asynchronously loaded list (to reduce start-up delays) or an incomplete list that displays a few records and more on request. Searching can bring the right records to the top of the incomplete list for easier selection.

This is not the same as the standard Windows combo-box control as the drop-down list is more configurable in terms of display and event handling. It also goes without saying that keyboard and mouse interaction with this combo control should try to retain the efficiencies of the simpler drop-down control. These include keyboard events such as ENTER, ESC, TAB, SHIFT-TAB, arrow keys, etc. and mouse events like scrolling, clicking (inside and outside the control), etc.

One key design principle when creating this control should be that it should have the capability to entirely displace all instances of the standard drop-down control. This is required for the sake of consistency and uniformity in user interface to avoid user confusion.

Comments (6)

By Krishna, January 7, 2008

There was a comment from the author of Martian Geek to my last post on the necessity of project management asking, “How would you justify the success of start-ups which might be running well with very less ppl with managerial expertise? There also the person with the idea is not always the coder/implementer.” A very valid question and I thought I would answer it through a post instead of by another comment.

Start-ups are, by definition, very different from regular companies. In the software industry particularly, start-ups are very small, often consisting of a handful of people, maybe even just 1 or 2 people. Very often, they have very limited capital and considerable unpaid effort is spent by the founders. Start-ups have a high rate of failure, not surprising considering that every start-up is trying something new and untested.

A start-up can succeed if and only if it meets an unmet and substantial market need. If there are many start-ups in the same space, the one with the best overall value or the one with the criterion most useful to customers will succeed. But this is much easier said than done. For one thing, how does a startup know what the market needs?

The reality is that a startup doesn’t have the resources to extensively study the market needs before committing to building a product. For the most part, a startup possesses an idea that finds some acceptance among a limited sample of people. This is not to say that the idea is right or wrong. It is just that most startups don’t have a lot of information about the broader market and have to make assumptions.

And it doesn’t end there. However beautiful the idea, a startup has to take up the hard work of informing people about it, and getting them to try, purchase and use it. This requires heavy investment (time and money) in marketing, customer service and product updates. Finally, many external factors can affect customer decisions – the economy, technology infrastructure, the competition, etc.

As you can see, start-ups are risky. The point about risk is that managers are ill-suited to take on risk. One of the first things that a manager learns is to establish greater control and reduce as much risk as possible. That is why a manager talks about planning, documentation and processes. Managers would rather opt for a sub-optimal, risk-free solution than an optimum solution that carries the possibility of heavy loss. This is not necessarily a bad thing, but a startup is about extremes, not the happy medium.

Many start-up development practices would be frowned upon in a process-based company because they are not repeatable and sustainable in the long-term. But from a start-up’s point of view, they are perfectly valid, because the reasoning goes: If the start-up fails, then nobody cares. If the start-up succeeds, there will be enough resources and money to fix the problem.

The tight-knit nature of a start-up also makes many project management activities less relevant. Startups don’t manage time – they spend as many waking hours as possible on getting the product out. They don’t manage cost – they try not spending any dime if they can help it. They don’t manage quality and scope – they stick to an initial feature set and get out a working version as quickly as possible. They don’t manage communication – people just walk over and talk to each other. And so on.

This does not mean that if you have a lot of management experience, you cannot succeed in a start-up. It just means that you have to be more a producer than a manager. You have to accept more risks and downsides when there is the promise of greater rewards. You need a greater willingness to accept ideas and change your own views if it meets the greater needs. And importantly, you have to keep holding back on creating new processes that may introduce bottlenecks until you have reached your potential.

Now, as the start-up finds success, a new element is introduced – the startup now has something to lose. Since it is still growing, it has more to win than it has to lose, but it does have to protect what it has gained. And that is how a start-up morphs into a regular company. That is the beginning of process-based practices in a company, something that can potentially turn into bureaucracy, red-tape and inflexibility, if handled badly.

But that is how it starts and that is part of the evolution of any company. A start-up founder can continue to be successful if they can manage the increasing demands on the company. However, inexperienced founders find it difficult to relinquish past habits and learn new tricks. They have to hire or make way for more professional and experienced management at different levels in the company to take the company to greater success.

Summary: Start-ups succeed through taking on more risk and practicing less conventional management. As the start-up gains success and those gains need to be protected, more professional management must be introduced gradually, balancing both growth and security needs.

Comments (3)

By Krishna, January 2, 2008

A project manager is very often a liaison between 2 sets of people: Those who do the work (engineering) and those want the work (sponsors). The manager’s role is to bridge the gap of understanding between the two sides. One person I worked with referred to this role as being a “glorified coordinator”. The image of a sandwich, with the project manager in between, comes to mind.

The “middle man” concept of a manager raises an important question: Why don’t the sponsors talk to the engineers directly? Not only will this avoid the direct costs of having a manager, it also reduces the inevitable information loss when communication happens indirectly. Managers are usually the primary vilification target of popular business literature. So why not get rid of all the managers?

Such an approach, while tempting, has many pitfalls. First of all, even if you get rid of the title, you still have project management tasks. These include estimation, planning, controlling scope, managing risks, coordinating with other groups, and many other activities. You can call the person who does this a “senior programmer” if you may, but they are project management tasks.

Developers, untrained (theoretically or practically) in project management, can find it difficult to meet the demands of the job. For example, they may see technical risks, but fail to see business or human resource risks. They may lack the capability to coordinate and negotiate with people at different levels of authority, something that can challenge even senior project managers.

The project manager can sometimes bring a useful separation between the sponsors and the engineers, and can bring harmony to disputes regarding expectations. For example, when there is an argument over deadlines, the project manager is better placed to act in the best interests of both parties because she can see the business need and the technical challenges equally well. Since the project manager has no emotional attachment to either side, she can do a much better job of setting the right expectations.

While a project manager can add much value to a project, they can also over-manage. When a project is going reasonably well, a full-time project manager does not have much to do, and many of them resort to “active” project management. Sometimes this can be in the form of time-wasting status meetings. At other times, the manager may start becoming more intrusive into the work of the developers, creating little benefit and more angst.

The line between too much management and too little management is not very clear. It depends on the organization, project requirement and team members. But perhaps a simple rule may help: Take up the tasks that remove obstacles from the developer’s path and meet the needs of the sponsors.