Comments (2)

By Krishna, October 16, 2009

In short, no.

But why am I asking the question in the first place?

Well, much discussion in the technical blogs (such as the NOOP.NL Top 200) is centered around the idea about whether a mediocre programmer will land on a blog where somebody is talking about adopting a simpler approach to programming and then use that idea to stop improving himself and continue writing bad code.

It is a useless subject to argue about because bad programmers do not haunt technical blogs. This is how such a person works:

1. Come in at 9 am or whatever time you are supposed to come in.
2. Find out what task to do.
3. Is the task something that looks familiar? If so, go to Step 7.
4. Interrupt someone in the office. Do they know the answer? If so, go to Step 7.
5. Google for the answer. Is there anything there that works? If so, go to Step 7.
6. Complain to the project manager that the task cannot be done and get something that can be done.
7. Do the task.
8. Is it time to leave? If so, go to Step 11.
9. Is there any other task? If so, go to Step 3
10. Kill time until it is time to leave.
11. Leave.
12. Enjoy the rest of the day.

What they don’t do is spend time visiting technical websites or blogs unless they happen to be landing on them via Google. And those Google visits are only for the purpose of finding some code that works. And issues like intellectual property rights be thrown to the winds.

So let us be clear about the audience when we are arguing about technical debt. We are having an argument among people who know how to write good code and how to test it properly. Some of them use a few more tools and a little more process than the rest, who are constrained either by their lack of knowledge, awareness or business situation to imitate them. No one among this crowd is writing bad code, as in copy-and-paste coding or bad variable names.

Therefore what we are really discussing about and what needs to be answered by people are:

1. You are a good developer who gets things done without Process X or Tool Y.
2. How much beneficial for you would it be to use X or Y?
3. How much time, effort and money do you need to invest for using X or Y to get started in and then be proficient?
4. What is the return on investment if you use X or Y?
5. Let’s understand if X or Y makes sense in your situation, including timelines, budgets, technical environment.

There are literally thousands of programming tools, techniques and processes out there, all claiming to increase productivity. So whatever the “best thing” you are promoting is, you have to justify it in these terms to the good programmers. And someone who cannot justify that is simply selling snake oil.

As for the other programmers, they are out there enjoying the day and definitely not dreaming in code.

Comments (2)

By Krishna, October 14, 2009

Martin Fowler clears up some of the confusion around the term “technical debt“:

The debt metaphor reminds us about the choices we can make with design flaws. The prudent debt to reach a release may not be worth paying down if the interest payments are sufficiently small – such as if it were in a rarely touched part of the code-base.

So the useful distinction isn’t between debt or non-debt, but between prudent and reckless debt. [...]

there [is] a difference between prudent and reckless debt, there’s also a difference between deliberate and inadvertent debt. The prudent debt example is deliberate because the team knows they are taking on a debt, and thus puts some thought as to whether the payoff for an earlier release is greater than the costs of paying it off. A team ignorant of design practices is taking on its reckless debt without even realizing how much hock it’s getting into.

It seems that Martin is suggesting that the best debt you should have is the prudent-inadvertent debt, where you only understand the technical debt after working with the application. It is not a case of recklessness where you decided to ignore design, or be content in your ignorance. Or a case where you weighed all the pros and cons and then decided for the quick-and-dirty approach. The prudent-inadvertent debt is something that happened despite your best intentions and efforts.

The way to explain this is that there are potentially different ways to design an application. For example, if you take the creational design patterns, you may have to choose between competing patterns. You may start using a Factory pattern, but then need to change that for something more flexible. Or you start out with a relational database design and then later find you have to denormalize or use a NoSQL database because the application became too successful. It is hard to predict the changes in requirements and sometimes, you have to make the decision using present day considerations.

There are always gray areas in those quadrants. Assume that you are building an online application. It is impossible to know ahead of time whether you will get 100 visitors per day or 10,000 or a million. You could choose to target a million, but that could be over-engineering the application. Maybe 1,000 seems a happy medium or maybe you are making a terrible mistake of under-estimating traffic. Making design decisions in these cases can seem reckless or prudent after the fact.

Perhaps the real mistake is to think of design as an independent process. Much is tied to business decisions and forecasts. And where your technical debt falls is determined by those business factors.

Comments (0)

By Krishna, October 2, 2009

Hope you had the time to read through all the essays linked to in yesterday’s post about the reactions to Joel Spolsky’s article “The Duct-Tape Programmer”. Here is my take on it.

The use of the words “Duct-Tape” was only chosen by Joel as a flame bait, as he admits on his Twitter feed. Joel has a history of using colorful terms (like “architecture astronauts“) to represent concepts (and get reactions) and this was just one of them. People need to get over the post title and look at what he was trying to say.

What Joel intended to say (I presume) was, “Program with the tools and technologies you know instead of choosing exotic and less-understood technologies and methodologies“. That is a sane, non-controversial statement that you can apply to most software development, because it reduces the risk of failure by using unproven software components.

Joel is also right in pointing out that many programmers are afraid to say “No” to the latest stuff because they are “too afraid of looking stupid“. Programmers are proud people and they hate to admit that they don’t know something. They are also eager to try out new things. When you have a person in the team strongly advocating something new, they want to investigate it. That doesn’t necessarily mean it is going to fail, especially if the new technology is helpful. It is just that nobody knows any better until they try it.

But Joel is wrong in saying, “any kind of coding technique that’s even slightly complicated is going to doom your project“. That is too broad a statement and, taken at face value, it essentially justifies learning nothing new. Every new technique always comes with a learning curve that you have to get through to master it. The issue is: How much of a learning curve is it and what is the cost-benefit in terms of development speed?

For example, take a technology like ORM (object-relational mapping) or a better unit testing framework. You can build a perfectly working application without them and using your existing tools. And you could also build one using them, but let’s say you have to learn. You may be able to save time in the long run, but you have deadlines to meet. So what do you do? Well, if the deadlines are further off, you could make an effort to understand the technology and see if you are comfortable enough to use it. On the other hand, if your release dates are very close and non-negotiable, maybe you should try experimenting on another project. Maybe learn it in your spare time so that you are ready to use in your next project.

As for the reactions, on a second reading, I felt Uncle Bob’s take on it very even-handed. Cedric launched an unnecessary attack on Uncle Bob, but he had some other good points about Agile inflexibility (To self: future post perhaps). As for linking Joel with StackOverflow, Jeff Atwood, not Joel, deserves most of the credit. Also, don’t forget that StackOverflow used some of the latest technologies (ASP.NET MVC and Linq-to-SQL) that the developers were not familiar with when they started coding. Now, they did go past the initial deadlines that they had set, but despite great transparency about what they were doing, no competitor emerged.

Now, Joel’s post is surprising in the context of FogBugz. The thing is, FogBugz has been around for a long, long time (since 2001). But Joel continues to write “Shipping is a feature“, almost as if he is paranoid about some competitor of FogBugz. And he frequently writes about people who talk to him about some new stuff, with his description of the guy with the coffee mug in hand coming to his office sounding a bit too detailed to be fictitious. So is this some kind of internal FogCreek debate about the existing FogBugz design?

Comments (6)

By Krishna, August 28, 2009

Because it is not as bad as saying, “Don’t blame me for it.”

“It works on my computer” is one of those things that reminds me of how pedestrians behave as drivers and vice versa. The pedestrian feels that the driver should be careful and drive safely, but the same person, as a driver, hates all the jaywalking pedestrians. All programmers, at some point in time, have replied to a bug stating it works on their system, but it drives their team leaders (once programmers themselves) crazy when they hear them.

To the manager, this statement sounds insane. He thinks, “Why does it matter whether it works on your system? We are not talking about your system. We are talking about a customer who is unable to do their work because the application doesn’t work as it is supposed to? Why are you being so callous?”

The developer, simultaneously, thinks, “There shouldn’t be any problem with the application. After all, I tested it a million different ways. It passed testing and we didn’t have any problems so far. Maybe the person at the customer site is using an unsupported environment. Maybe they are missing a necessary component or turned off something for security reasons.”

The developer remembers the several times in the past when he had to deal with a supposedly huge problem in the application and then found out that the user did not have the network cable plugged in (yes, Google too stopped working!). Or that some IT wise guy decided to turn off JavaScript (all those nasty XSS exploits). Or even when the customer was specifically told not to use anything less than Windows XP, and they decided anyway to run the application on a Windows 98 box (it happened to be lying around).

Maybe this time, it is different, but probably not.

Yes, the developer knows it is not his fault. But saying that makes the developer sound too defensive. Better to use an actual fact, i.e., the application has been known to work. It is not a problem with the code he wrote. Inside, the developer also knows that he will have to fix the problem despite this statement. But he wants to win this mini-battle to maintain his credentials.

So, manager, when you hear a developer say, “It works on my system“, don’t burst into flames. Just say without any sarcasm, “OK, it is good to know that the application is working here. [significant pause] Now, can you call the customer and get it working on their system?“

Comments (2)

By Krishna, August 27, 2009

Tim Barcz has an interesting post about striving for software with zero defects:

I think we should not shy away from holding ourselves to an excessively high standard.  Frankly, I’m more familiar with how to do it as a developer having years of building software to gain knowledge and experience from.  I don’t have all the answers on how do this from the other aspects but I don’t think this means we stop trying.  I think quality should be the first and highest goal.  Speed should follow quality.  If you’re keeping a high quality standard and feel overwhelmed that the work is backing up behind you and you can’t ensure the highest quality, please don’t hesitate to talk to me about it and we’ll see if priorities can be reworked.

I agree with the general principle in the context of Tim’s work environment, which seems to be software development in a large company. Within that context, it makes sense to strive for as much quality as possible. I also like Tim’s statement, “Speed should follow quality“. This is important because only by explicitly stating that quality supersedes deadlines will you get the required behavior from your team members.

If you want better quality, it is not sufficient to just set goals. It is also important to make sure that your team is aligned towards achieving that goal. This means when it comes to decisions that require prioritizing quality and something else, you always choose quality. This is not easy, especially when the benefits of quality take a longer time to materialize and the effects of delays in project deliverables are more visible.

Having said that, striving for zero defects does require taking note of the following real world situations:

1. Bugs will still occur for a variety of reasons. You still need to plan for handling them after the fact. As they say, hope for the best, prepare for the worst.
2. Any high-quality environment necessarily requires closer review of bugs to avoid future occurrences. Take care to stay focused on processes and not on people.
3. A low number of bugs could result in statistical anomalies such as disproportionate number of bugs from a few persons. Be careful how you interpret such data.
4. Be mindful about the opportunity cost of fixing every single bug, especially those which could be resolved by less expensive methods (changing the hardware/software environment, training and so on)

Also be careful of unintended consequences. A strict focus on low number of bugs could result in developers trying to reduce “bug count” instead of actual bugs, by re-classifying bugs or simply not reporting them. Avoid situations where metrics can be gamed.

I don’t remember the book I read this, but there was a case study about how the hospitals that reported more errors had better quality healthcare. The reason was that the doctors and nurses in those hospitals were not afraid to report their errors and so were able to honestly discuss them and find ways of preventing them in the future. The hospitals reporting fewer errors were focused on error count which were probably tied to some draconian reward-and-punishment system.

In other words, quality has to come from the bottom-up. Trying to enforce quality from the top can lead to distortions. Tim seems to be advocating the first and I am all for that.

Comments (1)

By Krishna, July 19, 2009

Tom DeMarco has written a thoughtful and introspective article on software engineering (PDF). In that, he questions some of his early work on metrics, wondering if it is still relevant. He suggests that there was too much focus on time and budget deadlines when the more important goal should have been making great software.

strict control is something that matters a lot on relatively useless projects and much less on useful projects. It suggests that the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value. To my mind, the question that’s much more important than how to control a software project is, why on earth are we doing so many projects that deliver such marginal value?

DeMarco’s article can be interpreted in different ways, but I think he, like many people, is trying to reconcile how to find a balance between getting results from a software project and managing unpredictable human beings. Many software engineering principles find their inspiration from methodologies used in other engineering disciplines and industries where the “human effect” has less of an impact on project success.

Let’s take a software engineering concept like metrics. They are very useful, but they suffer from many problems:

1. Measuring something changes the behavior of the team with regard to that metric. If you measure bug count, it will decrease, but perhaps at the expense of something else that is not measured.
2. Collecting and analyzing metrics takes time and money. As DeMarco points out, controlling a software project is more useful when the marginal value is less, but if you spend money on metrics, how much ROI do you get?
3. Metrics collection affects the human relationship aspect of team dynamics. How do you balance your trust in metrics with your trust in the delivery capability of the team?

So while they provide valuable insight into the present state of the software project, they have a distorting effect. But if you throw out metrics entirely, you are flying blind with no idea about the quality or completion of the project. So there has to be a happy medium somewhere.

The problem with the formalized concept of software engineering is that it gives you the illusion of control. The project manager assumes that by following a set of processes, they will achieve the goals of the project. If there is any deviation, it has to do with not follow the methodology to the letter. Something was missed, or something was not done 100%.

We can see this at work with how people approach CMMi, Agile, PMP or any other methodology. Each has its own merits, but the implementors (project managers, PMOs, quality teams) want complete adherence to the methodology (straight or tailored) and deviations are treated as non-comformance. If a project fails, it is always a mistake of not following the principles properly.

And like DeMarco, I feel that this is wrong. No methodology can give you the whole truth. There are methods and techniques to help improve software quality and productivity. But there are never any silver bullets and we have to let go of assumptions that we can find sure-fire ways of producing perfect software. As the software field continues to innovate, we have to also innovate in ways of managing teams and producing better software.

Finally, read DeMarco’s analogy about controlling the teenager again. Many project managers and team leaders commit exactly that mistake – treating their team as a bunch of teenagers that must be controlled, measured and made conformant to a set of policies and processes to make them successful. What hubris! Why not trust them to do the right thing?