Comments (0)

By Krishna, March 8, 2010

Stephen Forte makes what he thinks is a case against having “rock-star” developers in a team. You can read the post, but, in short, there was a programmer “John” on his team who picked coding arguments with the rest of the team. The final straw was that John burnt the whole team by directly moving his changes to production without going through the QA process and bringing down the site.

Stephen’s example is not very strong. A great programmer, by definition, should be someone who also understands and advocates good build and deployment practices. John was not such a person. It is also quite possible that a lower-grade programmer could have made the same error if they were under pressure from some management big-shot. You have to blame the incentives here.

In any case, these kind of train wrecks are easily preventable. An analogous example is locking the door if you are in the bathroom. Yes, people should knock before they try to enter it. But you cannot force everyone to follow such “best practices”, especially what is foremost in their mind is not the prospect of causing embarrassment to another person. Instead, what you can do is prevent someone from causing damage if it is really important to you.

So with respect to production servers, the simplest solution is to lock down access and grant it only to people who have a stake in following the process. This small group usually includes the system administrator responsible for the box, the project manager (or team leader, as the case may be) and the QA manager. All others, especially programmers, are more motivated to say that the work is done than to prevent any damage because of a mistake.

I am actually not in favor of providing access to the lead programmer, either, even if they are very experienced. The path should go through QA, who test the build or patch on a Test server that is as similar to the production server in terms of capability, environment and data as possible. One QA person is in charge of delivery and they sign off on releasing the build.

In a smaller organization, there will some overlap between roles, but do ensure that the person in charge of QA has the responsibility and sole authority for releasing code to production. That person should be independent of any influence by managers to speed up delivery of projects before adequate testing is done.

The other step is to be absolutely uncompromising about crossing management lines. There has to be one person (whatever the title) who has the authority and responsibility for accepting tasks for the development team. I am using those 2 keywords again, but what it means that only one person can accept a new task from someone outside the team, and once they accept it, they are fully responsible for it. This person maintains the backlog and if any manager wants to circumvent the hierarchy and talk directly to a developer, they will own it including breaking the schedule of other managers, crashing the servers and so on. And also, perhaps most important to them, that the task would be dropped without notice once the backlog manager learns about it.

Now, Stephen defines a rock star developer as “someone who, while talented, thinks that they are the ultimate guru and that everything should be done their way.” As I see it, you have hired a manager when all you needed was a programmer who can follow instructions. There was already a manager (you!) on the team. John doesn’t belong. His personality is such that he should be leading the team and swimming or sinking with the decisions he makes. But he probably doesn’t have the necessary resume to land that role in any company yet. In the meantime, he is a nightmare on the team because he tries to manage his peers. That is the real problem.

The difficulty in avoiding such hires is that during the hiring process, the rock star developer is sometimes indistinguishable from the talented, easy-going developer. Both are bursting with energy and ideas, some of which are actually pretty good too. You only learn the full extent of their behavior after a few weeks. The easy-going developer is comfortable with accepting authority and there is a give-and-take with respect to ideas. The rock star developer doesn’t understand why compromises are sometimes necessary and is quick to condemn differences in opinion as stupidity or evil. We all do the same to some extent, but the rock star developer’s condition is almost pathological.

The obvious route is to let the developer go, but if you don’t have a choice (in the short term), then one way to handle them is to divert them into tasks that require heavy lifting in research and development. They are now isolated from the rest of the team and the day-to-day grind, but they are kept extremely busy because they have to deliver something that works. This is something that I suspect rock star developers like, because they get to play the full court from analysis to recommendation. However, ensure that you have set the right parameters for accepting the outcome of the research so that you don’t get a recommendation that you will be forced to reject, thus exacerbating the problem.

Comments (1)

By Krishna, October 21, 2009

Rob Conery pointed out an interesting comment on his IoC post:

In this simplified example above you switched from 3 lines of easily understandable code to something that requires code in the global.asax, another couple classes, and an interface. The end result contains an extra line or two of code, but now the whole thing is “reusable” and can be unit tested. I happen to agree with Joel’s quote which you openly mocked… it *is* more convoluted. It *is* harder to read and grok. And this was an overly simple example… I’m sure it gets more convoluted with large projects.

I am going to ignore IoC for a minute and take this in a general context. Some junior programmers have a tendency to write lengthy code instead of splitting the code into different classes and methods. The code, in itself, is not bad. I am not talking about poor variable names, bad formatting, copy-and-paste or low level silliness. But it is structurally deficient in the sense that

1. Each method will have many lines of code doing different things. In other words, the method has multiple responsibilities.
2. The coding is algorithmic, i.e., although they are coding in an object-oriented language, the coding is very linear in terms of execution.

I used to have a different perspective about people who did this kind of coding, but over time, I have concluded that this has got to do with the (learnable) ability of programmers to deal with complex systems consisting of different building blocks.

Junior programmers need to see, feel and know the entire program. They feel lost and confused otherwise. This is only possible if the code is one place and it has a very linear structure. This is enough for simple programs. But more complex programs require more code.

Well, it happens that a tree-like structure of code is also easy to understand because it is a easy progression from a linear code structure. You start from the root (the main method), navigate linearly, traverse each branch (if you come upon one), re-trace your steps and keep going until you are done. If the tree structure is not deep, you don’t have to keep too many things in your head and the complexity is managed.

The tree-structure is achieved by moving chunks of code into different sub-routines. But here is where the problem starts. You don’t get a true tree structure because you can call different sub-routines from different parts of your program – almost like having a leaf shared by different branches. Now, this is not a problem when the methods are very simple, but when they start manipulating data, it starts getting complex again. Another problem is that in modern event-driven programming environments, there is no “root” of the program. The linearity has been broken. What you have is a graph that has many inter-connections (or dependencies). Once again, complexity rears its ugly head.

We have different techniques to manage this complexity. Object-oriented design (especially encapsulation), refactoring techniques and design patterns help us manage the increasing complexity of software applications. But some of the techniques seem counter-intuitive if you have never used them.

For example, a junior programmer can reduce some complexity in his code by isolating similar code and moving it into a method with parameters. This reduces the code and decreases the complexity. On the other hand, moving data and code into a new class seems to increase code. It seems to imply more code to understand and worry about.

But, of course, this is not true. The point is to move code into a place where it can be completely forgotten. You create the new class that can be tested on its own and then only worry about the code that interacts with it. To a large extent, we already do this by tapping into the framework classes and APIs.

I want to emphasize that this is a learnable skill. These are all well-documented techniques waiting for some investment of time and effort. As you learn them and use them more in your code, you will become familiar with how they work. Some of these are easier and/or more commonly used than others, so it may take a while to be an expert in them all. The point is to keep learning and keep becoming a better programmer.

Comments (4)

By Krishna, October 13, 2009

Ted Neward had a post wondering why we don’t see more small-scale applications being built with tools as it used to be in the past with FoxPro.

[...] many people I care about [...] made a healthy living off of building “line of business” applications in FoxPro, which Microsoft has now officially shut down. For those who did Office applications, Visual Basic for Applications has now been officially deprecated in favor of VSTO (Visual Studio Tools for Office), a set of libraries that are available for use by any .NET application language, and of course classic Visual Basic itself has been “brought into the fold” by making it a fully-fledged object-oriented language complete with XML literals and LINQ query capabilities.

Which means, if somebody working for a small school district in western Pennsylvania wants to build a simple application for tracking students’ attendance (rather than tracking it on paper anymore), what do they do?

I am in two minds about this. In agreement with Ted, it would be nice to crank out quick business applications using simple tools. In fact, many applications that are built using .NET or Java could be automated using Excel or Access, not to say even a Word document that is put in a shared folder! It is one of the characteristics of human beings that they will spend more money than they need for things that they could have lived without.

The problem, unfortunately, is that people also display the following characteristics:

• They get new ideas, i.e., when they see what software can do, they want it to do more.
• They change their mind about how things should work.
• They are not organized: they need the software to protect them from their own mistakes or provide them more information to help them out.
• They don’t understand the millions of man hours that have gone into fine-tuning the functionality in sophisticated off-the-shelf products and demand that in their own applications.

This is all apart from external business reasons for application changes. So there is no “simple” application because the moment it is built, people want to change it.

For the developer, there are 2 choices: Walk away after building it, or continue to make changes for the customer. But even in the first chance, there may be a warranty/guarantee period where the developer continues to be involved with the product. And these changes become more and more difficult with the tools that we had in the past.

There are many dimensions where the “simple” tools of the past made it more difficult. Consider deployment – it is infinitely easier to manage deployment on one web server than worry about many different Windows (or DOS!) machines. Consider simplicity – plain JavaScript versus JQuery. Consider separation of concerns in different architectural layers instead of code lying in the different layers, including the “business” layer and triggers.

There are many more tools today to learn, but they make it simpler, not harder, to code. Ted has a point that the standalone developer has to go through several weeks of learning. But if we are talking about college students, those 6 weeks is spread across a few years of study – not a big deal. But even if it is a person making a new career move into programming, there has to be some investment in educating oneself. Otherwise the programmer will end up fielding more and more support calls instead of moving from one project to another.

Also,  it is possible to build smaller applications using a limited set of present-day technologies. If you look at the 5-week list that Ted put up, it is plausible that a good programmer can write a small application without using or knowing much of them. For example, Joel Spolsky and Co. probably don’t use many of the tools in the .NET space or at least they hadn’t been using them during the Wasabi Era.

Finally, I don’t see it necessarily as a disaster that people have to rewrite software once in a while. Sometimes, a business can only afford to spend so much money building an application. If the business grows, maybe they will throw out what they have to and get something better. Just as they would buy a larger conference table or bigger coffee maker after throwing what they have into the dumpster. Sometimes, the smaller investment helps them get to the next level, where they have to spend more money.

Comments (4)

By Krishna, October 9, 2009

There are a lot of them, but I keep coming across one all the time. Here is an example (emphasis mine):

Brad Fitzpatrick, born in 1980, started to learn programming at the age of 5. In high school he went on to create a voting booth script called FreeVote, which he says earned him as much as 27 cent per click on banner ads back then (making for 25, 27 grand per month).

I touched my first computer (for playing chess) at the age of 14. I touched my first mouse (at a science fair) at the age of 19. I wrote my first C program at the age of 21. I have done a lot of programming since then, but the 16-year difference is a huge gap.

The reason is that my family and I didn’t have the money to buy a computer, nor did the schools and under-graduate colleges I studied have them for programming purposes. So I had to wait till my graduate studies. Those institutions have the right facilities today, but back in the 1980’s and 1990’s, India was just emerging from the lost years of the License Raj. The cohort I belong to share the same experience. We probably read a lot more books than a similar American demographic, but programming, not so much. (I remember reading a book on BASIC, but no computer to type it on.)

Time equals knowledge. A vast majority of the senior Indian programmers had their first experience with software development in college. While looking at reasons for the relative low proportion of software products coming out from India, this must be kept in mind. The Indian developers did not know enough programming to start a company or write great software during the early years of their youth, which are usually the most productive.

This is rapidly changing now. With the tremendous improvement in living standards over the past two decades, millions of Indian children are being raised in families that have “the” hardware! The gap in software innovation and entrepreneurship will surely decrease as these children grow up to become software engineers.

Comments (0)

By Krishna, October 8, 2009

Daniel Auger defines a new term:

The Legacy Programmer Boss is the manager who had a successful programming career in the past but sees many modern concepts as language baubles, academics, and anti-patterns because they are out of the loop. The fact that they once had a high degree of expertise gives them confidence in making mis-judgements.

This is a problem for every programmer who gets promoted to a role where they have to spend some portion of their time managing other programmers. Usually, the point is that the senior programmers is more experienced and knowledgeable than the other programmers. And so, it is a better use of their time to take on higher-level tasks (architecture, design) and provide guidance to the others.

So far, so good. But as time goes by, the other programmers also gain expertise. And they spend more time exploring and experimenting than the senior programmer. Partly because the latter has to be focused and specialized (though not always). The junior programmers also tend to be more receptive of newer ideas and possibilities.

When the senior programmer is faced with choices involving newer technologies, they find it difficult to understand its value. First, because they don’t know anything about them. Second, they have already finished successful projects without using them.

To take an extreme example, take a programmer back in the good old days when they used tape to record programs. She did not have the luxury of an IDE, multiple compile-run cycles, testing and so on. She had to get the program right on the first try because it would be hours or days before she could run it again. So, she would wonder why modern programmers need so many sophisticated tools when they could design the code before they start writing a line?

Or take an experienced C/C++ programmer wondering what the big deal is with programmers complaining about pointers, memory management and multiple inheritance. Or a Java programmer amazed at why some developers are obsessed with syntactic sugar. And on and on – take your pick in newer languages, tools, methodologies, components, etc.

Everyone is prone to this syndrome. Something worked for you in the past. So without even thinking, you tend to dismiss newer developments. The only antidote is to keep up with the new stuff through software communities (real and virtual), reading books and blogs and so on. More variety and more exposure will reduce incidences of acting like a legacy programmer boss.

Comments (0)

By Krishna, October 7, 2009

A lot of the brouhaha over the duct-tape controversy reminded me of “The Innovator’s Dilemma” which I have mentioned a few times on this blog, but never really elaborated upon. Well, it is like this. People wonder why companies which are market leaders in a segment fail to see competitors surging ahead with better technologies. Usually, people attribute this to arrogance and pigheadedness of the management in those companies.

Clayton Christensen’s research tells a different story. It is not mismanagement, but good management that lead to successful companies being toppled. Market leaders use good management techniques to discover and promote products that have a probability of better market acceptance and hence good sales and profits. While doing so, they discard many products or technologies that do not look promising. While their determination is usually correct, sometimes one of these discards manages to become a better product and surpass the market leader.

The timing here is the critical piece. When a technology is rejected by the market leader, it is inferior to the current technology used by that company. If it wasn’t, it would be poor judgment to reject it. In fact, several such inferior technologies are rejected. And rightly so, because it is impossible to predict which would improve in the future. When you hear about some startup rejected by some company and then going on to make billions, you have to take it in the context of hundreds of such startups that were also rejected and ended up in the deadpool.

Christensen calls the upstart technologies “disruptive” as opposed to “sustaining” technologies which are embraced by the market leader because they have a greater potential for faster results and profits. “Disruptive” technologies may seem primitive or difficult compared to present technologies and although they may have promise, no one is sure if that promise will materialize.

So what does this have to do with coding?

Well, when it comes to coding, there are some tactics that produce instant results. Better hardware, better software tools (IDEs, refactoring tools, etc.), adding another tester, reducing programmer distractions, some refactoring methods, etc. are some examples. Whether you have the money or circumstances to do these is beside the point. No one argues against them theoretically.

But there are other steps that are not so straightforward. One is buying or reusing a component instead of writing it. There is the possibility that the tool may not satisfy some important cases and then you have to rewrite it or write some workarounds. Another example is adopting a newer technology or methodology that is not well understood in the industry or even in your company.

In the second scenario, the right short-term management decision is to stick with what you know and get the job done. This is good management because it avoids risk and ensures that company projections are met.

But, if the company continues to operate in this mode, the sequence of correct short-term management decisions will end up with a long-term disaster for the company. Yes, many rights will create a wrong. This is a case where you need someone to provide long-term leadership.

Here is the long-term problem. Other companies will embrace the newer technologies and methodologies, which might get better over time, and they will use that advantage to leap-frog over your company. Your company will be stuck polishing its skills in the older technologies that no one wants. Even hiring will be difficult because employees do not want to work in such older technologies.

Take a simple example. Let’s say your company knew plain old ASP and continued to build applications in it even though ASP.NET came into the market. In the short-term, it was easier for your employees to build an ASP application than learn ASP.NET and build one, perhaps committing some mistakes that required costly trouble-shooting. In the long-run, your competitors are building similar applications faster with higher quality. Your employees are leaving for those companies and you cannot hire anybody who wants to work with ASP.

Now substitute ASP.NET with all the latest “architecture astronaut” stuff that Spolsky derides.

True leadership would be to set apart time, money and resources to investigate new technologies and ideas constantly. Build pilots and understand migration paths to newer technologies. Learn about the communities around them and how you can get help to obtain workarounds (because there always will be a need for them). Understand what vendors, customers and employees think about where they want to go.

As I talked about before, if someone is showing up at your door suggesting some new idea, the worst thing you can do is write a blog post mocking them. They are telling you something that may be valuable. At the very least, you must invest some time to understand what in the world they are talking about and whether it may make sense for you.

Otherwise, projects tend to become like abandoned houses with weeds growing over the walls. Even if you do nothing, they become outdated and impossible to work with. I know people who are working with technologies that are decades old because there was a good risk-averse manager who couldn’t provide leadership for the long-term.

Comments (0)

By Krishna, September 22, 2009

Programmers and managers are obsessed with finding the best way to solve a particular challenge. This is a good thing because that is what quality is all about. Sometimes, though, it can be taken to extremes and the project gets stuck in discussions of minutiae.

I was reminded about this when I read about the Chandler project in Scott Rosenberg’s “Dreaming in Code“. Much of the book is about the innumerable discussions that the team members had about requirements and design which lead to the project being at standstill for significant periods of time. Also how they kept second-guessing themselves and changing decisions in mid-stream.

This is not unique to Project Chandler. Projects go through this phase where the programmers need to understand their end goals and how they plan to get there. The question is how much time do you plan to spend on it and how soon you can make decisions to get on with the construction phase? Where is the point at which you have diminishing returns from further analysis?

I wrote a post sometime back about the pitfalls of soft coding, which is writing an application in such a way that the user can configure its features ad infinitum. Soft coding exists, in part, because the team could not make a decision about what the user wants. Instead, the project team kicks the decision to the user’s court.

You can also see this in design, where there is a lot of hand-wringing about writing loosely coupled code. The idea is that you should be able to remove any class or library at any time, plug in a replacement and just have to change one line in a configuration file without compiling. So much engineering effort goes into writing such maintainable code, when in reality, most of the code will never be changed again in the lifetime of the product. And most of such decoupling is quite unnecessary because it could easily be substituted by someone taking two minutes to change the code.

Now, you may think my previous paragraph is stupid. How dare I promote tightly coupled software? But there lies the problem. Many programmers have been bitten by bad code written by bad developers that they have gone into over-engineering mode. Instead of having to change code, let’s make it so that the users can dynamically change the behavior of the software. Instead of having to recompile code, let’s make it so that some administrator can change an XML file.

Sometimes, I feel that programmers are stuck creating compilers when they should be creating applications. Compiler, as in, give users the ability to specify in minute detail how the application should behave at run-time and allow them to change their minds whenever they wish. Whatever else an application may do, you can guarantee that the “Settings” module of the application will grow and grow, as programmers delegate decisions to users.

This tendency to favor configuration over convention is perhaps due to a fear of being wrong. It is less risky to allow users to make the choice than find out what the more intuitive way of doing something is and implement that. So keep on building indirection upon indirection.

We need to break out of this cycle. Keep the application as simple as possible. Follow best practices, but don’t beat them to death. Start coding sooner. Build prototypes to validate your requirements. Build pilots to validate your design approach. Channel user feedback into refactoring the application. But just don’t sit there re-examining your decisions and trying to come up with the perfect design.

Comments (0)

By Krishna, September 12, 2009

A significant part of software literature is devoted to stories of software failures. The best classics of our industry like Fred Brooks’s “The Mythical Man-Month” are based on the analysis of the failures of software projects. Software engineering was created as a way to reduce the number of abandoned projects and keep budgets and timelines under control. In recent decades, there has also been a focus on difficult user interfaces in desktop applications and web sites.

Introspection is good for any profession and there is still much scope for improvement in the software field. But we have come a long way in the last several years and it is time to look at the progress that we have made.

First, software development has never been easier and of greater quality. This has been the result of many important developments such as the sophistication and maturity of software development environments (compilers, IDEs, refactoring tools, testing tools), widespread use of object-oriented programming, automatic garbage collection, virtual machines, availability of high-quality open source software and more.

Second, Agile practices have moved us away from the old metrics used to measure success. We have smaller, but more frequent development targets that have made it easier for meeting them. In other words, our approach to building software has moved away from big-bang construction to incremental, achievable goals. Our failures are fewer and because they are smaller, they are simpler to correct.

Third, we have been able to take care of much of the “low-hanging fruit” in usability problems. Software applications and websites have become much more intuitive. We will continue to have innovation in user interfaces, but usability problems are fewer today. Take any 1990s book about UI problems – many of those issues have disappeared today.

The problems that we see today are less specific to software development. Software failures are increasingly “business” failures. By that, I mean, software is developed on time and under budget, but it is not what the market wants. The iPhone App store has 50,000 applications: many will disappear because of insufficient marketing. Many online web applications will fail because of competition, not because they were poorly developed. More and more, software development seems to resemble manufacturing: the more resources you can spend on marketing and production, the better the outcome will be.

So, like other disciplines, the technical specifics of software development will become only one pillar in the operation of software firms. We will operate at a high level where the biggest challenge will be understanding the needs of our customers and marketing to them rather than how to write the software without bugs. The scope for improving coding practices will always be there, but it will be a sideshow.

Comments (0)

By Krishna, August 26, 2009

Guy Kawasaki had a series of posts where he called out various types of people (venture capitalists, engineers, marketers, etc.) on untrue things that they are used to saying. There are situations when using a tough word is appropriate, but this is one case where Kawasaki is just wrong. Most people are truthful most of the time. Good business sense demands that you trust them, because if you are always watching out for lies from people, you are in deep trouble.

Now I understand the possible Digg-bait nature of the title. Also perhaps, one may give Kawasaki the benefit of the doubt in that he was talking about “white lies” and attributing incompetence rather than malevolence to each category of people. And that seems to be the case with the set of engineer lies, where Kawasaki says that he has a soft spot for engineers, but you shouldn’t really take them at their word.

I find the list a bit disturbing because it is a clear case of management divorced from what is happening in engineering. Every project requires good project management, otherwise it will only end in problem-hiding and finger-pointing. It is not about assigning a *project manager*, but whether the team has a grasp on the what, when and how of the project at all times.

Let’s take each of the points Kawasaki mentions (Do read the entire post so that you understand his argument):

1. We’re about to go into final testing: If this is a lie, it means that there was no project plan. It also means that there were no regular project releases so that incremental testing could be done and pertinent user feedback rolled back into the product.
2. Even my mother can use it: That is why you need testing by end users, as early in the project as possible. And it can be done on the cheap too.
3. Our beta sites really love the product: If this is a lie, you need to choose a better set of beta sites and people to test the product, and find a better way of getting honest feedback. Whether users will pay for the software is directly related to how much they love the product. How you make money off it is a question for the business owner and the marketing/sales folks, not the developer of your organization.
4. We’d save time if we’d just start the whole project over from scratch: This is probably true in some cases, but the most relevant point is that you have an engineer with the wrong skills. Usually, the engineer who said this does not have an idea of the technology, language or framework used and they express this sentiment because they would be more productive in their specialized environment. Spend time and money to find someone who has worked in the same environment. And if you find such a person and they still say the same thing, then your existing code base really sucks and you probably should start the whole project over from scratch.
5. This time we got it right: I am puzzled why Kawasaki thinks this is a problem. If this is a lie, you have serious trust issues with the engineer.
6. It works on my computer: OK, I grant this. I will probably write a longer post about this point, but it is a more complicated issue than it seems at first glance. The application probably works fine on more than “one computer in the world” and the developer is tired after being through many situations where the user has failed to follow some simple explicitly-stated instructions. One solution is to have customer support people field the calls from non-computer-savvy people first and let through only complex issues to developers.
7. We have great testers and a great bug-tracking system: I suppose this lie is coming from an engineer at a startup hyping stuff to an entrepreneur, because I cannot see that happening inside a company. I would hardly think that this is a common lie even in that situation, but Kawasaki should know better with his experience in dealing with startups. In any case, once again, you need to have testing early in the game, and by testers who are independent of the developers. I believe the technical term is “due diligence”.
8. I don’t know much about sales or marketing: I don’t know what point Kawasaki is trying to make here, but why would you care whether your engineer thinks sales and marketing are easy? I see jokes all the time about programmers and managers, but that doesn’t prevent them from doing their jobs.
9. What I’m doing is scalable: If the engineer is clearly junior and incompetent to answer this question, why was it asked in the first place?
10. This will take five minutes to fix: I grant this too. But this will only be a problem if you don’t have a good testing regiment. If you have frequent builds and you have a well-defined build-test-release cycle, you won’t get this answer.

Most of these “lies” seem to stem from lack of proper testing. If you release frequently and have independent testers, things will usually not get to the stage where management loses trust in the engineers. On the contrary, it will build confidence as new features are rolled out and the system becomes more stable as bugs are fixed. A regularly updated plan would also be good.

Comments (0)

By Krishna, August 25, 2009

Alan Skorkin has a good post about how people tend to use concepts like YAGNI and TDD to justify whatever they are doing, instead of thinking about what they are doing:

Every practice you use, including TDD and YAGNI has a certain threshold which I call the stupidity threshold. While you’re using the practice as a tool giving due consideration to all other concerns, you’re fine. But, as soon as the use of a particular practice becomes an objective in and of itself, you have crossed the practice stupidity threshold and will find yourself in trouble eventually.

Obviously, this sort of behavior is not unique to Agile practices. Take any methodology (in software development or business management) and you will find “true believers” who seem to think that some principle is infallible, even though it was invented or discovered by some human being(s) who are nowhere near perfection.

An amazing thing that I am seeing in the software industry right now is the whole-hearted acceptance of Agile practices by organizations who were previously married to rigid methodologies. Part of it is that Agile has gone mainstream by presenting its practices in the wine bottles that decision-making executives prefer: certifications, expensive consultants, great-sounding names for Agile practices (like Scrum), project management software (with bar graphs and all!)

This is both good and bad in a way. The good thing is that there is no more need to convince people of the virtues of Agile. The problem is that a lot of people (and I mean, A LOT) are coming to Agile with many ideas and attitudes from their current way of working. Prominent among these attitudes is the slavish adherence to processes.

Quality departments in organizations have what is called a “non-conformance”, a red flag raised when some person or department is not following a process correctly. Such “non-conformance” items do not look good when a group is evaluated, and so people try to minimize them by trying to follow every process to the letter, even if it makes no sense in a particular situation. They want to avoid having to fill in all the paperwork to justify any deviation from the agreed-upon process.

So what happens when you bring that mentality to Agile? All you have done is replace one set of processes with another set, and failed to understand the need to provide good business value to customers by building quality software. This behavior is not the exclusive domain of large organizations. Even a small team can fall into this trap by thinking in terms of processes instead of thinking of the overall goals.

The merit of Agile practices is that very talented people have thought about them and found them better to existing practices. But that never means you should leave your brain in the fridge when you start using them in your organization. Use data from your experiences to decide what is working, what should be tweaked and what should be abandoned. Read more case studies of Agile use in other companies and use what may be applicable to you.