Comments (0)

By Krishna, March 8, 2010

Stephen Forte makes what he thinks is a case against having “rock-star” developers in a team. You can read the post, but, in short, there was a programmer “John” on his team who picked coding arguments with the rest of the team. The final straw was that John burnt the whole team by directly moving his changes to production without going through the QA process and bringing down the site.

Stephen’s example is not very strong. A great programmer, by definition, should be someone who also understands and advocates good build and deployment practices. John was not such a person. It is also quite possible that a lower-grade programmer could have made the same error if they were under pressure from some management big-shot. You have to blame the incentives here.

In any case, these kind of train wrecks are easily preventable. An analogous example is locking the door if you are in the bathroom. Yes, people should knock before they try to enter it. But you cannot force everyone to follow such “best practices”, especially what is foremost in their mind is not the prospect of causing embarrassment to another person. Instead, what you can do is prevent someone from causing damage if it is really important to you.

So with respect to production servers, the simplest solution is to lock down access and grant it only to people who have a stake in following the process. This small group usually includes the system administrator responsible for the box, the project manager (or team leader, as the case may be) and the QA manager. All others, especially programmers, are more motivated to say that the work is done than to prevent any damage because of a mistake.

I am actually not in favor of providing access to the lead programmer, either, even if they are very experienced. The path should go through QA, who test the build or patch on a Test server that is as similar to the production server in terms of capability, environment and data as possible. One QA person is in charge of delivery and they sign off on releasing the build.

In a smaller organization, there will some overlap between roles, but do ensure that the person in charge of QA has the responsibility and sole authority for releasing code to production. That person should be independent of any influence by managers to speed up delivery of projects before adequate testing is done.

The other step is to be absolutely uncompromising about crossing management lines. There has to be one person (whatever the title) who has the authority and responsibility for accepting tasks for the development team. I am using those 2 keywords again, but what it means that only one person can accept a new task from someone outside the team, and once they accept it, they are fully responsible for it. This person maintains the backlog and if any manager wants to circumvent the hierarchy and talk directly to a developer, they will own it including breaking the schedule of other managers, crashing the servers and so on. And also, perhaps most important to them, that the task would be dropped without notice once the backlog manager learns about it.

Now, Stephen defines a rock star developer as “someone who, while talented, thinks that they are the ultimate guru and that everything should be done their way.” As I see it, you have hired a manager when all you needed was a programmer who can follow instructions. There was already a manager (you!) on the team. John doesn’t belong. His personality is such that he should be leading the team and swimming or sinking with the decisions he makes. But he probably doesn’t have the necessary resume to land that role in any company yet. In the meantime, he is a nightmare on the team because he tries to manage his peers. That is the real problem.

The difficulty in avoiding such hires is that during the hiring process, the rock star developer is sometimes indistinguishable from the talented, easy-going developer. Both are bursting with energy and ideas, some of which are actually pretty good too. You only learn the full extent of their behavior after a few weeks. The easy-going developer is comfortable with accepting authority and there is a give-and-take with respect to ideas. The rock star developer doesn’t understand why compromises are sometimes necessary and is quick to condemn differences in opinion as stupidity or evil. We all do the same to some extent, but the rock star developer’s condition is almost pathological.

The obvious route is to let the developer go, but if you don’t have a choice (in the short term), then one way to handle them is to divert them into tasks that require heavy lifting in research and development. They are now isolated from the rest of the team and the day-to-day grind, but they are kept extremely busy because they have to deliver something that works. This is something that I suspect rock star developers like, because they get to play the full court from analysis to recommendation. However, ensure that you have set the right parameters for accepting the outcome of the research so that you don’t get a recommendation that you will be forced to reject, thus exacerbating the problem.

Comments (0)

By Krishna, January 28, 2010

I fail to comprehend how someone can write this (emphasis mine):

Overdesign is also very common with databases. People get hung up on rules like “third normal form”. Sometimes when you over-normalize a database is [sic] makes it impossible to query the data later on. If your application is going to require reports, sometimes its better to design the database with the reports in mind. In other words, let the reports dictate how the data is stored.

This kind of advice is insane, ridiculous stuff. Take a minute and read what the third normal form means. Not following the third normal form means essentially having duplicate data in different places in your database creating a serious nightmare in maintaining data integrity. What you gain in reporting, you will lose when you start writing code to ensure that data is consistent everywhere.

Normalizing a database does not make it “impossible” to query data. If you have many tables because of normalization, work is harder because you need multiple joins to do the trick. But if you know elementary SQL, you can get it done easily. On the other hand, if you haven’t normalized properly, you have very silly problems. For example, in the Wikipedia example, to find out how many players were born on the 1st of January, you have to write code that does not count the birthdays of multiple players twice. In a normalized database, it is an extremely simple query.

It is ironic that the author talks about “overdesign” while falling victim to premature optimization. First, understand what normalization is supposed to do and how it helps you. Learn SQL or use tools that can help you get your data out easily. Profile your application and find ways to improve performance. And only if everything fails, then go for denormalization. And that too, only after understanding its downsides and how to prevent them.

Read my previous article on the same subject.

Comments (0)

By Krishna, November 29, 2009

Anecdotes are dangerous. Scott Koon has one about fixing a doorknob, linking it to the now-infamous Spolksy duct-tape programmer post and writes:

Think about that the next time you reject a new programming tool because you think it might take too long to learn or it’s different. Instead of doing the hacky way you KNOW will work, check to see if there is a more elegant or permanent solution. Because if you aren’t careful, your cheap and easy workaround might end up sticking around in the project long enough for it to become a problem to you.

People keep missing the point. There is a situation where we have bad programmers who use whatever tools they know to get things done. Everything about that situation is wrong because you have bad code – lengthy, mangled, unmaintainable. There is no argument about what should be done to those developers.

The other situation (and what the real argument is about) is the case of an experienced developer who has been successful in the past and who would rather use what is working for them than use something they are unfamiliar with. Because the tool has been successful for them, they do not accept that it may become a problem in the future. An example could be as simple and recent as an ASP.NET developer choosing to continue with WebForms than use the (arguably better) MVC framework.

Choosing to dismiss an experienced developer’s older tool as a “cheap and easy workaround” is a misunderstanding of the situation. The old way of doing things may have been the gold standard a few years/months ago. The new tool may work better, but only for a person who has spent enough time learning it to master its nuances. And there may be many such new tools, each of them making extravagant promises. So we also have the problem of making the right choice.

In a sense, this is the classic Technology Adoption Lifecycle problem, having to do with how fast people accept new technologies.

Technology Adoption Lifecycle (CC Wikipedia)

This was for firms using a particular technology, but it could apply to programmers moving to new technologies. When do they think a particular technology is good enough for them? Or when the current technology is unsustainable for them?

Comments (1)

By Krishna, November 27, 2009

Brilliant post on testing by Michael Bolton (emphasis mine):

Bug Investigation and Reporting (time spent on tests that find bugs)Test Design and Execution (time spent on tests that don’t find bugs)

Module	Time spent on tests that find bugs	Time spent on tests that don’t find bugs	Total Tests
A	0 minutes (no bugs found)	90 minutes (45 tests)	45
B	10 minutes (1 test, 1 bug)	80 minutes (40 tests)	41
C	80 minutes (8 tests, 8 bugs)	10 minutes (5 tests)	13

[...]If we are being measured based on the number of bugs we find (exactly the sort of measurement that will be taken by managers who don’t understand testing), Team A makes us look awful—we’re not finding any bugs in their stuff. Meanwhile, Team C makes us look great in the eyes of management. We’re finding lots of bugs! That’s good! How could that be bad?

On the other hand, if we’re being measured based on the test coverage we obtain in a day (which is exactly the sort of measurement that will be taken by managers who count test cases; that is, managers who probably have an even more damaging model of testing than the managers in the last paragraph), Team C makes us look terrible. “You’re not getting enough done! You could have performed 45 test cases today on Module C, and you’ve only done 13!”

And yet, remember that in our scenario we started with the assumption that, no matter what the module, we always find a problem if there’s one there. That is, there’s no difference between the testers or the testing for each of the three modules; it’s solely the condition of the product that makes all the difference.

The obvious larger lesson here is that if you are going to use metrics, comparing numbers is not going to do anything. You have to be willing to spend the time to understand what is going on. Too often, managers use metrics as a club to hit employees on productivity, something that is enormously counter-productive. As employees find a way to make the metrics towards what the manager wants instead of the needs of the project.

This is also a case for collecting fewer metrics so that you can spend greater time analyzing them. For example, A looks pretty good on test coverage, but there might be the possibility that we haven’t written some necessary tests. Or now that we know C is pretty buggy, we may need to look at the development process – what stage these bugs are getting introduced and why?

In several recent posts, I have talked about the quality versus productivity issue in mature software organizations. Since deadlines loom larger, they are often given higher priority by developers and testers at the expense of quality. And so, managers have to make a special effort to emphasize quality and avoiding shortcuts. Creating productivity metrics swings the pendulum in the other direction. It introduces pressure to get more done instead of getting things done right. So instead, when you evaluate these kind of metrics, stop looking at “finding lots of bugs” or “getting enough done”, but instead at the direction the product quality is taking.

Comments (2)

By Krishna, November 22, 2009

Dave Nicolette asks about C# versus Java:

Consider two hypothetical programming languages such that:

Language A

• Supports closures, and
• Can only run on one platform, so that customers who use the language for mission-critical apps are locked into a single operating system vendor.

Language B

• Doesn’t support closures, and
• Runs on all mainstream platforms, so that customers who use the language have flexibility to change their technical environment as their business needs change or as the relative cost of ownership of different platforms changes.

Which of the two programming languages is "better?"

We saw the same failed argument about Java on the desktop a decade ago. Write once, run anywhere. One reason it didn’t work was the dominance of Windows. You were better served with just using the most productive programming platform for Windows, whichever that was or whichever you preferred.

Back in the 1990s and early 2000s, when we had a plethora of choices of web servers, application servers and databases, using Java as a multi-platform language made sense. Today, most Java development is coalescing around Linux, Apache, mySQL and Tomcat/JBoss, making it a de-facto standard.

An experienced Java developer may have had the experience of porting from JRun to WebLogic, or from Sybase to Oracle. But if you are starting today to develop a new application on the aforementioned stack, is such a move likely? All the tools are open-source thus avoiding a need to reduce costs. They are also proven to be scalable, as many high-traffic sites can testify.

Metrics about platform changes would be helpful here, but if they bear out what I say, there is no actual cross-platform advantage offered by Java when you are not changing platforms. You may have other arguments for choosing Java, but this is not one.

Comments (2)

By Krishna, November 17, 2009

Zef Hemel writes about the inscrutable error messages that he gets when making typos in Ruby on Rails, Scala and JBoss Seam. I recently saw a new programmer struggle with the RoR errors and I have been a victim of some of the problems that occur when you type a string wrong in Java. For those who have had the same experience, it can be deeply frustrating to realize that you spent a whole day debugging a typo while managing only to spend minutes on the application logic.

Looking at some of the comments on his posts (especially the RoR post), a few people suggest that Zef would not be blocked if he knew what he was doing. This is true for the reason that once you see an error and know how to fix it, you will know what to do in the future. But it misses the point about what people new to the framework would feel when they use it. If a tool or framework cannot attract new followers easily, it will remain a niche.

The biggest problem with many useful open source and commercial software development tools is precisely this. The creators assume that since the target audience is the programming community, they can get away with unfriendly user interfaces that would not be tolerated in any product meant for general audiences. Or to be accurate, they don’t actively assume this, they don’t even realize that this is an issue.

Here is the crux. Programmers, senior or junior, have limited time to experiment with new tools. They are already productive with the tools they know and so wish to see the same ease-of-use and speed in new tools that they would like to add to their repertoire. Programmers learn by downloading the tool, installing it and trying out a few examples. If they can get something to run and can tinker with it, 90% of the struggle is over. They can keep experimenting and becoming a master of the tool.

On the other hand, if the programmer, after downloading the tool, sees a meaningless error message and then has to spend an hour Googling the term and tracing the error, then they will ditch the tool. Sooner if the tool has a tendency to keep throwing block after block.

There is a condescending school of thought that suggests that good programmers should be able to overcome any hurdles and it is better that the tool is complex enough to only attract the best in the business and maintain a community of experts. This, of course, assumes that good programmers want to spend days learning a non user-friendly tool that has a small user base and more likely to fade out with that kind of philosophy.

What can be done about it? I think there is a business model in here somewhere for a company to create more user-friendly and integrated versions of products. Say, an installation that bundles different products (say Eclipse with JBoss Seam and mySQL), and has a central administrative interface to configure and trouble-shoot. Or an IDE add-in that analyzes the code for possible typos and provides alerts.

Maybe somebody should create a search/answer engine for errors and exceptions. When you land on the answer page, it will provide the different possible scenarios that could have caused the error message, along with their probabilities. It could list the steps that you could take to identify the error.

Comments (0)

By Krishna, November 15, 2009

Jon Skeet has a long post on how programmers fall prey to pretty nasty bugs because they aren’t aware of the details of what they are doing. And he offers the following thoughts:

First, try not to take on more complexity than you need. If you can absolutely guarantee that you won’t need to translate your app, it’ll make your life a lot easier. If you don’t need to deal with different time zones, you can rejoice. Of course, if you write a lot of code under a set of assumptions which then changes, you’re in trouble… but quite often you can take the “You ain’t gonna need it” approach.

Next, learn just enough about the problem space so that you know more than your application’s requirements. You don’t need to know everything about Unicode – but you need to be aware of which corner cases might affect your application. You don’t need to know everything about how denormal number representation, but you may well need to know how rounding should be applied in your reports. If your knowledge is just a bit bigger than the code you need to write, you should be able to be reasonably comfortable.

Pick the right platforms and libraries. Yes, there are some crummy frameworks around. There are also some good ones. What’s the canonical answer to almost any question about java.util.Calendar? Use Joda Time instead. There are similar libraries like ICU – written by genuine experts in these thorny areas. The difference a good library can make is absolutely enormous.

To add to this, I would say that it is very important to enroll in a computer science degree in college before embarking on a programming career. I know there are very good programmers who have learnt everything on their own. But for the general population, it is a good idea to be fully involved in 2-to-4 year degree that provides you some basics, such as what floating-point representation means. Of course, you still have to do the learning, but your physical presence in a classroom is a higher probability that you may learn something.

While recruiting new employees, I always take a look at their academic scores. Although a good academic record does not guarantee a good programmer, a bad record invariably points to a poor software developer, with very few exceptions. Apart from the obvious IQ reason, here are some of the ways why this is so.

1. Getting things done: All other things being equal, a person who pays attention to end results is more valuable. If a person has high IQ, but doesn’t translate that to results, it usually means that they don’t care about delivering results. They may like learning and tinkering, but they will not deliver.
2. Lack of in-depth knowledge: An exam is not the perfect way to measure knowledge and application of knowledge, but it is among the best systems we have. If someone scores a passing grade on an operating systems course, they are probably missing a lot of important information.
3. Problems with authority: Innovation and breaking rules are important, but you also need some level of conformance to the goals of the organization. Poor scorers are sometimes rebels. They don’t feel the need to impress anyone (parents, teachers, etc.) or sticking to schedules.
4. Lack of perseverance: Some people simply lack the determination to tackle tough subjects. They may do well in a controlled setting, but when faced with challenges, they may need a lot of attention.

An example of an exception may be that somebody had a medical or family problem that affected their grades. Or they had a temporary blip that they got over. As always, you have to consider the whole picture. But definitely, academics should be a big part of your hiring process.

Comments (1)

By Krishna, October 21, 2009

Rob Conery pointed out an interesting comment on his IoC post:

In this simplified example above you switched from 3 lines of easily understandable code to something that requires code in the global.asax, another couple classes, and an interface. The end result contains an extra line or two of code, but now the whole thing is “reusable” and can be unit tested. I happen to agree with Joel’s quote which you openly mocked… it *is* more convoluted. It *is* harder to read and grok. And this was an overly simple example… I’m sure it gets more convoluted with large projects.

I am going to ignore IoC for a minute and take this in a general context. Some junior programmers have a tendency to write lengthy code instead of splitting the code into different classes and methods. The code, in itself, is not bad. I am not talking about poor variable names, bad formatting, copy-and-paste or low level silliness. But it is structurally deficient in the sense that

1. Each method will have many lines of code doing different things. In other words, the method has multiple responsibilities.
2. The coding is algorithmic, i.e., although they are coding in an object-oriented language, the coding is very linear in terms of execution.

I used to have a different perspective about people who did this kind of coding, but over time, I have concluded that this has got to do with the (learnable) ability of programmers to deal with complex systems consisting of different building blocks.

Junior programmers need to see, feel and know the entire program. They feel lost and confused otherwise. This is only possible if the code is one place and it has a very linear structure. This is enough for simple programs. But more complex programs require more code.

Well, it happens that a tree-like structure of code is also easy to understand because it is a easy progression from a linear code structure. You start from the root (the main method), navigate linearly, traverse each branch (if you come upon one), re-trace your steps and keep going until you are done. If the tree structure is not deep, you don’t have to keep too many things in your head and the complexity is managed.

The tree-structure is achieved by moving chunks of code into different sub-routines. But here is where the problem starts. You don’t get a true tree structure because you can call different sub-routines from different parts of your program – almost like having a leaf shared by different branches. Now, this is not a problem when the methods are very simple, but when they start manipulating data, it starts getting complex again. Another problem is that in modern event-driven programming environments, there is no “root” of the program. The linearity has been broken. What you have is a graph that has many inter-connections (or dependencies). Once again, complexity rears its ugly head.

We have different techniques to manage this complexity. Object-oriented design (especially encapsulation), refactoring techniques and design patterns help us manage the increasing complexity of software applications. But some of the techniques seem counter-intuitive if you have never used them.

For example, a junior programmer can reduce some complexity in his code by isolating similar code and moving it into a method with parameters. This reduces the code and decreases the complexity. On the other hand, moving data and code into a new class seems to increase code. It seems to imply more code to understand and worry about.

But, of course, this is not true. The point is to move code into a place where it can be completely forgotten. You create the new class that can be tested on its own and then only worry about the code that interacts with it. To a large extent, we already do this by tapping into the framework classes and APIs.

I want to emphasize that this is a learnable skill. These are all well-documented techniques waiting for some investment of time and effort. As you learn them and use them more in your code, you will become familiar with how they work. Some of these are easier and/or more commonly used than others, so it may take a while to be an expert in them all. The point is to keep learning and keep becoming a better programmer.

Comments (2)

By Krishna, October 16, 2009

In short, no.

But why am I asking the question in the first place?

Well, much discussion in the technical blogs (such as the NOOP.NL Top 200) is centered around the idea about whether a mediocre programmer will land on a blog where somebody is talking about adopting a simpler approach to programming and then use that idea to stop improving himself and continue writing bad code.

It is a useless subject to argue about because bad programmers do not haunt technical blogs. This is how such a person works:

1. Come in at 9 am or whatever time you are supposed to come in.
2. Find out what task to do.
3. Is the task something that looks familiar? If so, go to Step 7.
4. Interrupt someone in the office. Do they know the answer? If so, go to Step 7.
5. Google for the answer. Is there anything there that works? If so, go to Step 7.
6. Complain to the project manager that the task cannot be done and get something that can be done.
7. Do the task.
8. Is it time to leave? If so, go to Step 11.
9. Is there any other task? If so, go to Step 3
10. Kill time until it is time to leave.
11. Leave.
12. Enjoy the rest of the day.

What they don’t do is spend time visiting technical websites or blogs unless they happen to be landing on them via Google. And those Google visits are only for the purpose of finding some code that works. And issues like intellectual property rights be thrown to the winds.

So let us be clear about the audience when we are arguing about technical debt. We are having an argument among people who know how to write good code and how to test it properly. Some of them use a few more tools and a little more process than the rest, who are constrained either by their lack of knowledge, awareness or business situation to imitate them. No one among this crowd is writing bad code, as in copy-and-paste coding or bad variable names.

Therefore what we are really discussing about and what needs to be answered by people are:

1. You are a good developer who gets things done without Process X or Tool Y.
2. How much beneficial for you would it be to use X or Y?
3. How much time, effort and money do you need to invest for using X or Y to get started in and then be proficient?
4. What is the return on investment if you use X or Y?
5. Let’s understand if X or Y makes sense in your situation, including timelines, budgets, technical environment.

There are literally thousands of programming tools, techniques and processes out there, all claiming to increase productivity. So whatever the “best thing” you are promoting is, you have to justify it in these terms to the good programmers. And someone who cannot justify that is simply selling snake oil.

As for the other programmers, they are out there enjoying the day and definitely not dreaming in code.

Comments (2)

By Krishna, October 14, 2009

Martin Fowler clears up some of the confusion around the term “technical debt“:

The debt metaphor reminds us about the choices we can make with design flaws. The prudent debt to reach a release may not be worth paying down if the interest payments are sufficiently small – such as if it were in a rarely touched part of the code-base.

So the useful distinction isn’t between debt or non-debt, but between prudent and reckless debt. [...]

there [is] a difference between prudent and reckless debt, there’s also a difference between deliberate and inadvertent debt. The prudent debt example is deliberate because the team knows they are taking on a debt, and thus puts some thought as to whether the payoff for an earlier release is greater than the costs of paying it off. A team ignorant of design practices is taking on its reckless debt without even realizing how much hock it’s getting into.

It seems that Martin is suggesting that the best debt you should have is the prudent-inadvertent debt, where you only understand the technical debt after working with the application. It is not a case of recklessness where you decided to ignore design, or be content in your ignorance. Or a case where you weighed all the pros and cons and then decided for the quick-and-dirty approach. The prudent-inadvertent debt is something that happened despite your best intentions and efforts.

The way to explain this is that there are potentially different ways to design an application. For example, if you take the creational design patterns, you may have to choose between competing patterns. You may start using a Factory pattern, but then need to change that for something more flexible. Or you start out with a relational database design and then later find you have to denormalize or use a NoSQL database because the application became too successful. It is hard to predict the changes in requirements and sometimes, you have to make the decision using present day considerations.

There are always gray areas in those quadrants. Assume that you are building an online application. It is impossible to know ahead of time whether you will get 100 visitors per day or 10,000 or a million. You could choose to target a million, but that could be over-engineering the application. Maybe 1,000 seems a happy medium or maybe you are making a terrible mistake of under-estimating traffic. Making design decisions in these cases can seem reckless or prudent after the fact.

Perhaps the real mistake is to think of design as an independent process. Much is tied to business decisions and forecasts. And where your technical debt falls is determined by those business factors.