Comments (0)

By Krishna, January 28, 2010

I fail to comprehend how someone can write this (emphasis mine):

Overdesign is also very common with databases. People get hung up on rules like “third normal form”. Sometimes when you over-normalize a database is [sic] makes it impossible to query the data later on. If your application is going to require reports, sometimes its better to design the database with the reports in mind. In other words, let the reports dictate how the data is stored.

This kind of advice is insane, ridiculous stuff. Take a minute and read what the third normal form means. Not following the third normal form means essentially having duplicate data in different places in your database creating a serious nightmare in maintaining data integrity. What you gain in reporting, you will lose when you start writing code to ensure that data is consistent everywhere.

Normalizing a database does not make it “impossible” to query data. If you have many tables because of normalization, work is harder because you need multiple joins to do the trick. But if you know elementary SQL, you can get it done easily. On the other hand, if you haven’t normalized properly, you have very silly problems. For example, in the Wikipedia example, to find out how many players were born on the 1st of January, you have to write code that does not count the birthdays of multiple players twice. In a normalized database, it is an extremely simple query.

It is ironic that the author talks about “overdesign” while falling victim to premature optimization. First, understand what normalization is supposed to do and how it helps you. Learn SQL or use tools that can help you get your data out easily. Profile your application and find ways to improve performance. And only if everything fails, then go for denormalization. And that too, only after understanding its downsides and how to prevent them.

Read my previous article on the same subject.

Comments (2)

By Krishna, January 13, 2010

Phil Haack writes about the lack of confirmation dialogs when deleting an item in the Netflix queue.

I have noticed something similar on the Amazon wish lists. It avoids an unnecessary click, but at the same time, allows you to quickly undo the action if you had accidentally clicked the first time.

This reminds me of one of my pet peeves: people who complain about the Delete confirmation dialog box for files. Generally, the person who makes this complaint “cannot imagine” how stupid the operating system architects and developers are there. The user wants to delete something. You have a Recycle Bin. Why doesn’t the operating system just get out of the way? And who reads the dialog box anyway? One complainer suggested that this was just a way for the developer to shift responsibility onto the user. Since the user clicked Yes, the programmer is no longer responsible if the user accidentally made a mistake. And the implication is that the programmer is lazier and needs to own the responsibility. Or something of the sort.

Obviously, I have no idea what the intention of the programmers were. But let’s assume that there was no confirmation dialog box and the second you clicked Delete, the file is deleted. That is great if you meant to hit the Delete key. But what if you had meant to hit the “End” key (to navigate to the end of a list of files) and your stubby finger pressed down both the Delete and End key. What would probably happen is that the currently selected file gets deleted, you navigate to the end of the file list and you don’t realize that your file has just been zapped.

But what about the Trash can, you say? The problem is that if you don’t know the file has been deleted, you may not bother to check its contents before emptying it, especially if you had been deleting other files. You may not even realize the mistake until days or weeks have passed and then even a disk recovery service may not be able to help you.

I elaborate on this seemingly minor issue because although user interface issues are more complex than they seem at first glance, because they don’t seem so complex, everyone has an opinion. Even those who have no clue about user interaction or the design choices that made a choice necessary, even if it was not the most optimal. This was best illustrated by a recent comic “How a Web Design Goes Straight to Hell” by Oatmeal Comics. As the author notes,

I actually had a client include their mother in the design process so that she could provide feedback and criticism. [...] You are no longer a web designer. You are now a mouse cursor inside a graphics program which the client can control by speaking, emailing and instant messaging.

It is a tough world out there for user interface and user interaction designers.

Comments (0)

By Krishna, January 12, 2010

Bruce Schneier (the author of “Applied Cryptography” and other books) explains why security theater (where security measures do not do much to improve real security) has its benefits:

Tamper-resistant packaging for over-the-counter drugs started to appear in the ’80s, in response to some highly publicized poisonings. As a countermeasure, it’s largely security theater. It’s easy to poison many foods and over-the-counter medicines right through the seal — with a syringe, for example — or to open and replace the seal well enough that an unwary consumer won’t detect it. But in the ’80s, there was a widespread fear of random poisonings in over-the-counter medicines, and tamper-resistant packaging brought people’s perceptions of the risk more in line with the actual risk: minimal. [...]

We make smart security trade-offs — and by this I mean trade-offs for genuine security — when our feeling of security closely matches the reality. When the two are out of alignment, we get security wrong. Security theater is no substitute for security reality, but, used correctly, security theater can be a way of raising our feeling of security so that it more closely matches the reality of security. It makes us feel more secure handing our babies off to doctors and nurses, buying over-the-counter medicines and flying on airplanes — closer to how secure we should feel if we had all the facts and did the math correctly.

The relation to the business world is “management theater”. There are these tons of management processes that don’t do anything to improve the success of the project or prevent its failure. Instead, they are to reassure stakeholders that “proper” management is in place.

The lesson from Bruce’s article is that even though we may want to throw all of such processes out the window, they may actually have an use. Instead of customers becoming tense and worried, and upper management trying to micro-manage the project, some of these processes may convey that the project is truly under control and the project team can be trusted. It can also reduce finger-pointing and place emphasis on improving items that hinder the project.

The tricky thing is to know what process to keep in and what to take out. What Bruce’s article suggests is to ask stakeholders or understand them to see what keeps them comfortable. For example, they get project updates in a specific format at regular intervals. Or the build, testing and deployment process is done in a specific way and is visible to them. The customer may not even read the report, but the fact that they have it keeps them happy.

Comments (2)

By Krishna, January 4, 2010

A hilarious take from Ron Rosenbaum:

But don’t call it outside the box anymore, please. By now, the injunction to “think outside the box” has become inside the box airline-magazine management-guru cliché. Please, some of you should get back into the box, please, and take your quirky Power Points with you. What about thinking outside outside the box? Not outside the box but not inside the box again, either. Transcend the box.

I am sure some of you remember this classic puzzle. The first time you see the puzzle, learn the solution and hear the words “go outside the box”, it is very powerful. But after a few times, it sounds banal. And when you hear it from someone who is totally uncreative, you actively start hating the phrase.

Unfortunately, that is the fate of all such cute catchphrases or ideas. Most of the best ones have been ruined by overuse in popular culture. And others have been tainted by people who have forgotten  how human beings talk.

Things like, “Give your 110%“, “Work smarter, not harder“, etc. are good concepts, but are now hated management clichés because too many bad managers have used them to absolve themselves of their responsibility in managing projects better. This also goes for observations such as Parkinson’s Law and the Peter Principle. When they are misunderstood, they can be very harmful in the hands of a bad manager with too much time in his hand and very few brain cells in his head.

Incidentally, have you noticed that good managers tend to be the most productive and bad managers the most destructive when they are least busy? More time for good managers means that they have been able to delegate their tasks well and they have more time to spend at a higher level (learning, analysis and planning). A bad manager feels too insecure to be seen with free time and so they start micro-managing or implementing something without understanding it well.

Why do bad managers use hated catchphrases so frequently? I guess it has to do with intellectual laziness. Poor managers may work very long hours, but they all never seem to have any time to sit and evaluate whether they are doing the right thing. Using management speak makes them feel more important and wiser, giving them a false sense of control and correctness. Plus, the authority inherent in these phrases and ideas makes them very convenient to beat down ideas from the underlings.

On the same topic, some good stuff from Malcolm Gladwell (emphasis mine):

[...] “The Dark Side of Charisma.” [...] argued that flawed managers fall into three types. One is the High Likability Floater, who rises effortlessly in an organization because he never takes any difficult decisions or makes any enemies. Another is the Homme de Ressentiment, who seethes below the surface and plots against his enemies. The most interesting of the three is the Narcissist, whose energy and self-confidence and charm lead him inexorably up the corporate ladder. Narcissists are terrible managers. They resist accepting suggestions, thinking it will make them appear weak, and they don’t believe that others have anything useful to tell them.

Comments (3)

By Krishna, January 3, 2010

As I did last year and the year before that, I am here again with the list of best and worst books I read in 2009. To be clear, these are the books I read, not the books published in 2009. This year, I read over 90 books: Includes a high percentage of fiction, but does not include a few (bad) non-fiction books I returned before completing. So here goes:

Best Books I Read in 2009

• The Big Switch, by Nicholas Carr: How cloud computing will reshape the IT infrastructure of companies, big and small. Nick Carr compares this to the the transformation of companies in the last century from using internal power generators  to getting their power from a grid. He paints a scary picture of how cloud computing will result in greater unemployment and privacy intrusions. I don’t fully agree with all his predictions, but he raises some important concerns.
• The Drunkard’s Walk, by Leonard Mlodinow: A beautiful book on statistics – Who said math couldn’t be fun? At the same time, this is a serious book that exposes some common misconceptions about probabilities and expectations.
• The Business of Software, by Michael Cusumano: A great example of why you should read books instead of solely relying on piecemeal wisdom in articles. Cusumano gives us a wide overview of the software business and what companies need to do to survive and thrive in the long-term. Instead of only looking at the outliers (giants like Microsoft or Google), he uses case studies of small to mid-size companies to make his arguments.
• Code, by Charles Petzold: The author has written a beautiful book for beginners to computer hardware. Although I was familiar with the contents, it was still enjoyable to see how Petzold explained the concepts. If you want to start anyone on understanding computers, this is the book to hand them.
• Creative Capitalism, by Michael Kinsley, et. al.: A bunch of essays from Bill Gates, Warren Buffett and several economists about if and how capitalism can be leveraged to solve some of the greatest problems (poverty, disease, under-development) facing humankind in our age.

Honorable mentions go to Orbiting the Giant Hairball (Gordon MacKenzie), Buy-ology (Martin Lindstrom), Common Wealth (Jeffrey Sachs), and Why We Make Mistakes? (Joseph Hallinan).

My favorite author for 2009 is Scott Rosenberg, author of “Say Everything” and “Dreaming in Code“. Why don’t my favorite authors find a place in my favorite books? The answer is: why does the Best Director lose out on the Best Picture award?

Worst Books I Read in 2009

• The Inmates are Running the Asylum, by Alan Cooper: This is a seriously misguided book that makes programmers the scapegoats for every problem associated with software. Whatever good  points there are have been overshadowed by the incredible vitriol and hatred. Read my detailed take.
• Everything You Knew about CSS is Wrong, by Rachel Andrew, Kevin Yank: This is a special award for the most deceptive book title of the year. Instead of a detailed book about CSS, all it does is provide a different method for creating a grid in CSS.
• A Little History of the World, by E H Gombrich: An older book supposed to be a “classic” on world history, except for the fact that it doesn’t consider many historical events outside Europe. And the prejudice against other cultures and religions doesn’t help, either.
• The Creative Habit, by Twyla Tharp: A semi-autobiography disguised as a self-help book. The best I can say about this book is that you could pick up a few pointers about generating ideas after wading through the rest of the fluff.
• ENIAC, by Scott McCartney: Intrigue sells better than technology. At least, that seems to be the motto of the author who decided to highlight the politics and power games surrounding the development of the first electronic computer instead of elaborating upon the technical details and challenges.

Other disappointments included Donald Norman’s famous “The Design of Everyday Things” that turned out to be very outdated. I learnt that it is crazy to obsessively read all of Seth Godin’s past books as they seem to mostly say the same thing over and over again with different examples; it is perhaps time for him to write a full-fledged book. Same goes for Scott Adams: Dilbert just gets tiring after a while.

As for fiction, my hero, P G Wodehouse, turned out to have feet of clay. Having read several of his books last year, I can say that while Wodehouse has a few enormously hilarious books, it is clear that there was a marked decline (repetition and loose plots) towards the end of his career. The Albert Uderzo-only Asterix books are terrible as are the first two non-color Tintin books. And the less said about “Netherland”, the better.

Comments (3)

By Krishna, December 15, 2009

I am agnostic about pair programming. I believe that having strong programmers and a good code review process offers at least as much benefit as (if not more than) full-fledged pair programming, but if it is working for someone, I don’t see any reason to object. Researching which brought me to Obie Fernandez’s strange self-congratulatory article on pair programming.

Apart from the evidence-less accusations about what “most” software developers, managers and firms seem to do, it also has some silly ideas about hiring good developers. Read:

The best way to hire talent is to drop them in actual work situations for at least a week and see what happens. [...] Our candidates spend their interview week rotating on actual production code, pairing with the same people they’ll be working with if they’re hired. Everyone gets a say in whether to hire a new recruit, and hesitance from a developer that actually paired with them means they do not get hired.

I don’t understand in what universe a talented programmer is going to take a week’s worth of leave to work as part of an interviewing process for an obscure company. Maybe a desperate, unemployed programmer? Since the best programmers are already attracted and hired by the big firms (Google, Microsoft, Facebook, etc.), is this any way to recruit them away?

This is simply a case of “don’t know what is missing“. The company simply doesn’t realize how drastically it has shrunk the pool from which it can hire and assumes that it has the best possible developers.  There is some benefit to doing your homework before hiring someone, but this is a practice that has gone too far into the other direction.

Comments (0)

By Krishna, November 30, 2009

It was fun reading Michael Dubakov’s take on technical debt through his story of Arthur and the princess. But then the questions started. The competition was rigged from the beginning, wasn’t it? After all, Arthur made it to the final round, didn’t he? What if the weather had been perfect for the next few months?

Illustrating a concept through stories and anecdotes makes an article more interesting and more likely to be read. But it is a tactic that is fraught with risk and can, at times, flirt with deception.

To begin with, a story is obviously fiction and you can make up whatever you want to. This means that you also hand a licence to believers in the opposing view to create their own stories which end differently. So a person who believes in quick-and-dirty coding can make up a story where careful planners starve to death while the risk-taker wins and lives happily ever after.

Also, stories can be pretty condescending. The worst examples I have come across are the Ken Blanchard books like “Who Moved My Cheese?” and other imitators. The writer has an agenda and he/she manipulates the shallow characters in the book to further it. Few non-fiction writers have the skill to develop a compelling story and meaningful characters. An example to the contrary being “The Goal” by Eliyahu Goldratt.

Anecdotes are, at least (or hopefully), truthful. But every such tale represents a single data point. They do not tell the whole story. They lend themselves to cherry-picking and presenting a view completely opposite to reality. Writers need not be malicious to do this; only the incidents that promote their ideas may stay in their memory.

One legitimate use of anecdotes is debunking a generalized statement. For example, if someone says, “X is always true” and you remember an incident to the contrary. But this one application is usually exploited to create FUD. We see that in newsrooms, where although the crime rate is declining, crimes are reported as though there is a huge crisis.

Since writers, including me, like to use stories and metaphors because it makes our writing richer, readers have to be on alert. Writers themselves may be true believers in something and may not realize how they are trying to manipulate their readers.

Comments (0)

By Krishna, November 29, 2009

Anecdotes are dangerous. Scott Koon has one about fixing a doorknob, linking it to the now-infamous Spolksy duct-tape programmer post and writes:

Think about that the next time you reject a new programming tool because you think it might take too long to learn or it’s different. Instead of doing the hacky way you KNOW will work, check to see if there is a more elegant or permanent solution. Because if you aren’t careful, your cheap and easy workaround might end up sticking around in the project long enough for it to become a problem to you.

People keep missing the point. There is a situation where we have bad programmers who use whatever tools they know to get things done. Everything about that situation is wrong because you have bad code – lengthy, mangled, unmaintainable. There is no argument about what should be done to those developers.

The other situation (and what the real argument is about) is the case of an experienced developer who has been successful in the past and who would rather use what is working for them than use something they are unfamiliar with. Because the tool has been successful for them, they do not accept that it may become a problem in the future. An example could be as simple and recent as an ASP.NET developer choosing to continue with WebForms than use the (arguably better) MVC framework.

Choosing to dismiss an experienced developer’s older tool as a “cheap and easy workaround” is a misunderstanding of the situation. The old way of doing things may have been the gold standard a few years/months ago. The new tool may work better, but only for a person who has spent enough time learning it to master its nuances. And there may be many such new tools, each of them making extravagant promises. So we also have the problem of making the right choice.

In a sense, this is the classic Technology Adoption Lifecycle problem, having to do with how fast people accept new technologies.

Technology Adoption Lifecycle (CC Wikipedia)

This was for firms using a particular technology, but it could apply to programmers moving to new technologies. When do they think a particular technology is good enough for them? Or when the current technology is unsustainable for them?

Comments (1)

By Krishna, November 27, 2009

Brilliant post on testing by Michael Bolton (emphasis mine):

Bug Investigation and Reporting (time spent on tests that find bugs)Test Design and Execution (time spent on tests that don’t find bugs)

Module	Time spent on tests that find bugs	Time spent on tests that don’t find bugs	Total Tests
A	0 minutes (no bugs found)	90 minutes (45 tests)	45
B	10 minutes (1 test, 1 bug)	80 minutes (40 tests)	41
C	80 minutes (8 tests, 8 bugs)	10 minutes (5 tests)	13

[...]If we are being measured based on the number of bugs we find (exactly the sort of measurement that will be taken by managers who don’t understand testing), Team A makes us look awful—we’re not finding any bugs in their stuff. Meanwhile, Team C makes us look great in the eyes of management. We’re finding lots of bugs! That’s good! How could that be bad?

On the other hand, if we’re being measured based on the test coverage we obtain in a day (which is exactly the sort of measurement that will be taken by managers who count test cases; that is, managers who probably have an even more damaging model of testing than the managers in the last paragraph), Team C makes us look terrible. “You’re not getting enough done! You could have performed 45 test cases today on Module C, and you’ve only done 13!”

And yet, remember that in our scenario we started with the assumption that, no matter what the module, we always find a problem if there’s one there. That is, there’s no difference between the testers or the testing for each of the three modules; it’s solely the condition of the product that makes all the difference.

The obvious larger lesson here is that if you are going to use metrics, comparing numbers is not going to do anything. You have to be willing to spend the time to understand what is going on. Too often, managers use metrics as a club to hit employees on productivity, something that is enormously counter-productive. As employees find a way to make the metrics towards what the manager wants instead of the needs of the project.

This is also a case for collecting fewer metrics so that you can spend greater time analyzing them. For example, A looks pretty good on test coverage, but there might be the possibility that we haven’t written some necessary tests. Or now that we know C is pretty buggy, we may need to look at the development process – what stage these bugs are getting introduced and why?

In several recent posts, I have talked about the quality versus productivity issue in mature software organizations. Since deadlines loom larger, they are often given higher priority by developers and testers at the expense of quality. And so, managers have to make a special effort to emphasize quality and avoiding shortcuts. Creating productivity metrics swings the pendulum in the other direction. It introduces pressure to get more done instead of getting things done right. So instead, when you evaluate these kind of metrics, stop looking at “finding lots of bugs” or “getting enough done”, but instead at the direction the product quality is taking.

Comments (1)

By Krishna, November 26, 2009

From the Domain-Driven Design Site:

[A] great deal of effort has gone into the design of networks, databases, and other technical dimension of software. Books have been written about how to solve these problems. Developers have cultivated their skills.

Yet the most significant complexity of many applications is not technical. It is in the domain itself, the activity or business of the user. When this domain complexity is not dealt with in the design, it won’t matter that the infrastructural technology is well-conceived. A successful design must systematically deal with this central aspect of the software.

Those of us with a software background have a tendency to get mired in the technical details of implementation. All of those discussions around the duct-tape programmers and unit tests and inversion of control. In some ways, it is a good debate to have. But they are just tools to do the actual work, which is to provide solutions to real-world problems.

And in the real world, there is enormous complexity. If you take a business application, there are so many rules and constraints demanded by various departments in the organization. Ignore software for a moment and think of any organization you have worked in. Remember the employee manual? That is just the start, isn’t it? There are so many other de-facto rules for inter-personal or inter-group coordination because of organizational history, etiquette, bureaucracy, etc.

Then there are outside rules by government and standards bodies, to name a couple. The sheer volume of these rules makes it very difficult for any one person to truly understand them all, let alone the inconsistencies and relationships between them. To make matters more complicated, these rules are changing always. Not everything all at once. But one rule here, one standard there – this month, that month. And your requirements keep shifting ground. There is never a fixed target.

That is the real reason why building software is complex. And it is not going to be solved simply by having a better tool or process which only help if you have somehow managed the complexity of the problem. Sometimes, the real-world complexity is, in fact, too difficult to manage. And the best you can do is take a small piece of the problem and try to solve it. Or maybe you can influence the real world to get rid of a business need like, say, end some silly bureaucratic process, and avoid the need to build a complex piece of software.