Comments (1)

By Krishna, November 26, 2009

From the Domain-Driven Design Site:

[A] great deal of effort has gone into the design of networks, databases, and other technical dimension of software. Books have been written about how to solve these problems. Developers have cultivated their skills.

Yet the most significant complexity of many applications is not technical. It is in the domain itself, the activity or business of the user. When this domain complexity is not dealt with in the design, it won’t matter that the infrastructural technology is well-conceived. A successful design must systematically deal with this central aspect of the software.

Those of us with a software background have a tendency to get mired in the technical details of implementation. All of those discussions around the duct-tape programmers and unit tests and inversion of control. In some ways, it is a good debate to have. But they are just tools to do the actual work, which is to provide solutions to real-world problems.

And in the real world, there is enormous complexity. If you take a business application, there are so many rules and constraints demanded by various departments in the organization. Ignore software for a moment and think of any organization you have worked in. Remember the employee manual? That is just the start, isn’t it? There are so many other de-facto rules for inter-personal or inter-group coordination because of organizational history, etiquette, bureaucracy, etc.

Then there are outside rules by government and standards bodies, to name a couple. The sheer volume of these rules makes it very difficult for any one person to truly understand them all, let alone the inconsistencies and relationships between them. To make matters more complicated, these rules are changing always. Not everything all at once. But one rule here, one standard there – this month, that month. And your requirements keep shifting ground. There is never a fixed target.

That is the real reason why building software is complex. And it is not going to be solved simply by having a better tool or process which only help if you have somehow managed the complexity of the problem. Sometimes, the real-world complexity is, in fact, too difficult to manage. And the best you can do is take a small piece of the problem and try to solve it. Or maybe you can influence the real world to get rid of a business need like, say, end some silly bureaucratic process, and avoid the need to build a complex piece of software.

Comments (0)

By Krishna, November 23, 2009

Jurgen Appelo publishes a list of delegation checklist items from Johanna Rothman and Esther Derby

Is the risk factor of delegating this work adequately addressed? [...]
Do the people have the skills to do this particular kind of work?
Do the people have the right format for the work products to use?
Do the people have the tools necessary to be successful?
Do the people know what the results should look like?
Have you set the boundary conditions for the work (e.g. budget, time, resources, quality)? [...]

This is all very good, but it seems to be more plain-and-simple managing than delegating. Say you are a manager. You have a project with discrete tasks. You have people to do the tasks. You give them the tasks. The checklist above would be exactly the same.

Delegation is a little different. The tasks that you delegate are the ones that you are best able to handle, but assign to someone who is less capable. They may, in fact, have never done such a task before and so usually do not have the skills or knowledge to do them. So delegation is about teaching another person something new. Therefore, it has more risk than the typical task you assign to your team members.

In the short-term, delegation can be risky and it can result in work of low quality and other problems. In the long run, you will have more capable and well-rounded individuals in your team. So there has to be an element of leadership in delegation that looks at the big picture.

Comments (2)

By Krishna, November 22, 2009

Dave Nicolette asks about C# versus Java:

Consider two hypothetical programming languages such that:

Language A

• Supports closures, and
• Can only run on one platform, so that customers who use the language for mission-critical apps are locked into a single operating system vendor.

Language B

• Doesn’t support closures, and
• Runs on all mainstream platforms, so that customers who use the language have flexibility to change their technical environment as their business needs change or as the relative cost of ownership of different platforms changes.

Which of the two programming languages is "better?"

We saw the same failed argument about Java on the desktop a decade ago. Write once, run anywhere. One reason it didn’t work was the dominance of Windows. You were better served with just using the most productive programming platform for Windows, whichever that was or whichever you preferred.

Back in the 1990s and early 2000s, when we had a plethora of choices of web servers, application servers and databases, using Java as a multi-platform language made sense. Today, most Java development is coalescing around Linux, Apache, mySQL and Tomcat/JBoss, making it a de-facto standard.

An experienced Java developer may have had the experience of porting from JRun to WebLogic, or from Sybase to Oracle. But if you are starting today to develop a new application on the aforementioned stack, is such a move likely? All the tools are open-source thus avoiding a need to reduce costs. They are also proven to be scalable, as many high-traffic sites can testify.

Metrics about platform changes would be helpful here, but if they bear out what I say, there is no actual cross-platform advantage offered by Java when you are not changing platforms. You may have other arguments for choosing Java, but this is not one.

Comments (2)

By Krishna, November 17, 2009

Zef Hemel writes about the inscrutable error messages that he gets when making typos in Ruby on Rails, Scala and JBoss Seam. I recently saw a new programmer struggle with the RoR errors and I have been a victim of some of the problems that occur when you type a string wrong in Java. For those who have had the same experience, it can be deeply frustrating to realize that you spent a whole day debugging a typo while managing only to spend minutes on the application logic.

Looking at some of the comments on his posts (especially the RoR post), a few people suggest that Zef would not be blocked if he knew what he was doing. This is true for the reason that once you see an error and know how to fix it, you will know what to do in the future. But it misses the point about what people new to the framework would feel when they use it. If a tool or framework cannot attract new followers easily, it will remain a niche.

The biggest problem with many useful open source and commercial software development tools is precisely this. The creators assume that since the target audience is the programming community, they can get away with unfriendly user interfaces that would not be tolerated in any product meant for general audiences. Or to be accurate, they don’t actively assume this, they don’t even realize that this is an issue.

Here is the crux. Programmers, senior or junior, have limited time to experiment with new tools. They are already productive with the tools they know and so wish to see the same ease-of-use and speed in new tools that they would like to add to their repertoire. Programmers learn by downloading the tool, installing it and trying out a few examples. If they can get something to run and can tinker with it, 90% of the struggle is over. They can keep experimenting and becoming a master of the tool.

On the other hand, if the programmer, after downloading the tool, sees a meaningless error message and then has to spend an hour Googling the term and tracing the error, then they will ditch the tool. Sooner if the tool has a tendency to keep throwing block after block.

There is a condescending school of thought that suggests that good programmers should be able to overcome any hurdles and it is better that the tool is complex enough to only attract the best in the business and maintain a community of experts. This, of course, assumes that good programmers want to spend days learning a non user-friendly tool that has a small user base and more likely to fade out with that kind of philosophy.

What can be done about it? I think there is a business model in here somewhere for a company to create more user-friendly and integrated versions of products. Say, an installation that bundles different products (say Eclipse with JBoss Seam and mySQL), and has a central administrative interface to configure and trouble-shoot. Or an IDE add-in that analyzes the code for possible typos and provides alerts.

Maybe somebody should create a search/answer engine for errors and exceptions. When you land on the answer page, it will provide the different possible scenarios that could have caused the error message, along with their probabilities. It could list the steps that you could take to identify the error.

Comments (0)

By Krishna, November 15, 2009

Jon Skeet has a long post on how programmers fall prey to pretty nasty bugs because they aren’t aware of the details of what they are doing. And he offers the following thoughts:

First, try not to take on more complexity than you need. If you can absolutely guarantee that you won’t need to translate your app, it’ll make your life a lot easier. If you don’t need to deal with different time zones, you can rejoice. Of course, if you write a lot of code under a set of assumptions which then changes, you’re in trouble… but quite often you can take the “You ain’t gonna need it” approach.

Next, learn just enough about the problem space so that you know more than your application’s requirements. You don’t need to know everything about Unicode – but you need to be aware of which corner cases might affect your application. You don’t need to know everything about how denormal number representation, but you may well need to know how rounding should be applied in your reports. If your knowledge is just a bit bigger than the code you need to write, you should be able to be reasonably comfortable.

Pick the right platforms and libraries. Yes, there are some crummy frameworks around. There are also some good ones. What’s the canonical answer to almost any question about java.util.Calendar? Use Joda Time instead. There are similar libraries like ICU – written by genuine experts in these thorny areas. The difference a good library can make is absolutely enormous.

To add to this, I would say that it is very important to enroll in a computer science degree in college before embarking on a programming career. I know there are very good programmers who have learnt everything on their own. But for the general population, it is a good idea to be fully involved in 2-to-4 year degree that provides you some basics, such as what floating-point representation means. Of course, you still have to do the learning, but your physical presence in a classroom is a higher probability that you may learn something.

While recruiting new employees, I always take a look at their academic scores. Although a good academic record does not guarantee a good programmer, a bad record invariably points to a poor software developer, with very few exceptions. Apart from the obvious IQ reason, here are some of the ways why this is so.

1. Getting things done: All other things being equal, a person who pays attention to end results is more valuable. If a person has high IQ, but doesn’t translate that to results, it usually means that they don’t care about delivering results. They may like learning and tinkering, but they will not deliver.
2. Lack of in-depth knowledge: An exam is not the perfect way to measure knowledge and application of knowledge, but it is among the best systems we have. If someone scores a passing grade on an operating systems course, they are probably missing a lot of important information.
3. Problems with authority: Innovation and breaking rules are important, but you also need some level of conformance to the goals of the organization. Poor scorers are sometimes rebels. They don’t feel the need to impress anyone (parents, teachers, etc.) or sticking to schedules.
4. Lack of perseverance: Some people simply lack the determination to tackle tough subjects. They may do well in a controlled setting, but when faced with challenges, they may need a lot of attention.

An example of an exception may be that somebody had a medical or family problem that affected their grades. Or they had a temporary blip that they got over. As always, you have to consider the whole picture. But definitely, academics should be a big part of your hiring process.

Comments (0)

By Krishna, November 12, 2009

David writes in response to Joel’s worries about fast growth:

All these business [Facebook, eBay, Oracle] rely heavily on the network effect: Their product is more attractive than the competition because of their market share. [...] Do you know what kind of software doesn’t have the advantages of the network effect? Ours. One Highrise user doesn’t give a hoot whether we have 10,000 or 100,000 customers. Jane Doe doesn’t benefit if we sign up any other customers this year. As long as there’s a sound business behind the product, she doesn’t care about anyone else. In other words, there are no network effects.

Do you know what other kind of software isn’t affected by the network effect? Bug tracking. I don’t care who else is using Trac as long as it’s great software. It doesn’t benefit us to know that the Shopify guys are using it too (short of just sharing tips and tricks). Again, no network effects.

This is an oversimplification of the debate. There are several ways that you can lose in a market if your competitors has way more customers than you do.

First, you simply will not have the resources to keep up with the development schedule of your competitor. If they have 10 times the people and the money, they will keep crunching out new generations of their software while you are still fixing bugs from a year ago. If the price points remain the same and any prospective customer does a comparison of the products in your space, you will lose.

Second, you cannot keep your biggest customers from jumping ship to your competitor, when and if the other software becomes much better than yours. Those customers will have the resources to migrate necessary data to your competitor. You will be left with smaller customers and lower profits.

Third, you will find hiring and retaining employees difficult. They will want to work for the market leader if they have a choice. And the market leader will want to steal them if your company is on their radar screen. So once again, you are at a disadvantage. Not only are you resource-poor, but your quality of resources is also weaker than your competition.

Fourth, if for some reason, you pose a threat to the sales of the larger company, they have the resources to beat you to gain important customers. Maybe they undercut prices or offer custom services. If they really want to get a client, a large company will go to lengths that you cannot afford. Gaining and keeping enterprise customers is a very tough job.

In short, there is a vicious cycle where, regardless of where you stood before the surge by the competitor, you are losing on all fronts – sales, marketing, programming, customer service, quality, etc.

Once there is a big gap between your competitor and you in terms of sales and size, usually the game is over, but it takes a while for it to sink in. Entrepreneurs are sometimes too full of hubris to admit defeat. They imagine themselves in a David-vs-Goliath fight where they use their strategic brilliance to make a spectacular comeback. The problem is that the geniuses are all working for the other side.

Comments (2)

By Krishna, November 8, 2009

A friend shared Joel Spolsky’s new column on Inc Magazine which asks if slow growth means slow death and suggests a couple of ways to grow faster:

Step One, I think, is to pluck off our biggest competitors. We’re pretty certain that we’ve already built a great product that meets our customers’ needs — but there are still too many cases where we find out that, for some reason, someone went with the other guy. So that’s the development team’s mission for 2010: to eliminate any possible reason that customers might buy our competitors’ junk, just because there is some dinky little feature that they told themselves they absolutely couldn’t live without. I don’t think this is going to be very hard, frankly. Developing great software is something I’m pretty sure we’re good at.

Step Two is something I’m not particularly good at. Not in the least bit. We have to build up our sales force. The bottom line is, we just don’t do enough selling. I’ve been working on the assumption that a product naturally creates demand for itself and the sales team just helps fulfill that demand. But I’ve realized that I have things backward. I’ve come to understand that a sales team drives demand. My problem is that I’ve never been able to figure out how to hire good salespeople. For a guy who wrote a book on how to hire great programmers, it’s mortifying how incompetent I’ve been at enlarging the sales team, which, right now, consists of one terrific account executive and a dog. (I’m just kidding. There’s no dog.)

I will tackle the slow growth/slow death issue in a later post, but what does it mean when you want to try to rapidly accelerate the growth of your business, especially when it is sales-driven instead of marketing-driven, as in the case of FogCreek? Is it just about taking more risk, as Joel seems to think?

Risk-taking is a big part of higher growth, as you have to spend money on sales personnel, marketing, customer service and technical staff while you are uncertain about the returns from such investment. Sometimes, spending money within your comfort zone is not enough to make a difference, so you may have to either take on large liabilities (bank loans) or get investors who will share control of your company.

But there is a different story with reaching for fast growth with a sales-driven approach. Which is that everything about the structure and environment of your organization will fundamentally change. First, it will change from an engineering-driven company to a sales-driven one. This means, that no longer will engineers call the shots about the direction and delivery of the software. The sales people will do that and the engineering staff will have to follow.

The reason is that salespeople are expensive and they only go after large customers. Every potential customer can bring in revenue from tens of thousands to millions of dollars. Let us say that a customer is willing to pay a site license that is equal to 25% your current revenue, but wants to add three non-trivial features that is not on the development road map. You have a choice to make here. If you decide that the integrity of your application is more important, then not only will you be writing off a huge dollar amount, but you will anger the salesperson who will probably quit. Which is not what you set out to do.

So, you take on new customers and do further customizations and changes that take the product away from its initial vision. This is not necessarily a bad thing from a business perspective, because it may meet your business goals. But it definitely will establish a new, different pecking order in your organization. And probably some of your technical employees (especially the earliest ones) will leave the organization because their personal growth needs do not coincide with those of the organization.

In a perfect world, sales and engineering could co-exist happily. But in the real world, money talks and the priorities of the salesforce always come first. Once again, there is nothing wrong in being sales-driven. Higher profits can bring financial benefits and security to everyone in the company and that may be the dream of many of them. Just don’t think that the organization and work environment will stay the same once you change your focus.

Comments (2)

By Krishna, October 26, 2009

Using the new StackExchange platform opened for beta, a few friends and I set up a new cricket question and answer site at http://cricket.kridaya.com. The site has been active for a few days now and it has been an interesting experience working with it in an admin role. I am a fan of StackOverflow, the programming Q&A site, but operating a StackExchange site gives you a different perspective and more insight into how other users behave.

One differentiating feature on StackExchange is the use of OpenID for registration. As a technical user, I like OpenID, because I don’t have to keep remembering the password for every single site I visit. A few clicks and I am logged in. What could be easier and more convenient than that? Everyone should use OpenID, right?

Not so fast. Here are some lessons from our experiences with talking to users who tried to use the site.

First, people have no idea what OpenID is. So they are confused and skip the registration step. People would feel less lost if you tell them, “Login with your Google, Yahoo! or AOL account” (and whatever other sites support OpenID). It won’t hurt to leave out the word “OpenID” itself.

StackExchange’s statement “Log in with OpenID” is more technically accurate than “Register with OpenID“, but it baffles users because they don’t understand how to register on the site. They don’t realize that a new account is created when they login with an OpenID account.

The standard expectation for users visiting a website is that they want to see a Login form (User name, Password and a Submit button), a “Remember Me” checkbox,  ”Forgot Password” link and a “Register” link. The last takes them to a page where they can submit their information including an email for validation purposes. There is a lot of friction, but this is what users expect. When they see something different, they get antsy and don’t know what to do.

Users have been participating on the site (answering questions) without registering on the site, despite the fact that the OpenID link is available on every answer form. StackExchange (apparently) does not provide the ability to force users to register before they can answer questions.

I don’t have any data for this thought, but I also wonder if users who click through for Google’s OpenID authentication get spooked and wonder if they are allowing another site access to their Google account, which is not true. In the traditional registration mechanism, there is a clean separation between one site and the other and so this issue never comes into the picture.

From an administration perspective, there is another problem. Users who login through OpenID and do not update their profile show up as “unknown (google)”. We don’t have their name, email address or any other information as a means to contact them in the future. Strangely, the system captures more information from the unregistered users who answer questions.

In conclusion, OpenID is good in theory, but may be a little premature for non-technical audiences.

Comments (0)

By Krishna, October 25, 2009

Highly recommended – “Don’t Just Roll the Dice“, a free e-book by Neil Davidson.

An excerpt:

Although scientifically purer, it often doesn’t make sense to change a single variable at a time. Theoretically, you shouldn’t change the price of your product, your discounting strategy and the types of bundle that you sell, all at the same time. But practically, it can be the right thing to do. It’s more useful to fix the problem than to understand why it’s broken. When a scientist goes on a blind date that doesn’t work out then, in theory, he should fix one variable at a time, and re-run the date. First, he should change the partner but go to the same film and buy the same flowers. Next, he should keep the partner the same, vary the film and keep the flowers the same, and so forth. But the pragmatist in him will, or should, change the girl, the film, the flowers, and buy some new clothes and shave too. If it works, he might not understand why, but at least he’ll have a girlfriend.

Complement this with some reading on basic microeconomics.

Comments (1)

By Krishna, October 21, 2009

Rob Conery pointed out an interesting comment on his IoC post:

In this simplified example above you switched from 3 lines of easily understandable code to something that requires code in the global.asax, another couple classes, and an interface. The end result contains an extra line or two of code, but now the whole thing is “reusable” and can be unit tested. I happen to agree with Joel’s quote which you openly mocked… it *is* more convoluted. It *is* harder to read and grok. And this was an overly simple example… I’m sure it gets more convoluted with large projects.

I am going to ignore IoC for a minute and take this in a general context. Some junior programmers have a tendency to write lengthy code instead of splitting the code into different classes and methods. The code, in itself, is not bad. I am not talking about poor variable names, bad formatting, copy-and-paste or low level silliness. But it is structurally deficient in the sense that

1. Each method will have many lines of code doing different things. In other words, the method has multiple responsibilities.
2. The coding is algorithmic, i.e., although they are coding in an object-oriented language, the coding is very linear in terms of execution.

I used to have a different perspective about people who did this kind of coding, but over time, I have concluded that this has got to do with the (learnable) ability of programmers to deal with complex systems consisting of different building blocks.

Junior programmers need to see, feel and know the entire program. They feel lost and confused otherwise. This is only possible if the code is one place and it has a very linear structure. This is enough for simple programs. But more complex programs require more code.

Well, it happens that a tree-like structure of code is also easy to understand because it is a easy progression from a linear code structure. You start from the root (the main method), navigate linearly, traverse each branch (if you come upon one), re-trace your steps and keep going until you are done. If the tree structure is not deep, you don’t have to keep too many things in your head and the complexity is managed.

The tree-structure is achieved by moving chunks of code into different sub-routines. But here is where the problem starts. You don’t get a true tree structure because you can call different sub-routines from different parts of your program – almost like having a leaf shared by different branches. Now, this is not a problem when the methods are very simple, but when they start manipulating data, it starts getting complex again. Another problem is that in modern event-driven programming environments, there is no “root” of the program. The linearity has been broken. What you have is a graph that has many inter-connections (or dependencies). Once again, complexity rears its ugly head.

We have different techniques to manage this complexity. Object-oriented design (especially encapsulation), refactoring techniques and design patterns help us manage the increasing complexity of software applications. But some of the techniques seem counter-intuitive if you have never used them.

For example, a junior programmer can reduce some complexity in his code by isolating similar code and moving it into a method with parameters. This reduces the code and decreases the complexity. On the other hand, moving data and code into a new class seems to increase code. It seems to imply more code to understand and worry about.

But, of course, this is not true. The point is to move code into a place where it can be completely forgotten. You create the new class that can be tested on its own and then only worry about the code that interacts with it. To a large extent, we already do this by tapping into the framework classes and APIs.

I want to emphasize that this is a learnable skill. These are all well-documented techniques waiting for some investment of time and effort. As you learn them and use them more in your code, you will become familiar with how they work. Some of these are easier and/or more commonly used than others, so it may take a while to be an expert in them all. The point is to keep learning and keep becoming a better programmer.