A few months ago, I had derided the use of riddles and quiz in interviews. My reasoning was that they measure the wrong things, not how the person would actually perform in day-to-day work. In my company, we have tried to implement what we think is a better way to evaluate candidates, namely, getting them to do a straightforward programming test. Candidates are provided the necessary tools such as Visual Studio. They can take their time to write the code. This method is not something unique to us: many other companies use it too.
There are a few aspects of this strategy:
- The problem is non-trivial: It requires the person to develop a well-structured application, failing which the person produces no or incorrect results. Thus, candidates who are not strong in planning how to write their code are easily eliminated.
- The problem does not have a clever solution: It does not require knowledge of obscure algorithms or data structures. It is not something that can be committed to memory. Instead, it requires use of existing experience.
- The inputs and outputs are well defined: The programmer knows what input to process and what would be the output format. One would expect a good programmer to work according to the specifications. So if someone provides the answer in a different format, it is a sign that they don’t read or understand written specifications well.
- The solution can explain the level of technical strength: If there is only one way to solve the problem, then you cannot classify the candidate as a weak, average or strong candidate. All you are doing is establishing a hurdle for the person to cross to get to the next stage. Instead, the problem should allow different solutions, by looking at which you can understand the strengths and weaknesses of the individual.
The solution provided by the candidate should provide insights into how the person organizes code, follows coding standards, uses system libraries and data structures and is aware of the latest improvements to the language. This would not be possible if you rely on problems that only require a few lines of code.
Using a tight time restriction is neither necessary nor useful. For a smaller problem, the time difference between candidates is insignificant. Good quality code may take more time to write, but there is a net time savings because of less maintenance. So allow your candidates to take enough time to plan and write the code. If any candidate seems to be taking too much time than the average person, you can talk to them and wind down the session.
It is surprising how many candidates who look good on paper perform poorly in these tests. It is not even about the quality of the code. Many programmers seem to ignore the instructions given and produce their own output. One person even hard-coded the final answer! At the same time, you also see a few gems who write very clear and elegant code, and your decision becomes a little easier.
This is not a perfect or complete method by any means. You must complement it with one or more interviews to learn additional details about the person.