Imran On Tech

Programming Knowledge versus Programming Ability May 16, 2007

For some reason the single most common type of question on “programming language” exams seems to be of the form:

Which of the following are invalid variable names in Java?

1. $char
2. 1MyNumber
3. case
4. _int

(paraphrased from SCJP Exam for J2SE)

Imagine what a similar question might be in a football test:

Which of the following would result in a direct free kick to the opposing team ?

• attempts to kick an opponent
• attempts to trip an opponent
• attempts to jump at an opponent
• attempts to spit at an opponent

(paraphrased from the FIFA rules)

While it might be slightly useful for someone playing football to be able to answer the above correctly, the entire concept of trying to measure someone’s ability by measuring their knowledge is fundamentally flawed.

And that’s the fundamental problem with question posed above, as well as with programming exams in general.

There seems to be a popular misconception that if a software developer knows a lot about programming then it follows that they’re a good developer.

Returning to the football analogy imagine you were back at school picking players for your team. Who gets picked first:

1. The kids who are athletic and good at football
2. The kids who are athletic and can make a good run down the pitch, even if they’re football specific skills aren’t that great.
3. The kids who aren’t athletic but know how to play football
4. The kids who aren’t athletic and know nothing about football

If you had to pick a team now, made out of players you didn’t know anything about. how would you pick – would you give them all an exam about football rules and theory or would you have them run around the pitch kicking a ball ?

Hopefully most people will pick the second option and end up with a team that can actually play football well. However in the software profession there seems to be a sizable contingent of developers who hold the view that if someone knows a lot about programming then they’re a good programmer.

But it’s not just the pro-certification lobby who think like this, to take another example from a blog I read last month:

I don’t do well in programming tasks during interviews, and I’ve love someone to come into my comments and tell me I can’t program based on this event. No, I’ve only faked it while working for Nike, Intel, Boeing, John Hancock, Lawrence Livermore, and through 14 or so books–not to mention 6 years of online tech weblogging.

(from here, apologies to Shelley Powers for picking on her blog, but it’s one of the more eloquent blog posts on this topic that stuck in my mind, hence me quoting it).

Note that everything in that paragraph was about knowledge rather than ability. I’ve seen the CV’s of some vastly incompetent programmers who I’ve known and they’ve had phrases on their CV along the lines of “was involved in project X at big company Y”, when in actuality they were relegated to writing test scripts. Yet they could perfectly well talk about the project, what the development steps were, what the problems encountered were, how the team solved them, etc. These are all knowledge questions, not ability questions.

Similarly writing a book speaks about your knowledge of a language, not your ability as a programmer. Consider the following quote, would you want to work with this developer on a C++ project:

I have not written production software in over 20 years, and I have never written production software in C++. Nope, not ever. Furthermore, I’ve never even tried to write production software in C++, so not only am I not a real C++ developer, I’m not even a wannabe.

If you hadn’t already guessed – that quote is from Scott Meyers (one of the worlds most respected C++ authors). Not that I doubt Scott’s programming ability; but hopefully it does show that knowledge doesn’t necessarily mean experience.

So next time you’re trying to work out if someone’s a superstar coder, pause for a moment and make sure you’re considering both their programming knowledge and their programming ability, and not letting one substitute for the other.