Programming Knowledge versus Programming Ability May 16, 2007
Posted by Imran Ghory in Uncategorized.trackback
For some reason the single most common type of question on “programming language” exams seems to be of the form:
Which of the following are invalid variable names in Java?
- $char
- 1MyNumber
- case
- _int
(paraphrased from SCJP Exam for J2SE)
Imagine what a similar question might be in a football test:
Which of the following would result in a direct free kick to the opposing team ?
- attempts to kick an opponent
- attempts to trip an opponent
- attempts to jump at an opponent
- attempts to spit at an opponent
(paraphrased from the FIFA rules)
While it might be slightly useful for someone playing football to be able to answer the above correctly, the entire concept of trying to measure someone’s ability by measuring their knowledge is fundamentally flawed.
And that’s the fundamental problem with question posed above, as well as with programming exams in general.
There seems to be a popular misconception that if a software developer knows a lot about programming then it follows that they’re a good developer.
Returning to the football analogy imagine you were back at school picking players for your team. Who gets picked first:
- The kids who are athletic and good at football
- The kids who are athletic and can make a good run down the pitch, even if they’re football specific skills aren’t that great.
- The kids who aren’t athletic but know how to play football
- The kids who aren’t athletic and know nothing about football
If you had to pick a team now, made out of players you didn’t know anything about. how would you pick – would you give them all an exam about football rules and theory or would you have them run around the pitch kicking a ball ?
Hopefully most people will pick the second option and end up with a team that can actually play football well. However in the software profession there seems to be a sizable contingent of developers who hold the view that if someone knows a lot about programming then they’re a good programmer.
But it’s not just the pro-certification lobby who think like this, to take another example from a blog I read last month:
I don’t do well in programming tasks during interviews, and I’ve love someone to come into my comments and tell me I can’t program based on this event. No, I’ve only faked it while working for Nike, Intel, Boeing, John Hancock, Lawrence Livermore, and through 14 or so books–not to mention 6 years of online tech weblogging.
(from here, apologies to Shelley Powers for picking on her blog, but it’s one of the more eloquent blog posts on this topic that stuck in my mind, hence me quoting it).
Note that everything in that paragraph was about knowledge rather than ability. I’ve seen the CV’s of some vastly incompetent programmers who I’ve known and they’ve had phrases on their CV along the lines of “was involved in project X at big company Y”, when in actuality they were relegated to writing test scripts. Yet they could perfectly well talk about the project, what the development steps were, what the problems encountered were, how the team solved them, etc. These are all knowledge questions, not ability questions.
Similarly writing a book speaks about your knowledge of a language, not your ability as a programmer. Consider the following quote, would you want to work with this developer on a C++ project:
I have not written production software in over 20 years, and I have never written production software in C++. Nope, not ever. Furthermore, I’ve never even tried to write production software in C++, so not only am I not a real C++ developer, I’m not even a wannabe.
If you hadn’t already guessed – that quote is from Scott Meyers (one of the worlds most respected C++ authors). Not that I doubt Scott’s programming ability; but hopefully it does show that knowledge doesn’t necessarily mean experience.
So next time you’re trying to work out if someone’s a superstar coder, pause for a moment and make sure you’re considering both their programming knowledge and their programming ability, and not letting one substitute for the other.
Imran On Tech 
The Scott Meyers quote is great. (I have nothing constructive to say, but nice post)
So what is the best way to test ability?
Another comment. Programmer grow by solving problems. The language used is ony important because it allows a programmer to solve different sets of problems. If I had 20 year developing shopping carts in php my real experience level would be low because it only one type of problem in one language. Now if I spent 20 years developing a variety of applications in PHP my experience level would be much higher even if php was the only language I ever used.
Just My Two Cents 🙂
“Note that everything in that paragraph was about knowledge rather than ability.”
Except for the rather important first sentence, that you completely disregarded, which suggests that the actual issue might be an inability to do well at any task under pressure and/or supervision. That’s a trait shared by up to 20% of the population – go read Elaine Arons’ books – and by me. It may come as a shock to you to discover that someone’s performance in a highly artificial, and artificially pressurised, situation may not actually bear any resemblance to their performance in general; if so, you probably need to get out less.
Nice of you to link directly to the post you were quoting, too… but then, being able to check people’s assertions is so passé.
Good post.
It is easier to judge football ability by watching people play for 5 minutes than judging programmers by watching them work for 5 minutes. You cannot give job applicants a 100k+ line programming project during an interview.
So people hiring are stuck either talking about projects they did, asking technical questions, asking them to submit examples of their work, or asking them to solve toy problems. Technical questions and toy problems work well if they are created to show whether someone understands a relevant concept. Unfortunately, I think many technical questions are created just for the sake of asking technical questions.
If 10% of good programmers cannot answer the question you picked, and 10% of bad programmers can, it is still a good way to narrow 200 applicants down to 10 interviews.
My experience is if you ask a variety of easy technical questions that relate to important concepts almost everyone either gets most of the questions right or all the questions wrong. You would be surprised how many applicants cannot answer easy questions.
Technical questions only become irrelevant when they become too technical (such as the questions asked). Ask players what shape a football is – odds are anyone who gets that wrong doesn’t know how to kick.
Hey, I’m sorry to other with such a trivial question but I’m new to logging and was wondering how you were able to have a top nav bar with a “home” and “about” tabs? I’m trying to get a tab thing going but I am a rookie and was wondering if you could help me out? Is it a plugin?
thank you so much for your time
~adrianrhen.com
There are things in life where people tend to make assumptions and conclusions about themselves, but in reality they know all too well the blurred lines that they fabricate to elevate their chances for professional acceptance – which is quite true for hackers, programmers and developers alike.
Nice post, bdw.
>If 10% of good programmers cannot answer the question you picked, >and 10% of bad programmers can, it is still a good way to narrow 200 >applicants down to 10 interviews.
More likely down to 28 or so, assuming a typical applicant pool… But yeah.
[…] Imram on Tech has a post making the point that there is a distinction between one’s knowledge of programming arcana […]
Amazing !
The analogy is awesome, even i have wondered some similar situations, whenever a programmer goes for an interview, all he is asked to is just to wirte a low level logic say something like factorial or fibonacci. Or he is questioned about the stupid Syntaxes or return types of some grandma methods APIs, how can you judge a person with this? Progamming Ability, is something realted to Thinking, now with remebering. Syntax I can anytime refer to the APIs, that is not a deal. But thinking is the matter which really matters.
See whether he can think, dont see whether he remebers old old stuffs.
Great post. You’re right, ability is what people should look for. Knowledge is just one aspect of someone’s ability to program.
I was lucky. My lecturer told me not to turn up to lectures, but gave me a “philosophy of programming” lunchtime session. I was “apprenticed” to a guy who started in 1963 on ICLs.
They taught PROGRAMMING. They didn’t teach anything about languages.
And when I’ve mentored, getting students who thought they knew about programming because they could write C code, I drummed into them what I consider the two most important practices of good programming: INDENTING so you can read the structure easily and COMMENTING so you can read intent.
I thoroughly recommend “The Tao of Programming” : it’s spot on
Interesting post, I think a good candidate would have a mix of both language knowledge and the practice of programming. Football makes a difficult analogy because as a field of knowledge and practice it is quite well-defined and limited. A good programmer can use the basics of a language to carry out any task but if they can take advantage of a utility feature or a framework to do they job faster and more reliably then they will be the better candidate.
Some SJCP derived questions are pointless trivia that you learn purely to get jobs: e.g. the size of primitives, whether true and false are keywords or not. However if some has passed SJCP 5.0 say, you would expect them to know how to use Regex with Strings, Generics and Collections for example. That’s going to make them more productive than someone who has taken the qualification and isn’t so sure how to use them.
Probably the best way to recruit is via recommendation and after that by pair programming but there is always going to be some kind of unfair, arbitrary screening process to decide who to invest your recruitment effort in.
Excellent post. Reminded me of this great joke in Peopleware:
Juggler Interview
Circus Manager: How long have you been juggling?
Candidate: Oh, about six years.
Manager: Can you handle three balls, four balls, and five balls?
Candidate: Yes, yes, and yes.
Manager: Do you work with flaming objects?
Candidate: Sure.
Manager: …knives, axes, open cigar boxes, floppy hats?
Candidate: I can juggle anything.
Manager: Do you have a line of funny patter that goes with your juggling?
Candidate: It’s hilarious.
Manager: Well, that sounds fine. I guess you’re hired.
Candidate: Umm…Don’t you want to see me juggle?
— From Peopleware
Great,discussion but can we think this topic in different way,i mean
what is the most acceptable way to check the knowledge and ability.Remember if we say knowledge,it have different meaning for different people.But i believe that a person who prove himself at right time ,right place and right way that person is a sound in any type of crunch situation.At least this is true in some sense.
– Vinay @Servelots
Spot on. During a round of interviews of my last job move I was often asked “which operators can’t be overloaded in C++?”. I knew the answer, and I confidently rattled it off. I also referred to :: as the ‘scope resolution operator’ for extra points.
Of course, neither had any real baring on my ability to write C++. If it wasn’t for interviews, I would consider it useless trivia!
I do not entirely agree with you , at least based on the examples that you have mentioned. Firstly, if a person is any good, he or she must have the knowledge that pertains to their domain. Surely their experience in their fields would enable them to answer such knowledge based questions (I suppose, though, that the examples you have presented are purely didactic). Secondly, big corporations simply do not have the time to sift through the thousands of applicants using such time consuming processes. If the person is so experience-wise, it’s upto the person to be good in knowledge as well as showcase his or her special expertise.
[…] Programming Knowledge versus Programming Ability « Imran On Tech […]
Couldn’t agree more. I think I’m a good programmer, but all my jobs have revolved mostly around the (now EOL) Delphi language. I’ve done quite a bit of my own work with C#, but I’ve only once even had a return contact for a C# job based on numerous submissions.
It seems all my knowledge of sound practice, OO, and the like is useless. The only thing that matters is whether I’ve had enough experience *using C# and nothing but C#*.
One time I did get an interview where the language was C++ (there was miscommunication). I told the guy I’d never done any C++ since college, and he said it matters not. He had me do an exercise in pseudocode and called it good. It turns out that the particular environment was awful and I didn’t take the job, but I sure like that attitude – he was testing ability and didn’t give a rat’s arse about knowledge.
My current job is slated for outsourcing, so unless I can find a similar situation, my programming career is EOL (just like Delphi).
The programming knowledge and programming ability interesting topic. Programming knowledge anyone can have but programming ability depends of programmers apptitude which develop with experience.
Programming knowledge does not guarantee that person with that knowledge can actually write good structured program keeping it as simple as possible as we all know the maintenance of systems are most expensive.
To be sure, you raise a good point. In the interest of adding to the discussion, I will (partially) defend the typical programming question that you raise. Such a question will (one hopes) only appear on a very introductory exam. The point of the question is not so much that the examined person knows the particular rules of a particular language but knows that there may be rules about how variable names can be chosen in the language used in the course. One might expect that a reasonable exam would only have a few questions of that nature and that a significant part of the purpose of putting such questions on the exam is to attempt to build the confidence of the exam writer with a few trivial questions since many people have confidence issues that interfere with their abilities to perform on exams. Of course, that doesn’t mean that doing this works. In fact, it may accomplish exactly the opposite — the student may be prepared to think and then be faced with annoying fact recall questions.
Another issue to consider is the complete artificiality of exam settings. Just imagine a workplace where your boss said to you, I need you to write a bunch of functions with some specific behaviors and you are not allowed to consult with anyone else nor are you allowed to look at any reference materials and you have 3 hours to do all of the them.
I would be pleased if you were to offer up a constructive suggestion of what questions for introductory programming exams might look like.
Based on that quote, I would never hire Scott Meyers to do any real work, or even to consult.
Time and again in enterprise work, I’ve seen people who test well, who have supposed qualifications, but when it comes to real work, they don’t have the experience. Without experience, you’re worse than useless. If you can’t show real, working, shippable code that you’ve written, you are not a software engineer, you’re an amateur.
My standard interview questions for new team members are: 1) Write a JUnit test case for a class that does X (I don’t care about writing X, just want to see that you understand test-driven development, and it’s a good filter for people who don’t really know the language). 2) Show me an application you’ve written, and tell me about its development; I’m looking for enthusiasm and something that shows real experience.
People who don’t write software shouldn’t write software. Duh!
Excellent. I totally agree with you. Too many times I’ve seen people measured and compared inappropriately.
Now on the the hard part. How do we change the social dogmas in order to make the right choice?
Yes, it’s extremely hard to judge programmers in an interview. But the football analogy isn’t perfect. Like football, programming takes skill. But it also takes knowledge. In fact, the knowledge to skill ratio of programming is far, far higher than the knowledge to skill ratio for football. So it does make sense to measure knowledge in an programmer interview. Of course, knowledge is so much easier to measure than skill it’s tempting to stop there.
It’s really hard to learn about REAL skills in an interview with limited time. You can’t ask candidates to spend days working with you to develop sample apps, which is what it would take to truly assess them. Maybe you can ask for a one-hour exercise, but it better be a good position or the best candidates won’t bother. Maybe the best way to go is contract-to-hire, so you both get to “date” for a few months before locking in the employer/employee relationship.
[…] Programming Knowledge versus Programming Ability « Imran On Tech (tags: programming interview) Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages. […]
[…] Programming Knowledge versus Programming Ability For some reason the single most common type of question on “programming language” exams seems to be of the […] […]
I like the football example. While it looks like a good qustion, the knowledge is not merely irrelevant, it gives the opposite result to what you want. The player you should hire has no reason to know the details of the rule. The one who knows the answer probably learned it by kicking an opponent.
i will only say that i prefer people with ability and passion to solve a problem then a person who know a lot of textbook answers.
if you are wondering why? then answer is simple.. you can always learn the things that you don’t know about( or google it) but you can’t, that easily update your abilities or passion.
I like the last comment on “passion”. I think passion is critical. But, has to be the right kind of passion.
You don’t want someone who just knows about recursive descent parsers, or someone who has merely built a recursive descent parser–what you want it someone who has built a recursive descent parser to solve a real problem. (That couldn’t have been easily solved another way)
My formula:
Passion for Technology + Passion for Results = Great Hire
Good luck finding a way to measure these things!
Programming ability is really problem solving / analytical ability, right? This is very key for a good hire, because being successful requires solving a lot more than just programming problems – social problems, business problems, time management problems, etc.
Experience can be important, too. The first time is always slow, no matter has “smart” you are.
Another note – programing doesn’t always equal solving problems. I’ve seen many “experienced developers” that couldn’t solve problems of any significant difficulty.
Ok, read Shelly’s comments – I think her idea is great – if you agree with her position, you shouldn’t apply to any company that would use such “arrogant” tactics. I’m sure Google, Microsoft, Apple, Yahoo, Expedia, Oracle, and Sun would be relieve to not have to bother telling you no.
[…] Programming Ability In an excellent article, Irman has brought out the idea that measuring programming knowledge is different from measuring […]
[…] Programming Knowledge versus Programming Ability I’m also a strong believer in that there’s a large difference between knowledge and ability (this goes for anything) but wasn’t able to properly express it. This article does just that. […]
Thanks for sharing this information. Really is pack with new knowledge. Keep them coming.
This was a very interesting post as most of the better programmers that I know have little formal education on the subject.
Hey,
But how to identify this compentency objectively… that is the greatest challenge that we encounter… we do realise that having an ability to apply is important, but how do check this compentency in any fresher.
Hello webmaster
I would like to share with you a link to your site
write me here preonrelt@mail.ru
at which point can you call yourself a programmer?
i think the ability of the programmer can measure in the following :
1 – Ability to represent algorithm in a code .
2 – Ability to write/speak etc .. ,an Abstraction about way how to solve a problem .
3 – know whats going on out side his programming language world
I think the ability of the programmer can measure the problem..
1 – Ability to represent algorithm in a code .
2 – Ability to write/speak etc .. ,an Abstraction about way how to solve a problem .
3 – know whats going on out side his programming language world
Very well said. It’s comforting that there are at least a handful of people out there still with some common sense regarding this subject. The problem as I see it, is that standards are slipping badly across the entire industry. A person’s value is measured not simply by knowledge, but by the knowledge of ‘new generation’ sub-standard, bloated, useless crap that some idiots decide should become standard (.net, xml, and to be fair there’s some BAD open source stuff out there too).
The knowledge and ability to produce the fundamental building blocks is not being nurtured. Is this the intention of an elite and greedy minority or is it simply another mark of the stupidity of society? (maybe that was a bit harsh but these are the things that made me the cynical bastard I am today).
Anyway – bit of a rant but I just wanted to say thanks. I’ve been waiting a while to find other people speaking up about these kinds of issues (would you believe I even brought the subject up within mensa and was ignored)
Incredible. http://www.ketan.org is killer.
This is why I love http://www.ketan.org. Killer posts.
And this is the main reason I read imranontech.com. Shocking posts.
http://mismatchinglao.blogspot.com/
Бесплатная RPG онлайн игра Техномагия завоевала интерес тысяч пользователей различной возрастной категории оригинальным интерфейсом, геймплеем, игровым движком. Игра в стиле фэнтези совместила в себе элементы стратегии, тактики и логики. Мир Техномагии красочен и ярок, графика основана на флеш-анимации, при этом ее системные требования минимальны.
23 май 2011 Правильное питание, Диеты, Рецепты домашней кухни, советы как похудеть, кремлевская диета, калорийность, кулинария, рецепты салатов,
15 июн 2007 Как похудеть, если вы любите плотно поесть перед сном? Что делать, если овощи и кефир – не для вас? Один ответ – кремлевская диета!
Все о кремлевской диете и диетах для похудения, раздельное питание и калорийность продуктов, консультация диетолога. Кремлевская диета таблица и рецепты,
27 май 2011 Диеты, здоровое питание, низкокалорийные рецепты, лечение целлюлита, фитнес, упражнения, психология и опыт других.
Азбука Диет – информация о здоровье и правильном питании, диетах и похудении, На сайте мы собираем самые разные диеты: как для похудения (например,
To grow up your business with hire php programmer. we have certified secure server with SSL Secure Server, Security algorithms, Profiles and Valid users. Web Based Support Desk Software and Security is the area in which we have invested more and take it very seriously, which is why we have not conformed to the common safety practices and have gone further, making this issue a priority in the system and achieving Programming Knowledge versus Programming Ability Imran On Tech that is extremely safe. – Floyd
[…] https://imranontech.com/2007/05/16/programming-knowledge-versus-programming-ability/ […]
I wanted to thank you for this excellent read!!
I absolutely enjoyed every little bit of it. I’ve got you book-marked to look at new things you post…
I am sure this piece of writing has touched all the internet visitors, its really
really pleasant piece of writing on building up new weblog.
It’s really a nice and useful section of info. I’m content that you distributed this convenient facts with us. You should stop us informed this way. Appreciate your spreading.
I have been searching on-line greater than 3 hours these days, on the other hand certainly not found out almost any attention-grabbing write-up for instance your own property. It is really attractive price tag ample for me personally. Personally, in case almost all internet marketers plus people designed great articles as you most likely performed, the world wide web is usually a much more handy than any other time.
Hi there awesome website! Guy. Excellent. Awesome. We’ll search for your blog post along with make rss feeds on top of that? We are thankful to get many helpful information here in your post, we’d like produce far more approaches on this value, appreciation for expressing.
cerenis.ueuo.com