06 October 2011

Correctness is a Boolean

The most misleading thing about the way programming is taught is how programs are graded. Students are used to receiving a score that represents the sum of the correct parts of an assignment. You can think of it start as starting with a zero and earning points up to a maximum of 100, or as starting with 100 and losing points from there. As for me, I prefer to simply avoid thinking about it.

But when it comes to software, this kind of grading makes almost no sense whatsoever. Especially in the imperative languages that are generally taught to beginning students of programming, the effect of a statement depends crucially on the effects of the statements preceding it, and not just lexically, but temporally.

An error in a single line of a program is not necessarily local to that line, but more likely has effects that propagate forward in time or outward in an expression to cause further errors. By grading programs subjectively according to how many of the statements seem appropriately designed and sequenced, teachers influence students to treat programs as though their objective correctness is dependent on the mere sum of the subjective correctness of their parts.

Beginning programmers often have difficulty thinking above the level of individual statements in order to develop a high-level understanding of program structure and meaning. Professors don’t need to further their confusion by giving them a skewed representation of what makes a program right.

A program ought to be like a proof: a single flaw, and it’s not correct. “Correct” means that it does everything it ought to, and nothing it oughtn’t. That says nothing about the amount of work required to make an incorrect program correct, because that’s entirely subjective; it could be a difference as small as a single character, or as great as the entire program. The changes needed are not necessarily local to individual statements, nor even comprehensible at the statement level.

That’s why, if I ever end up teaching programming, I’ll grade my assignments on the basis of correctness and subjective style separately, and draw a clear distinction between the two. If your program is correct, you’re guaranteed a passing grade. All the points beyond that you gotta earn by writing elegantly. It’s only fair.


  1. Your analogy to proofs is actually not new:

    From "The Art of Proof" by Matthias Beck & Ross Geoghegan:

    "On Grading Homework - The Red-Line Method

    It is essential that the student regularly hand in written work and get timely feedback. One method of grading that we have found successful lessens the time-burden on the instructor and puts the responsibility on the shoulders of the student. It works like this:

    Certain theorems in the book are assigned by the instructor: proofs are to be handed in. The instructor reads a proof until a (real) mistake is found—this might be a sentence that is false or a sentence that has no meaning. The instructor draws a red line under that sentence and returns the proof to the student at the next meeting. No words are written on the paper by the instructor: it is the student’s job to figure out why the red line was put there. Pasting as necessary so as not to have to rewrite the correct part—the part above the red line—the student then hands in a new version, and the process of redlining is repeated until the proof is right.

    The instructor will decide on the details of this method: how many rewrites to allow, and whether to give the same credit for a successful proof on the sixth attempt as on the first. Another issue that arises is how to handle students’ questions about red lines in office hours. Some instructors will want to explain to the students why the red line was drawn. Another approach, which we have found successful, is to have the student read the proof aloud, sentence by sentence. Almost always, when the student reaches the redlined sentence it becomes clear what the issue is.

    In all this we are not looking for perfection of expression—that will hopefully come with time. We start with the attitude that a proof is just an explanation of why something is true, and the student should come to understand that a confused explanation is no more acceptable in mathematics than in ordinary life. But the red line should be reserved for real mistakes of thought. To put this another way, the student needs to believe that writing correct mathematics is not an impossible task. We should be teaching rigor, but not rigor mortis.

    We sometimes say in class that we will read the proof as if it were a computer program: if the program does not run, there must be some first line where the trouble occurs. That is where the red line is."

    Comically (unless you've read this book and are giving it a stealthy homage), they draw the analogy the other way, as well. I think grading of computer programs for courses might be best done using this method, though programs have things like functions, making it difficult to know where to underline first.

  2. @Joe:

    Thanks for that quote! Programs, proofs, and essays are all closely related in my mind, and that interrelationship has been a wellspring of good analogies for my writing about software. It’s good to have some reassurance that I’m not completely crazy.