An extremely interesting discussion about plus/minus grades over at Dean Dad’s makes me think about grading and whether it can really evaluate what a student has learned.

Let me describe part of an exam I give, which mixes objective questions with a single right answer and subjective questions that may have more than one right answer.

Section 1: Matching. Students are asked to match 6 statements from the readings with their authors.

Let’s say we read 30 authors during the semester. A student who knows 27 of them could get really unlucky and do very badly on this section. It all depends on the 6 I pick for the test. So—pretty subject to happenstance and could easily not reflect the real state of the student’s knowledge. I think maybe this is what Vicki (a physicist, fourth comment at Dean Dad’s) is talking about, when she says:

Anyone who thinks that even a well-crafted exam is giving them a system that shows real differences between a student with 85% and a student with 81% is fooling themselves, unless they’re using items that have been tested and evaluated for reliability and validity.

Not only does which 6 authors I pick affect the validity of this test, but the exact quotation I pull from the text could make a huge difference.

Okay. Section 2: Essay Questions. Here’s an example: “Discuss the relationship between Islam and long-distance trade.” This could be a book, of course, but students only need to hit the major ideas and details that we learned. (Disclaimer—I’m not being precise here at all, in either the question or the answers, as I haven’t actually used this question for a while and it’s not my specialty.)

Okay, to get full credit on this question, the student first has to know some pretty big ideas:

  1. Islam spread to many regions
  2. trade was integral to the spread of Islam

I’m pretty confident that if a student can’t get those two ideas, a bad grade will accurately reflect that they didn’t learn much.

Then, they need to back those major points up with some details. They need to list most of the places where Islam spread, not necessarily all. They need to offer up some examples about how trade and Islam interacted—e.g., traders carried the knowledge of Islam, some people converted to Islam in order to build trade links, the need to regulate trade helped entrench the Islamic legal system, etc. They need to reference some of the specific documents we read about Islam, to give concrete details of their examples.

The third relevant issue here is choice. Students don’t normally have any choice with the Matching. With Essays, the students are given a selection of questions. So in effect, they are given every opportunity to show off their own knowledge. I have no idea whether math/science tests normally offer choice.

So the conclusion I’m coming to here, based on a very small sample size of hypothetical and anecdotal data, is that even though students perceive objective questions as more “fair”, subjective questions can actually do a better job of evaluating what the student has really learned. That is, while tests are a blunt instrument, it might be easier for the humanities prof to turn the bludgeon into a knife than it is for the science prof—even though in theory, the science prof might be capable of achieving a scalpel.