Multiple choice test questions and students’ answers to them seem perhaps the simplest data we encounter as teachers. We pose a question. Students read it. Students give the correct answer or the incorrect answer. Tally the correct answers to measure each student’s understanding.
We can deconstruct the process into three components. We assume students:
| Understood the question |
| Knew the correct answer to the question |
| Marked the correct answer |
If we assume either “yes” or “no” for each of these three, we can see those questions are actually much more complicated than we think and may not give the clear indication of learning we think they
You can see in the table, columns A and C are the cases we assume to reflect the state of affairs” Students comprehend the question, and accurately mark their answer. (I am ignoring the potential that the teacher accurately knows the answer and recognizes it. Of course, I am hopeful they are using an automated quiz tool, but let’s focus only on the student here.)
Let’s consider the other situations, two of which give us false information, and two of which give us no information.
In B, the student made a mistake in recording their answer. It seems they know the material, but made a mistake in marking the answer. The result is we have false information.
In E, the student knew the answer the question we were asking but did not recognize it as they misunderstood the question. They also marked an answer other that what we expect. Again, we have false information about that they know. Further, it is possible the answer they gave was the correct one to the question they thought they were answering.
In D, the student knows the answer to the question and gives the answer, but they thought they were answering a different question. While this seems to give us the information we want, we can’t really be sure as they did understand what we being asked.
Of course, I have left off the other potential that the student just guessed the correct answer which also gives false information.
When a teacher looks more carefully at their tests or whatever instruments they use to evaluate students’ learning, they usually find they are not as reliable and valid as they assumed.
For me, this means teachers need to integrate many projects and activities in which students demonstrate what they have learned. It also means teachers need to be open to the possibility that their questions may not be as authoritative as they believe. Once teacher realize this, they begin to focus on understanding learning rather than “correct” answers.