Yiyang Fang, first-year undergraduate at Sciences Po Le Havre, statistically evaluates the prospects of people scoring at an economics examination by means of estimation.
LE HAVRE — The recent catastrophe known as the Introduction to the Economy mid-term examinations has triggered a wave of depression of the non-economic kind. So severe was the disaster that on the Le Havre campus, the school had to intervene with a price floor for the MCQ grades and an overall subsidy for the student body.
A great deal of students, I gather, felt cheated by the negative grading of the multiple-choice questions (MCQ) section, where correct answers were given one mark each, wrong answers -0.5 marks, and blanks 0.
A major problem, I am told, was that many students attempted to guess the answer rather than simply leaving it blank. This has even led to negative grades for the MCQ section, although thanks to the price floor these have been bumped up to 0.
But one should not panic. There is a perfectly rational way to approach this.
Let P be the probability that a chosen answer is correct.
The expected value, meaning the long-run aggregate payoff (the marks you’d expect to get for the question, on average, for an infinite number of trials), can be demonstrated by the following equation:
Long-run payoff = (1)(P) + (-0.5)(1-P)
This can be simplified to:
(1)(P) + (-0.5)(1-P) = P – 0.5(1-P) = 1.5(P) – 0.5
For now, let us just assume that if we do not know the correct answer, we will guess randomly. This means that out of four options, we shall choose one at random, with a 0.25 chance of it being correct. If we’re a little more capable and can eliminate, with absolute certainty, one of the options, then our odds improve — our randomly selected answer (out of three options) now has a ⅓ chance of being correct. And if we are able to narrow down our options further to just two, then it’s a coin toss whether the one we choose would be correct.
This can be expressed in a very simple payoff matrix:
| Outcome \ Options | 4 (P=0.25) | 3 (P=⅓) | 2 (P=0.5) |
| (1)(P) | 1(0.25) = ¼ | 1(⅓) = ⅓ | 1(0.5) = 0.5 |
| (-0.5)(1-P) | -0.5(0.75) = -⅜ | -0.5(⅔) = -⅓ | -0.5(0.5) = -¼ |
| Expected value (long-run aggregate payoff) | ¼ – ⅜ = -⅛ < 0 | ⅓ – ⅓ = 0 | ½ – ¼ = ¼ >0 |
| Optimal strategy | Leave blank | <Indifferent> | Guess the answer |
When your long-run expected payoff is less than 0, the optimal move is to leave the answer blank and take the secured 0 marks. When it is equal to 0, then it really does not matter in the long-run — both guessing and leaving it blank will, in the long run, give the same results. Perhaps you should make a decision solely based on your ability to stomach volatility. When the payoff is greater than 0, then it makes sense to guess the answer — you may get it wrong sometimes, but in the long run it is rewarding.
To put it in layman terms, this is how you should decide whether to make a random guess:
- If you know nothing, just leave it blank and take the 0, it’s a gift.
- If you know a little, then it doesn’t really matter.
- If you know a good deal but not everything, take a leap of faith!
So, you should guess if and only if you are faced between two choices, between which you know one must be the correct answer, but which you are completely unable to favour one over the other in even the slightest.
Of course, in reality, exam situations are never as clear-cut as my model is. Usually, we do not eliminate a certain number of options with absolute certainty (or none at all), then remain indifferent between the remaining options.
More likely, we think something like this: option (A) is definitely wrong, while options (B) and (C) seem possible but not very likely. (D) looks more promising, but I’m still not sure.
But the formula still works. If we choose to submit an answer, we will obviously choose the one we think has the highest probability of being correct. Input that probability into P – 0.5(1-P) and you get your payoff, and you know your optimal choice of action based on whether it is negative, positive or zero.
The difficulty, however, is assigning probabilities to the options. For this I have no solution, and I believe the field of behavioural economics has shown that humans are generally very bad at intuiting probabilities. One requirement, however, is clear — the total sum of probabilities must add up to 1. Additionally, since we have seen earlier that P=⅓ will give a long-run payoff of 0, we can rephrase it as such: if you believe the probability of your best guess being correct is >⅓, then submit it as an answer. But how to estimate this, c’est votre problème, pas le mien.
Or, we could just study harder and get everything correct. That seems to me the best course of action.
Le Dragon Déchaîné 