Gas station without pumps

2014 March 6

Guessing correction is not guessing penalty

Filed under: Uncategorized — gasstationwithoutpumps @ 06:19
Tags: , ,

It bothers me that even the New York Times, in an otherwise excellent article repeats a common misperception:

Students were docked one-quarter point for every multiple-choice question they got wrong, requiring a time-consuming risk analysis to determine which questions to answer and which to leave blank.

via The Story Behind the SAT Overhaul – NYTimes.com.

The guessing correction on the SAT meant that you did not have to do any “time-consuming risk analysis”—if you knew nothing about a question, it did not matter whether you guessed or left the question blank—your expected value was zero either way.

Under the no-guessing-correction system, students are forced to guess, or give up 1/5th of a point for each question left blank. Thus the new system rewards gamblers and punishes people who are cautious—rewarding a character trait that should have no influence on test scores.

Under the old system, you could guess randomly on everything you didn’t know if you wanted to—it wouldn’t hurt on average.  Under the new system, you are forced to.  Forced guessing will also penalize those who aren’t aware of time running out, and are forced to “put down their pencils” before getting a chance to randomly bubble the remaining questions. It will also penalize students who would formerly skip difficult questions, then come back to the blank ones later if they had time—leaving a question blank will come with a penalty that wasn’t there before.

I suppose it is too much for me to expected innumerate reporters (even for the NY Times) to know enough probability to understand expected values, but it saddens me that the College Board, in search of higher market share, is giving up on one of the things that they previously were doing correctly.

4 Comments »

  1. I am also bemused by the discussion of the “guessing penalty”, and “time-consuming risk analysis” since the analysis of the risk should be the simplest of probability calculations.

    But, I think in adding the psychological element (you do, here, in calling the guessing benefit a reward for gamblers), the argument would be that the old system penalized people whose sense of whether they know whether an answer is right or wrong is undermined by personality (say, they are enormously insecure, like those people who answer questions with questioning tones). The probability calculation is simple if you can know whether you know an answer, but if you are always unsure, it’s harder to use the guessing to your benefit. I wonder if there are any psychological studies examining the likelihood of these traits in different populations. My non data driven hypothesis would be that women would feel less comfortable having to assess risk and guessing after eliminating questions than men, but I have no idea if that’s been studied.

    I don’t have a strong feeling on changing the test so that there is a guessing benefit (though it makes the advice to fill in all the questions more important), but I share your distress on how the issue is discussed in the media.

    Comment by bj — 2014 March 6 @ 08:24 | Reply

  2. I’m arguing there was a required risk analysis in the old system, since people were told to guess if they could eliminate an option (i.e. eliminating one of the choices does change the expected value). That’s the “complicated risk analysis” and I think a lot of attention is paid to obsessing over that category of problem.

    Comment by bj — 2014 March 6 @ 08:26 | Reply

    • People were given an incorrect analysis in order to sell test prep courses. It saddens me that rather than debunking the test-prep claims about the value of guessing (it never hurt to guess, so no risk analysis was needed), reporters have blindly repeated the propaganda.

      Comment by gasstationwithoutpumps — 2014 March 6 @ 08:57 | Reply

  3. Another thought. The SAT could eliminate the “guessing penalty” without eliminating the “guessing correction” simply by giving 0.2 points for each blank answer. This would please the mathematically ignorant (something for nothing!), without having to penalize cautious people relative to gamblers.

    Comment by gasstationwithoutpumps — 2014 March 7 @ 10:20 | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: