The College Board is changing its scoring method for 5-way multiple-choice questions on the AP exams from 
Previously a student who knew nothing could either leave the question blank (score 0) or guess (0.2 change of getting a 1, 0.8 chance of getting -0.25, for an expected value of 0 and standard deviation of 0.5). Students who knew anything at all were better off guessing. Now all students are much better off guessing, even if they know less than nothing (that is, even if their chance of guessing the right answer is less than the uniform 0.2, they are still better off guessing).
Essentially, all students are now forced to guess, as the system has switched from one in which guessing was neutral to one in which guessing gets a bonus. Put another way, AP has introduced a penalty for leaving questions blank. This change is likely to have a gender-biased effect, as girls were more likely to leave questions blank when they didn’t know the answer, and boys more likely to guess wildly.
Since no one uses the raw scores, the shift in raw scores is completely irrelevant. Forced guessing will increase the variance on the exam scores slightly, and cautious students will be penalized substantially until they learn to guess even when they know that they have no idea. I can’t see any advantage to this change.
I think that the problem that the College Board is trying to address is one of terminology. For years, people have incorrectly called the neutralization of guessing a “guessing penalty”, when there was no penalty for guessing (expected value same as leaving question blank). So now we’ll have a guessing bonus instead.
Note: I’ve also heard that the AP test is going from 5-answer multiple choice to 4-answer multiple choice, supposedly to make the test easier. Why is that a desirable result?
I’m only aware of 2 tests that have a real guessing penalty, the AMC-10 and AMC-12. On those tests, the scoring is 
Gas station without pumps 
Of course, you’re leaving out the notion that many people actually guess *worse* than chance, because of the attractor answers (answers that pull for the most common mistakes or misconceptions).
Comment by Aimee Yermish — 2010 July 29 @ 09:32 |
But with the new scoring scheme, guessing worse than chance is better than leaving it blank!
Comment by gasstationwithoutpumps — 2010 July 29 @ 11:04 |
My initial instinct was to like the change, since I was one of those who never guessed on tests (back when I was in high school, anyway–I didn’t master that skill until I was 30+).
But then I thought of how my students performed on multiple-choice tests last year. I was the TA in charge of running the Scantrons through the machine and inputting the grades, so I frequently would go over the statistical reports etc. for fun. I’d pay careful attention if a lot of students missed a particular questions. The good students–the ones who I knew, from lecture, really “got it”– would either pick the right answer, or they’d go for the choice which was close to the right answer. The clueless students seemed to guess randomly.
For example, if the question was on stoichiometry, asking for the number of grams of product given a certain number of grams of reactant, the good students would frequently get the right answer, though sometimes they’d pick the number corresponding to the number of moles of product. In other words, they set up the problem correctly, they just forgot to do the last step. The bad students would frequently pick the number that looked good to them–“I put in 25 grams of reactant A so I should get out 25 grams of product C. Those other products (D, E) and reactants (B) shouldn’t affect the mass.”
Needless to say, I think I actually prefer favoring the educated guess!
Comment by unlikelygrad — 2010 July 29 @ 12:21 |
Yes, I believe that a proper scoring of multiple choice tests does not involve a right/wrong dichotomy, but a different score for each answer, as some wrong answers are more wrong than others. I’ve been thinking about ways to set the weights automatically (at least for tests with a huge number of takers), but I’ve not worked out the details yet. I’ll need to get my hands on some data and do some programming, I think.
Comment by gasstationwithoutpumps — 2010 July 29 @ 12:42 |
The College Boards AP wrong answer “penalty” is similar to their SAT scoring policy. While I do agree with the observations made here, I will share with you the guessing strategy that I share with all my SAT students: Know all the answers then you don’t have to guess! I find that for most students, spending time and energy on getting more answers right is much more productive than worrying about guessing strategies that have minimal effect on their scores!
Comment by Dan — 2010 July 29 @ 13:06 |
The public announcement of the scoring change has been made.
There is also advance information not on their website yet that in 2012 (but not 2011), the 5-answer multiple-choice questions will be replaced by 4-answer questions.
The rationale for both these changes is murky at best. Here is one “explanation” that was given:
That “explanation” makes no sense whatsoever and sounds like a PR person’s coverup. Of course, the change itself makes no sense, so it might be hard to find an explanation that justifies it.
Comment by gasstationwithoutpumps — 2010 September 4 @ 12:36 |
[…] AP creates penalties for not guessing July 2010 6 comments, 491 views. This post was just passing on info about changes coming in AP scoring. High school students seem to really believe that test-taking strategies are more important than knowing the subject when taking AP tests, so people look for information like this, rather than just learning the core material. […]
Pingback by 2010 in review « Gas station without pumps — 2011 January 2 @ 12:52 |
We just found out about the “no guessing penalty” today after the AP test. My son left ~10 questions blank because he thought there will be penalty for wrong answers. This is so unfair to kids who don’t know about the change.
Comment by Yan — 2011 May 2 @ 19:18 |
The new system is unfair, since it requires you to guess. The old system, which made guessing and not guessing have equal expected value, was much more reasonable.
Comment by gasstationwithoutpumps — 2011 May 2 @ 19:28 |
[…] AP creates penalties for not guessing […]
Pingback by Blogoversary « Gas station without pumps — 2011 June 5 @ 10:50 |
[…] I believe is mainly being viewed by middle schoolers trying to cheat on bridge-design exercises.) AP creates penalties for not guessing has also been popular this week (though it is something like 13th on my all-time list), but Advice […]
Pingback by AP exam time « Gas station without pumps — 2012 May 11 @ 09:45 |
[…] AP creates penalties for not guessing […]
Pingback by Second Blogoversary « Gas station without pumps — 2012 June 2 @ 18:15 |
Is this still the case?
Comment by anneisom — 2013 April 30 @ 15:34 |
AP exams still have no points off for wrong answers on the multiple choice part, so it is still better to guess than to leave things blank.
On the written parts of the exam, adding irrelevant material could count against you on a problem, so it is better not to add BS, but it is better to have something rather than nothing, since 0 is the lowest score you can get on a problem there also.
Comment by gasstationwithoutpumps — 2013 April 30 @ 18:52 |