SAT underpredicts GPA for women

I have heard that the SAT is unfair to women, because it under-predicts their college GPAs, and so a gender-neutral threshold would admit men who are likely to do less well in college than women who were excluded.

I was curious whether this claimed difference was true, or whether it was one of those urban legends that circulate based on a misunderstanding.

Luckily, College Board collects data that can address this sort of question and releases free reports that summarize their conclusions.  They don’t always ask the questions of the data that I would ask (see, for example, GPA or SAT?), but this question is precisely the sort that they do ask.  I found two relevant reports:

• Differential Validity and Prediction of the SAT from October 2008.
• The Validity of the SAT for Predicting Cumulative Grade Point Average by College Major from August 2012.

The first report supports the contention that women’s college GPAs are under-predicted by the SAT. What they did was to make regression models of first-year college GPA predicted by a single SAT subtest, all three SAT scores, high-school GPA, or the high-school GPA and all three SAT scores.  The SAT is more valid for women (that is, the correlation of the predicted first-year GPA with the actual first-year GPA is higher for women than for men).  So the SAT seems to be better for women than for men (more accuracy in the predictions).

But there is a systematic error: the women have higher first-year GPAs than predicted by the regression based on pooled data and the men have lower GPAs than predicted:

	Average number of standard deviations (actual-predicted) FYGPA
Gender	SAT-CR	SAT-M	SAT-W	SAT	HSGPA	SAT & HSGPA
Male	-0.14	-0.20	-0.11	-0.15	-0.08	-0.10
Female	0.12	0.17	0.10	0.13	0.07	0.09

Note that this bias is not unique to the SAT: the high-school grade point average shows the same bias, though not so extremely. Note that a difference in means of 0.28 standard deviations is a huge effect in such a large sample—much bigger than most educational interventions that are touted as panaceas.

Of course, one possible explanation is that women and men choose different majors in college and that different grading standards apply. For example, engineering and physical sciences have more men than women, but have the strictest grading standards, while education has more women than men and has the most grade inflation.

So the 2008 study alone can’t answer the question about whether the SAT score is biased against women.

The 2012 study splits up students it studies into 15 groups of majors (one large one is “undeclared”, who do much worse than all the other groups) and looks at predicting second-year cumulative GPA, which is a more stringent test of usefulness than first-year GPA.

With this study we can look for differences in grading standards by field (yep, education majors get much higher college GPAs than one would predict from high-school GPA and SAT, and computer and information science majors get much lower college GPAs, with a difference of 0.25 standard deviations between education and computer science). These differences overall are not quite as large as the gender differences in the first study, but that may just be because 2nd-year cumulative GPA has less bias than first-year GPA.  Indeed, totals over all majors show less bias than the 2008 study:

	Average number of standard deviations (actual-predicted) second-year cum. GPA
Gender	SAT-CR	SAT-M	SAT-W	SAT	HSGPA	SAT & HSGPA
Male	-0.09	-0.13	-0.08	-0.10	-0.06	-0.07
Female	0.08	0.11	0.07	0.09	0.05	0.06

The SAT now shows only a 0.19 standard deviation bias in prediction of second-year GPA (which is still huge).  But with the new study we can correct for the different numbers of men and women in different majors and the different grading in different majors, by taking the difference between the female and male residuals for each group of majors separately:

	female-male differential prediction
major group (abbreviated)	SAT-CR	SAT-M	SAT-W	SAT	HSGPA	SAT & HSGPA
ag/natural resources	0.24	0.28	0.19	0.22	0.15	0.16
bio	0.05	0.10	0.02	0.06	0.00	0.01
business	0.17	0.21	0.14	0.16	0.09	0.10
communication	0.26	0.28	0.21	0.24	0.14	0.17
comp sci	0.17	0.19	0.18	0.21	0.19	0.22
education	0.23	0.28	0.19	0.22	0.14	0.20
engineering	0.12	0.17	0.09	0.12	0.08	0.11
foreign lang	0.17	0.22	0.15	0.18	0.11	0.11
health	0.20	0.26	0.17	0.20	0.16	0.15
humanities	0.18	0.23	0.15	0.19	0.09	0.12
math+phys science	0.13	0.17	0.09	0.13	0.04	0.06
security	0.15	0.20	0.10	0.13	0.06	0.07
social science	0.15	0.19	0.12	0.16	0.07	0.10
social service	-0.03	0.00	-0.11	-0.05	-0.18	-0.15
undeclared	0.24	0.29	0.20	0.15	0.17	0.19

There is a very consistent bias in the prediction of the second-year GPAs of men and women, with the men consistently getting lower scores than predicted and the women getting consistently higher scores than predicted. The one anomaly is “Social Services and Public Administration”, but their sample had only 28 men from 9 colleges in that cluster of majors, so this is almost certainly a small-sample effect.  The extreme on the other end “Computer and Information Science” had only 80 women from 18 colleges, so may also be subject to a small-sample effect.

So the under-prediction of women’s scores is not an artifact of their choice of majors.  The SAT has a larger bias than high-school GPA, but both consistently over-predict men’s college GPAs.

I suspect that what we are seeing here are differences in studying and partying habits, once the students are out of parental control.  That is, the potential predicted by SAT and high-school GPA is not very different between men and women, but the men are much more likely to abuse alcohol and video games in college, to the detriment of their grades.  Of course, I have no data to back this theory up, as the College Board does not collect statistics on alcohol and video games.  There may be other, equally plausible theories why women do better in college than men, given similar SAT scores and high-school GPAs.

About these ads