Gas station without pumps

2012 September 12

SAT underpredicts GPA for women

Filed under: Uncategorized — gasstationwithoutpumps @ 21:53
Tags: , , , , ,

I have heard that the SAT is unfair to women, because it under-predicts their college GPAs, and so a gender-neutral threshold would admit men who are likely to do less well in college than women who were excluded.

I was curious whether this claimed difference was true, or whether it was one of those urban legends that circulate based on a misunderstanding.

Luckily, College Board collects data that can address this sort of question and releases free reports that summarize their conclusions.  They don’t always ask the questions of the data that I would ask (see, for example, GPA or SAT?), but this question is precisely the sort that they do ask.  I found two relevant reports:

The first report supports the contention that women’s college GPAs are under-predicted by the SAT. What they did was to make regression models of first-year college GPA predicted by a single SAT subtest, all three SAT scores, high-school GPA, or the high-school GPA and all three SAT scores.  The SAT is more valid for women (that is, the correlation of the predicted first-year GPA with the actual first-year GPA is higher for women than for men).  So the SAT seems to be better for women than for men (more accuracy in the predictions).

But there is a systematic error: the women have higher first-year GPAs than predicted by the regression based on pooled data and the men have lower GPAs than predicted:

Average number of standard deviations (actual-predicted) FYGPA
Gender SAT-CR SAT-M SAT-W SAT HSGPA SAT & HSGPA
Male  -0.14  -0.20  -0.11  -0.15  -0.08  -0.10
Female  0.12  0.17  0.10  0.13  0.07  0.09

Note that this bias is not unique to the SAT: the high-school grade point average shows the same bias, though not so extremely. Note that a difference in means of 0.28 standard deviations is a huge effect in such a large sample—much bigger than most educational interventions that are touted as panaceas.

Of course, one possible explanation is that women and men choose different majors in college and that different grading standards apply. For example, engineering and physical sciences have more men than women, but have the strictest grading standards, while education has more women than men and has the most grade inflation.

So the 2008 study alone can’t answer the question about whether the SAT score is biased against women.

The 2012 study splits up students it studies into 15 groups of majors (one large one is “undeclared”, who do much worse than all the other groups) and looks at predicting second-year cumulative GPA, which is a more stringent test of usefulness than first-year GPA.

With this study we can look for differences in grading standards by field (yep, education majors get much higher college GPAs than one would predict from high-school GPA and SAT, and computer and information science majors get much lower college GPAs, with a difference of 0.25 standard deviations between education and computer science). These differences overall are not quite as large as the gender differences in the first study, but that may just be because 2nd-year cumulative GPA has less bias than first-year GPA.  Indeed, totals over all majors show less bias than the 2008 study:

Average number of standard deviations (actual-predicted) second-year cum. GPA
Gender SAT-CR SAT-M SAT-W SAT HSGPA SAT & HSGPA
Male -0.09 -0.13 -0.08 -0.10 -0.06 -0.07
Female 0.08 0.11 0.07 0.09 0.05 0.06

The SAT now shows only a 0.19 standard deviation bias in prediction of second-year GPA (which is still huge).  But with the new study we can correct for the different numbers of men and women in different majors and the different grading in different majors, by taking the difference between the female and male residuals for each group of majors separately:

female-male differential prediction
major group (abbreviated)
SAT-CR SAT-M SAT-W SAT HSGPA SAT & HSGPA
ag/natural resources 0.24 0.28 0.19 0.22 0.15 0.16
bio 0.05 0.10 0.02 0.06 0.00 0.01
business 0.17 0.21 0.14 0.16 0.09 0.10
communication 0.26 0.28 0.21 0.24 0.14 0.17
comp sci 0.17 0.19 0.18 0.21 0.19 0.22
education 0.23 0.28 0.19 0.22 0.14 0.20
engineering 0.12 0.17 0.09 0.12 0.08 0.11
foreign lang 0.17 0.22 0.15 0.18 0.11 0.11
health 0.20 0.26 0.17 0.20 0.16 0.15
humanities 0.18 0.23 0.15 0.19 0.09 0.12
math+phys science 0.13 0.17 0.09 0.13 0.04 0.06
security 0.15 0.20 0.10 0.13 0.06 0.07
social science 0.15 0.19 0.12 0.16 0.07 0.10
social service -0.03 0.00 -0.11 -0.05 -0.18 -0.15
undeclared 0.24 0.29 0.20 0.15 0.17 0.19

There is a very consistent bias in the prediction of the second-year GPAs of men and women, with the men consistently getting lower scores than predicted and the women getting consistently higher scores than predicted. The one anomaly is “Social Services and Public Administration”, but their sample had only 28 men from 9 colleges in that cluster of majors, so this is almost certainly a small-sample effect.  The extreme on the other end “Computer and Information Science” had only 80 women from 18 colleges, so may also be subject to a small-sample effect.

So the under-prediction of women’s scores is not an artifact of their choice of majors.  The SAT has a larger bias than high-school GPA, but both consistently over-predict men’s college GPAs.

I suspect that what we are seeing here are differences in studying and partying habits, once the students are out of parental control.  That is, the potential predicted by SAT and high-school GPA is not very different between men and women, but the men are much more likely to abuse alcohol and video games in college, to the detriment of their grades.  Of course, I have no data to back this theory up, as the College Board does not collect statistics on alcohol and video games.  There may be other, equally plausible theories why women do better in college than men, given similar SAT scores and high-school GPAs.

2012 July 26

GPA or SAT?

Filed under: Uncategorized — gasstationwithoutpumps @ 22:01
Tags: , , , ,

I was just reading the College Board report today that looked at whether high school GPA, SAT score, or both was the best predictor of first-year college GPA: Students with Discrepant High School GPA and SAT Scores.

The interesting thing about this particular study was that they separated the students into 3 groups: those for which the SAT and HS GPA said the same thing, those for which the SAT was much higher than expected from GPA (smart but lazy? rebellious?), and those for whom the SAT was much lower than expected (slow but teacher pleasers? bad test takers?).

The correlation with first-year GPA was not good for any of the predictions (highest R2 value was 0.232 for regression using both HS GPA and SAT scores with the group for which the two predictors were in close agreement). The worst predictions were for using the high-school GPA when it it was much higher than the SAT (rampant grade inflation?)—only R2 0.127.

What interested me was that for both the “discrepant” groups, the lower of the two measures was the better predictor of future performance.  That raised an immediate question for me, which unfortunately the unimaginative researchers at College Board did not consider: how good a predictor would the minimum of the HS GPA and SAT score (both converted to Z-scores) be?  My guess is that it would be a better predictor than either alone, and probably better than the standard linear regression of both.  But I have no access to any data to confirm or refute that conjecture.

I suspect that success in college is best achieved by those who have both brains well tuned for test taking and a willingness to work within the system, so that a low SAT or HS GPA limits performance in college.  The minimum function serves as an “AND” operator, in a way that linear regression can’t really mimic.

2012 May 29

LinReg for physics class data graphing

Filed under: home school,Software — gasstationwithoutpumps @ 09:58
Tags: , , , ,

A blog I only recently subscribed to (Physics! Blog! by Kelly O’Shea) had a very nice plug (LinReg for physics class data graphing) for a graphing program I’d not heard of before: LinReg which is available free from Pomona.

My son and I use gnuplot, which is a moderately powerful script-based graphing program that produces good graphs and has a good parameter fitting command, but I’ve given up recommending it to people, because of the extreme difficulty in installing it.  It is also overkill for a lot of high school classes, where fitting a straight line is considered complicated enough.

LinReg looks like it is nearly ideal for high-school and middle school science classes.  It forces students to label their axes, use units, and express the precision of their measurements. It computes error values for the intercept and slope values, using a reasonable simulation approach (sampling Gaussian distributed points about each measurement and refitting).  Kelly claims that her Honors Physics students pick the program up quickly and choose to use it without prompting after the first few uses (unlike Excel, which they always see as a barrier rather than as a tool—an attitude towards spreadsheets that I share).

Data entry in LinReg seems to be mainly manual, which would be a big limitation for me even for the home-school physics class (the speed of sound lab generated several hundred data points just for the ladder measurements).  Because my son and I have successfully installed and mastered gnuplot, I see no reason to change to a more limited program, but I can see the attraction of using LinReg with a class, so that less time can be spent teaching the tool and more time using it.  The limited feature set looks like a very good match to most high-school science classes.

%d bloggers like this: