I was just reading the College Board report today that looked at whether high school GPA, SAT score, or both was the best predictor of first-year college GPA: Students with Discrepant High School GPA and SAT Scores.
The interesting thing about this particular study was that they separated the students into 3 groups: those for which the SAT and HS GPA said the same thing, those for which the SAT was much higher than expected from GPA (smart but lazy? rebellious?), and those for whom the SAT was much lower than expected (slow but teacher pleasers? bad test takers?).
The correlation with first-year GPA was not good for any of the predictions (highest R2 value was 0.232 for regression using both HS GPA and SAT scores with the group for which the two predictors were in close agreement). The worst predictions were for using the high-school GPA when it it was much higher than the SAT (rampant grade inflation?)—only R2 0.127.
What interested me was that for both the “discrepant” groups, the lower of the two measures was the better predictor of future performance. That raised an immediate question for me, which unfortunately the unimaginative researchers at College Board did not consider: how good a predictor would the minimum of the HS GPA and SAT score (both converted to Z-scores) be? My guess is that it would be a better predictor than either alone, and probably better than the standard linear regression of both. But I have no access to any data to confirm or refute that conjecture.
I suspect that success in college is best achieved by those who have both brains well tuned for test taking and a willingness to work within the system, so that a low SAT or HS GPA limits performance in college. The minimum function serves as an “AND” operator, in a way that linear regression can’t really mimic.
Gas station without pumps 
I think we need to remember that most students who go to college will have a certain amount of intelligence and talent. And usually the higher the SAT and GPA, the higher caliber college that a student would go to. Whichever college the student goes to, it is likely that the rest of the students are of equivalent intelligence. In such a setting, I am guessing grades fall into a near-normal kind of distribution, and it would be very difficult to match that data with the high school performance, when everyone had a similar performance in high school.
What do you think?
I think performance in college for the Average Joe is determined by how well he has understood surviving college. I wrote a post about it last week.
Comment by funnyphuppo — 2012 July 27 @ 02:57 |
I believe that you are right that sorting of students by HS GPA and SAT scores into different colleges results in lower correlation between those scores and college GPAs. In statistics, this is known as “interval censoring” and usually results in much reduced correlations. The best way around it is to only use data from large schools that have a wide distribution of HS GPA and SAT scores. But some of the biggest systems are highly sorted: the 25%–75% SAT score ranges for Cal State schools all have about a 100–120 point range on each of reading and math, as do the the UC campuses, but the centers of the distributions shift by 170 points even within the UC system (UC Merced is more like a another Cal State campus). Thus the differences between the campuses is bigger than differences within a campus. A student with a 700 reading score is in the bottom half of Stanford or UC Berkeley, but off the top of the charts at Cal State Los Angeles. Should that 700 predict good freshman grades or poor ones? What about different amounts of grade inflation at different campuses?
The high school GPA measures other things than intelligence (including things like willingness to do mind-numbing busywork), but it is true that students at a college are likely to have been selected for fairly similar characteristics on whatever properties the tests and GPA scores measure. It is a lot easier to teach students who are at a similar level, so college admissions are tuned to narrow the range of students they admit as much as possible (no matter what claims the colleges make about the value of diversity).
Comment by gasstationwithoutpumps — 2012 July 27 @ 11:37 |
Do you think a better correlation might be found between SAT and GRE/GMAT/MCAT scores?
Comment by funnyphuppo — 2012 July 27 @ 11:54 |
According to http://arxiv.org/abs/1004.2731, the correlation between SAT and GRE scores is about 0.75 (very high!) for University of Oregon students between 2000 and 2004. I’ve not seen a larger study, though College Board has probably done one.
Comment by gasstationwithoutpumps — 2012 July 27 @ 13:40 |
It surprises me that the lower of the two, GPA and SAT, was the better predictor of future performance. I would think that the GPA is indicative of the performance on the SAT.
Comment by Lawren Smith — 2012 July 30 @ 16:31 |
[…] They don’t always ask the questions of the data that I would ask (see, for example, GPA or SAT?), but this question is precisely the sort that they do ask. I found two relevant […]
Pingback by SAT underpredicts GPA for women « Gas station without pumps — 2012 September 12 @ 21:54 |