Gas station without pumps

2012 November 3

SAT today

Filed under: home school — gasstationwithoutpumps @ 22:18
Tags: , ,

My 16-year-old son took the SAT exam this morning.  This is the second time he’s taken it—the first time was 4½ years ago, at the end of 6th grade, for a talent search.  At that time, he got excellent scores on the reading and math, but a rather poor score on the writing (worst possible score on the essay).  We were not worried about the essay score then, since he had never been taught to write an essay, but his writer’s block go worse over the years, leading eventually to our home schooling him (see School decisions, part 2).  We’ve been getting him weekly sessions with a writing therapist, trying to improve his ability to overcome his inner censor and write more freely.  He’s always written well, when he has been able to produce output, but writing to a deadline has always been very difficult for him and timed writing nearly impossible.

It seems that the writing therapy paid off on today’s exam.  The essay prompt he was given (which he did not tell us, as he takes the required secrecy of the exams very seriously) was very similar to one he’d been given in a practice session with his writing therapist recently, so he could take the examples he had previously written about and reuse them for this new prompt.  I understand that this approach of reusing  previously crafted arguments is very common among high-school debaters and more common than it should be in college essays. I have some evidence for the essay reuse in college—I reviewed Phi Beta Kappa candidates one year and recommended rejecting one who had used essentially the same final paper in three different classes.  It may have been a fine essay, but reusing it was intellectual laziness that should not characterize Phi Beta Kappa honorees.  (This was some time ago—the more recent transcripts have only grades and not narrative evaluations that include the topics of final papers, so this sort of intellectual laziness is harder to detect now.)

I am a little bothered by how much my son benefited from the coaching by his writing therapist.  He clearly needed the extra help in getting past his writer’s block, and it seems to be helping in his other courses, letting his writing show more of what he is capable of, so I’m not at all sorry we are  buying him the therapy.  But I am concerned that the SAT essay is so coachable—that makes it much less useful as an exam, since it will detect ability to pay for coaching as much as it will detect ability and achievement. I would have been happy if his therapy made it possible for him to write an essay on an unfamiliar prompt despite the time pressure—having the therapy make the prompts familiar seems too much like teaching to the test.

Given the luck he had on getting a prompt close to something he had already thought about, this may be the last time he takes the SAT—we’ll know in a few weeks when we get the scores. Given that home-school GPAs are rather uninterpretable, we’re going to have to rely on SAT scores (and other standard test scores) to show his abilities to college admissions offices. We don’t need to show a 2400, but we do want to see a substantial improvement since 6th grade, when his total was 2050.  I’d like to see something above 2210 (top 1%ile), and if it weren’t for his writer’s block, I think a 2350 would be more representative of his abilities.

Assuming that the SAT is done with, his writing therapy can now focus on other forms of writing he needs to master—ones more like writing he will do in college and beyond.

2012 September 12

SAT underpredicts GPA for women

Filed under: Uncategorized — gasstationwithoutpumps @ 21:53
Tags: , , , , ,

I have heard that the SAT is unfair to women, because it under-predicts their college GPAs, and so a gender-neutral threshold would admit men who are likely to do less well in college than women who were excluded.

I was curious whether this claimed difference was true, or whether it was one of those urban legends that circulate based on a misunderstanding.

Luckily, College Board collects data that can address this sort of question and releases free reports that summarize their conclusions.  They don’t always ask the questions of the data that I would ask (see, for example, GPA or SAT?), but this question is precisely the sort that they do ask.  I found two relevant reports:

The first report supports the contention that women’s college GPAs are under-predicted by the SAT. What they did was to make regression models of first-year college GPA predicted by a single SAT subtest, all three SAT scores, high-school GPA, or the high-school GPA and all three SAT scores.  The SAT is more valid for women (that is, the correlation of the predicted first-year GPA with the actual first-year GPA is higher for women than for men).  So the SAT seems to be better for women than for men (more accuracy in the predictions).

But there is a systematic error: the women have higher first-year GPAs than predicted by the regression based on pooled data and the men have lower GPAs than predicted:

Average number of standard deviations (actual-predicted) FYGPA
Gender SAT-CR SAT-M SAT-W SAT HSGPA SAT & HSGPA
Male  -0.14  -0.20  -0.11  -0.15  -0.08  -0.10
Female  0.12  0.17  0.10  0.13  0.07  0.09

Note that this bias is not unique to the SAT: the high-school grade point average shows the same bias, though not so extremely. Note that a difference in means of 0.28 standard deviations is a huge effect in such a large sample—much bigger than most educational interventions that are touted as panaceas.

Of course, one possible explanation is that women and men choose different majors in college and that different grading standards apply. For example, engineering and physical sciences have more men than women, but have the strictest grading standards, while education has more women than men and has the most grade inflation.

So the 2008 study alone can’t answer the question about whether the SAT score is biased against women.

The 2012 study splits up students it studies into 15 groups of majors (one large one is “undeclared”, who do much worse than all the other groups) and looks at predicting second-year cumulative GPA, which is a more stringent test of usefulness than first-year GPA.

With this study we can look for differences in grading standards by field (yep, education majors get much higher college GPAs than one would predict from high-school GPA and SAT, and computer and information science majors get much lower college GPAs, with a difference of 0.25 standard deviations between education and computer science). These differences overall are not quite as large as the gender differences in the first study, but that may just be because 2nd-year cumulative GPA has less bias than first-year GPA.  Indeed, totals over all majors show less bias than the 2008 study:

Average number of standard deviations (actual-predicted) second-year cum. GPA
Gender SAT-CR SAT-M SAT-W SAT HSGPA SAT & HSGPA
Male -0.09 -0.13 -0.08 -0.10 -0.06 -0.07
Female 0.08 0.11 0.07 0.09 0.05 0.06

The SAT now shows only a 0.19 standard deviation bias in prediction of second-year GPA (which is still huge).  But with the new study we can correct for the different numbers of men and women in different majors and the different grading in different majors, by taking the difference between the female and male residuals for each group of majors separately:

female-male differential prediction
major group (abbreviated)
SAT-CR SAT-M SAT-W SAT HSGPA SAT & HSGPA
ag/natural resources 0.24 0.28 0.19 0.22 0.15 0.16
bio 0.05 0.10 0.02 0.06 0.00 0.01
business 0.17 0.21 0.14 0.16 0.09 0.10
communication 0.26 0.28 0.21 0.24 0.14 0.17
comp sci 0.17 0.19 0.18 0.21 0.19 0.22
education 0.23 0.28 0.19 0.22 0.14 0.20
engineering 0.12 0.17 0.09 0.12 0.08 0.11
foreign lang 0.17 0.22 0.15 0.18 0.11 0.11
health 0.20 0.26 0.17 0.20 0.16 0.15
humanities 0.18 0.23 0.15 0.19 0.09 0.12
math+phys science 0.13 0.17 0.09 0.13 0.04 0.06
security 0.15 0.20 0.10 0.13 0.06 0.07
social science 0.15 0.19 0.12 0.16 0.07 0.10
social service -0.03 0.00 -0.11 -0.05 -0.18 -0.15
undeclared 0.24 0.29 0.20 0.15 0.17 0.19

There is a very consistent bias in the prediction of the second-year GPAs of men and women, with the men consistently getting lower scores than predicted and the women getting consistently higher scores than predicted. The one anomaly is “Social Services and Public Administration”, but their sample had only 28 men from 9 colleges in that cluster of majors, so this is almost certainly a small-sample effect.  The extreme on the other end “Computer and Information Science” had only 80 women from 18 colleges, so may also be subject to a small-sample effect.

So the under-prediction of women’s scores is not an artifact of their choice of majors.  The SAT has a larger bias than high-school GPA, but both consistently over-predict men’s college GPAs.

I suspect that what we are seeing here are differences in studying and partying habits, once the students are out of parental control.  That is, the potential predicted by SAT and high-school GPA is not very different between men and women, but the men are much more likely to abuse alcohol and video games in college, to the detriment of their grades.  Of course, I have no data to back this theory up, as the College Board does not collect statistics on alcohol and video games.  There may be other, equally plausible theories why women do better in college than men, given similar SAT scores and high-school GPAs.

2012 August 29

Tests for 11th grade

Filed under: home school — gasstationwithoutpumps @ 11:48
Tags: , , , , , , , , , , ,

In Home schooling restarts, I listed the courses I thought my son would take this year.  In this post I’ll list the exams I think he’ll need.

test date register by comments
pSAT Oct 17 ? He needs to take pSAT this year for National Merit Scholarships, as he has a good chance of getting one.
SAT Oct 6, Nov 3, Dec 1, Jan 26, May 4, Jun 1 Sep 7, Oct 4, Nov 1, Dec 28, Apr 5, May2 I think that he should take an SAT test early this year, to find out how much more work he needs on SAT essays.  If necessary, he would take the SAT again in the Spring. SAT exams can’t be on the same day as SAT Subject tests, though multiple Subject tests can be the same day.
ACT Oct 27, Dec 8, Feb 9, Apr 13, Jun 8 Sep 21, Nov 2, Jan 11, Mar 8, May 3 ACT is not popular on the coasts, but I’ve heard that some students do better on the ACT than on the SAT (and vice versa).  If his SAT scores aren’t what we hope for, it may be worth doing an ACT test as well.
SAT Spanish with Listening Nov 3 Oct 4 It would probably be better for him to take this text next year, after one more Spanish class.  It is only offered in November.  He could take the SAT Spanish without listening on any other SAT day.
SAT World History Dec 1, Jun 1 Nov 1, May 2 This test may be a way to validate that his idiosyncratic “world history through history of science” covered an adequate amount of world history.  The SAT subject test should be less demanding than an AP test.
SAT Physics May 4, Jun 1 Apr 5, May 2 This test should be pretty easy, compared to the AP Physics C tests, but he’ll need to finish E&M first.  It will help with schools that require two STEM SAT Subject tests.
AP Physics C: Electricity & Magnetism May 14 March? We’ll have to find a school willing to administer this exam, as no high school in the county does Physics C.
AP Computer Science A May 8 March? He may be able to take this at Pacific Collegiate, as they have an AP computer science course.
AP Spanish Language May 8 March? This is scheduled for the same time slot as AP CS, so one of them has to be moved to the makeup slot.
AMC 12 Feb 5, Feb 20 Dec 13 He could take either or both AMC 12 tests this year.  (Taking both increases his chance of being invited to take the AIME test.)
AIME Mar 13?? If he does well on the AMC 12, he could be invited to take the AIME in mid March or early April—it doesn’t seem to have been scheduled yet.

Combining these tests with previous years’ he’ll have 5 AP test scores and 3–4 SAT Subject scores. We’ll have to check the list of colleges he might apply to, to see if any need 5 SAT Subject tests.

Other dates

He has several other scheduled activities this year that need to be planned around.

Activity date comments
Performance Jan 19, 20 Imaginary Invalid at Broadway Playhouse (WEST Ensemble Players)
Performance May 4, 5 The Outsiders at Broadway Playhouse, Saturday morning rehearsal may conflict with SAT. (WEST Ensemble Players)
Performances ? Dinosaur Prom Improv, probably every 6–8 weeks.
Oregon Shakespeare Festival May? Dates for trip not yet on AFE calendar.
County Science Fair Mar 9 If he decides to do science fair this year, he’ll have to register by Feb 15.
California State Science Fair Apr 29–30?? Schedule not posted yet. Only relevant if he decides to do science fair and gets chosen to represent the county at state.
International Science and Engineering Fair May 12–17 Only relevant if he decides to do science fair and gets chosen to represent the county at ISEF. Conflict with AP exams can be resolved by taking exam at ISEF.

2012 July 26

GPA or SAT?

Filed under: Uncategorized — gasstationwithoutpumps @ 22:01
Tags: , , , ,

I was just reading the College Board report today that looked at whether high school GPA, SAT score, or both was the best predictor of first-year college GPA: Students with Discrepant High School GPA and SAT Scores.

The interesting thing about this particular study was that they separated the students into 3 groups: those for which the SAT and HS GPA said the same thing, those for which the SAT was much higher than expected from GPA (smart but lazy? rebellious?), and those for whom the SAT was much lower than expected (slow but teacher pleasers? bad test takers?).

The correlation with first-year GPA was not good for any of the predictions (highest R2 value was 0.232 for regression using both HS GPA and SAT scores with the group for which the two predictors were in close agreement). The worst predictions were for using the high-school GPA when it it was much higher than the SAT (rampant grade inflation?)—only R2 0.127.

What interested me was that for both the “discrepant” groups, the lower of the two measures was the better predictor of future performance.  That raised an immediate question for me, which unfortunately the unimaginative researchers at College Board did not consider: how good a predictor would the minimum of the HS GPA and SAT score (both converted to Z-scores) be?  My guess is that it would be a better predictor than either alone, and probably better than the standard linear regression of both.  But I have no access to any data to confirm or refute that conjecture.

I suspect that success in college is best achieved by those who have both brains well tuned for test taking and a willingness to work within the system, so that a low SAT or HS GPA limits performance in college.  The minimum function serves as an “AND” operator, in a way that linear regression can’t really mimic.

2012 June 14

Ways in to University of California

Filed under: home school — gasstationwithoutpumps @ 16:38
Tags: , , ,

In a previous post, I talked about how to use various tests to satisfy the University of California a–g entrance requirements.  We are still planning to follow the outline described there, but it turns out that there are some other workarounds that are suitable for home-schoolers:

Mixing high school and community college classes

The University of California – a-g Guide describes how community college classes are counted for students being admitted as freshmen. There does not appear to be an upper limit for the number of units a high-school student can earn at community college (despite what our consultant teacher told us): “Even if high school seniors have a large number of units from community college coursework, they would still apply as freshmen, not transfers.”  Of course, other colleges probably have other policies.

UC-transferable courses count as “honors” classes (with a 1 point grade bonus in GPA calculations). According to the Assist website, both the Spanish classes my son took this year were UC-transferable, so count as honors.  Each counts as a 1-year high school class. If he decides to take (and can get into) a Theater Arts class at the community college, most of those would get bonus points also. Of course, since many of his classes don’t get grades, it isn’t clear that we’ll be able to honestly report a GPA anyway, so the bonus points may not matter.

Admission by exam

University of California – Admission by exam describes a different way of getting into UC.  This is not using exams to replace individual a–g course requirements (that’s at Options for Satisfying UC’s “A-G” Subject Requirements for Freshman Applicants Applying for 2012–13), but using just exams to establish eligibility. “If you don’t meet UC’s minimum requirements, you may be considered for admission to UC if you earn high scores on the ACT Plus Writing or SAT Reasoning Test and two SAT Subject Tests.”

This method is probably the easiest way for my son to establish eligibility.  Currently he has only one SAT Subject test, but he’ll almost certainly be taking another this year.  With his 6th-grade SAT (which had a rather low writing score) and his one Subject test, his “UC score” is already at 335.  He just needs to improve his writing subscore on the SAT slightly when he retakes it this coming year and get a 650 or better on another SAT Subject test to qualify for admission by exam.

There is one limitation: “You may not use an SAT Subject Test to meet these requirements if you have completed a transferable college course in that subject with a grade of C or better.” So taking a SAT Subject test in Spanish would not help with admission by exam, but physics would.  (Of course a SAT Subject test in Spanish might help with admission through the normal mechanism, or with admission to other universities.)

Admission by Exception

One method touted for home-schoolers is Admission by Exception, where students write why they should be allowed into UC in their personal statement in their application, and then contact the admissions offices directly.  Few students get in by exception (and it seems that many of them are recruited by the big football teams), and I would not recommend going this route unless there is literally no other way to get in.  We are not considering it.

« Previous PageNext Page »

%d bloggers like this: