Once again, Jay Mathews of the Washington Post looks at the wrong statistic to get ludicrous results: High school list shows hidden differences .
I’ll give him the benefit of the doubt on intentions: that his interest is really in raising the education levels of American high-school students, particularly in poorer neighborhoods, rather than in simply selling AP tests.
His measurement, though, is clearly wrong. What matters is whether students are learning, and if you want to measure at the AP level, what matters is how many students are passing the tests. Neither the number taking them nor the percentage passing are really relevant—both are easily manipulated by administrators and teachers controlling who takes the test. He praises himself for inventing a measure that depends not on student outcomes, but just on how many students are made to take a test (tests taken divided by graduating seniors). By his measures, a school that forced all students to take AP tests while teaching them absolutely nothing is a great school, while one that taught students well but did not encourage test taking was an abject failure. A school with an enormous dropout rate would do even better (fewer graduates to divide by).
There is an equally simple measure he could use that would be enormously more meaningful: the number of tests passed divided by the number of entering freshmen. (Even better would be to divide by the number of freshman who entered 4 years earlier, so that fluctuations in population would not add noise.) This measure would still have the desirable effect of pushing schools to challenge more students (you can’t get high numbers of tests passed if no one is preparing for them), but avoids much of the bad effect of pushing unprepared students into courses way over their heads (the number passing doesn’t go up if you add a lot of students at the bottom).
Mr. Mathews has a magical belief that simply exposing students to “rigorous courses” transforms them into well-educated students, whether or not they are capable of the work or learn anything in the courses. He has led an AP-for-everyone movement that has resulted in substantial watering down of AP courses, to the point where students who could have benefited from a college-level course in high school can no longer trust the AP label on a course to mean that (see High school course title inflation).
Since Mr. Mathews has been informed repeatedly over the years by many people that what he is measuring is really not the right thing, I can only assume that he is either completely resistant to learning from others or of admitting mistakes. (Either that, or his intent really is to sell AP exams, not to improve education.) What surprises me is that he continues to get huge corporate support for his endeavor. The corporations are not usually stupid or resistant to learning, so I can only assume that they do have some less savory reason for continuing to push Mr. Matthews’s nonsense.
I also noticed that even within his own distorted system, he is not above cooking the books. For example, he does not include Pacific Collegiate School, a lottery-entry charter with 337 tests taken and a graduating class of 62. In previous years he excluded them for spurious reasons (he thought the students were too smart, though he did not exclude test-entry schools for the gifted)—this year he does not even discuss their absence from his list (at least, not where I can find it).