Gas station without pumps

2014 October 25

Grading based on a fixed “precent correct” scale is nonsense

Filed under: Uncategorized — gasstationwithoutpumps @ 10:12
Tags: , , , , , ,

On the hs2coll@yahoogroups.com mailing list for parents home-schooling high schoolers to prepare for college, parents occasionally discuss grading standards.  One parent commented that grading scales can vary a lot, with the example of an edX course in which 80% or higher was an A, while they were used to scales like those reported by Wikipedia, which gives

The most common grading scales for normal courses and honors/Advanced Placement courses are as follows:

“Normal” courses Honors/AP courses
Grade Percentage GPA Percentage GPA
A 90–100 3.67–4.00 93–100 4.5–5.0
B 80–89 2.67–3.33 85-92 3.5–4.49
C 70–79 1.67–2.33 77-84 2.5–3.49
D 60–69 1.0–1.33 70-76 2.0–2.49
E / F 0–59 0.0–0.99 0–69 0.0–1.99
​Because exams, quizzes, and homework assignments can vary in difficulty, there is no reason to suppose that 85% on one assessment has any meaningful relationship to 85% on another assessment.  At one extreme we have driving exams, which are often set up so that 85% right is barely passing—people are expected to get close to 100%.  At the other extreme, we have math competitions: the AMC 12 math exams have a median score around 63 out of 150, and the AMC 10 exams have 58 out of 150.  Getting 85% of the total points on the AMC 12 puts you in better than the top 1% of test takers.  (AMC statistics from http://amc-reg.maa.org/reports/generalreports.aspx ) The Putnam math prize exam is even tougher—the median score is often 0 or 1 out of 120, with top scores in the range 90 to 120. (Putnam statistics from  http://www.d.umn.edu/~jgallian/putnam.pdf) The point of the math competitions is to make meaningful distinctions among the top 1–5% of test takers in a relatively short time, so questions that the majority of test takers can answer are just time wasters.
I’ve never seen the point of having a fixed percentage correct ​used institution-wide for setting grades—the only point of such a standard is to tell teachers how hard to make their test questions.  Saying that 90% or 95% should represent an A merely says that tests questions must be easy enough that top students don’t have to work hard, and that distinctions among top students must be buried in the test-measurement noise.  Putting the pass level at 70% means that most of the test questions are being used to distinguish between different levels of failure, rather than different levels of success. My own quizzes and exams are intended to have a mean around 50% of possible points, with a wide spread to maximize the amount of information I get about student performance at all levels of performance, but I tend to err on the side of making the exams a little too tough (35% mean) rather than much too easy (85% mean), so I generally learn more about the top half of the class than the bottom half.
I’m ok with knowing more about the top half than the bottom half, but my exams also have a different problem: too often the distribution of results is bimodal, with a high correlation between the points earned on different questions. The questions are all measuring the same thing, which is good for measuring overall achievement, but which is not very useful for diagnosing what things individual students have learned or not learned.  This result is not very surprising, since I’m not interested in whether students know specific factoids, but in whether they can pull together the knowledge that they have to solve new problems.  Those who have developed that skill often can show it on many rather different problems, and those who haven’t struggle on any new problem.

Lior Pachter, in his blog post Time to end letter grades, points out that different faculty members have very different understandings of what letter grades mean, resulting in noticeably different distributions of grades for their classes. He looked at very large classes, where one would not expect enormous differences in the abilities of students from one class to another, so large differences in grading distributions are more likely due to differences in the meaning of the grades than in differences between the cohorts of students. He suggests that there be some sort of normalization applied, so that raw scores are translated in a professor- and course-specific way to a common scale that has a uniform meaning.  (That may be possible for large classes that are repeatedly taught, but is unlikely to work well in small courses, where year-to-year differences in student cohorts can be huge—I get large year-to-year variance in my intro grad class of about 20 students, with the top of the class some years being only at the performance level of  the median in other years.)  His approach at least recognizes that the raw scores themselves are meaningless out of context, unlike people who insist on “90% or better is an A”.

 People who design large exams professionally generally have training in psychometrics (or should, anyway).  Currently, the most popular approach to designing exams that need to be taken by many people is item-response theory (IRT), in which each question gets a number of parameters expressing how difficult the question is and (for the most common 3-parameter model) how good it is at distinguishing high-scoring from low-scoring people and how much to correct for guessing.  Fitting the 3-parameter model for each question on a test requires a lot of data (certainly more than could be gathered in any of my classes), but provides a lot of information about the usefulness of a question for different purposes.  Exams for go/no-go decisions, like driving exams, should have questions that are concentrated in difficulty near the decision threshold, and that distinguish well between those above and below the threshold.  Exams for ranking large numbers of people with no single threshold (like SAT exams for college admissions in many different colleges) should have questions whose difficulty is spread out over the range of thresholds.  IRT can be used for tuning a test (discarding questions that are too difficult, too easy, or that don’t distinguish well between high-performing and low-performing students), as well as for normalizing results to be on a uniform scale despite differences in question difficulty.  With enough data, IRT can be used to get uniform scale results from tests in which individuals don’t all get presented the same questions (as long as there is enough overlap in questions that the difficulty of the questions can be calibrated fairly), which permits adaptive testing that takes less testing time to get to the same level of precision.  Unfortunately, the model fitting for IRT is somewhat sensitive to outliers in the data, so very large sample sizes are needed for meaningful fitting, which means that IRT is not a particularly useful tool for classroom tests, though it is invaluable for large exams like the SAT and GRE.
The bottom line for me is that the conventional grading scales used in many schools (with 85% as a B, for example) are uninterpretable nonsense, that do nothing to convey useful information to teachers, students, parents, or any one else.  Without a solid understanding of the difficulty of a given assessment, the scores on it mean almost nothing.

2014 October 15

Top 50 Colleges for Hispanic Students

UCSC recently got some good news, being top-ranked in BestColleges.com’s list of the Top 50 Colleges for Hispanic Students:

In 2012, 49% of Hispanic high school graduates enrolled at a postsecondary, public institution. This percentage surpassed that of white students for the first time, and Hispanic enrollment in colleges and universities, which has increased 240% since 1996, is expected to continue to grow. Many Hispanic students are the first in their families to attend college, so it is important for them to find a support system that will help them navigate degrees, financial aid, and their school and social obligations.

To make the transition from high school to college, many students may be looking for “Hispanic-friendly” schools. These are schools with a high concentration of Hispanic students already in attendance, or they have a cultural center that focuses on Latino/a, Chicano/a or Hispanic heritages.

Students may also look for a school that will protect their rights and ensure they receive the same quality education as non-Hispanic students. The Hispanic Association of Colleges and Universities (HACU) is an organization that strives to protect the educational rights of Hispanic students. It was instrumental in increasing funding from Title V of the Higher Education Act for Hispanic-Serving Institutions (HSIs). For the 2014 academic year, HACU convinced Congress to give $98 million to HSI undergraduate programs.

To create our rankings, we relied upon our normal methodology to find schools that rank well for academics. Our team then compared that list to the 242 HACU member schools in the U.S. to find the best schools for Hispanic, Latino/a and Chicano/a students. We included the percentage of Hispanic students currently enrolled at each college, along with in- and out-of-state tuitions to add more weight to our rankings. Each school on our list boasts a cultural center, degree programs, or scholarships dedicated to enhancing the experiences of Hispanic students.

The Schools

  1. University of California-Santa Cruz – Santa Cruz, CA
    Hispanic Students as Percent of Total Enrollees: 26%
    Graduation Rate: 91%
    Retention Rate: 74%
    Admissions Rate: 60%
    Tuition and Fees: $13,398 (in-state) and $36,276 (out-of-state)This public research university, located alongside the redwood forests and just under 10 miles from the coast [actually under 2½ miles from the campus entrance to the beach], offers 60 majors in 30 fields. Because of the network of UC campuses, students have a wealth of opportunities that extend beyond UC Santa Cruz. For Hispanic students, the Chicano Latino Resource Center, more commonly known as El Centro, offers a number of programs and resources to support and bolster the on-campus Hispanic community, including academic support, scholarships and financial guidance and social events geared towards unification and integration.
  2. San Diego State University – San Diego, CA
  3. University of California-Riverside – Riverside, CA
  4. Whittier College – Whittier, CA
  5. St. Edward’s University – Austin, TX
  6. California State Polytechnic University-Pomona – Pomona, CA
  7. University of La Verne – La Verne, CA
  8. University of Houston – Houston, TX
  9. Florida International University – Miami, FL
  10. California State University-Long Beach – Long Beach, CA
  11. University of California-Merced – Merced, CA
  12. University of St. Thomas – Houston, TX
  13. Woodbury University – Burbank, CA
  14. California State University-Fullerton – Fullerton, CA
  15. St. Mary’s University – San Antonio, TX
  16. University of New Mexico-Main Campus – Albuquerque, NM
  17. Texas State University – San Marcos, TX
  18. Fresno Pacific University – Fresno, CA
  19. California State University-Channel Islands – Camarillo, CA
  20. California State University-San Marcos – San Marcos, CA
  21. Cuny City College – New York, NY
  22. Mount St. Mary’s College – Los Angeles, CA
  23. California State University-Fresno – Fresno, CA
  24. Texas Lutheran University – Seguin, TX
  25. California State University-Stanislaus – Turlock, CA
  26. La Sierra University – Riverside, CA
  27. California State University-Monterey Bay – Seaside, CA
  28. New Mexico State University – Las Cruces, NM
  29. College of Mount Saint Vincent – Riverdale, NY
  30. California State University-Northridge – Northridge, CA
  31. California State University-San Bernardino – San Bernardino, CA
  32. Schreiner University – Kerrville, TX
  33. Cuny Lehman College – Bronx, NY
  34. Saint Peter’s University – Jersey City, NJ
  35. University of Texas-Pan American – Edinburg, TX
  36. University Of Texas-San Antonio – San Antonio, TX
  37. Texas A&M University-Corpus Christi – Corpus Christi, TX
  38. California State University-Bakersfield – Bakersfield, CA
  39. California State University-Los Angeles – Los Angeles, CA
  40. University of Texas at El Paso – El Paso, TX
  41. Texas A&M International University – Laredo, TX
  42. Eastern New Mexico University – Portales, NM
  43. St. Thomas University – Miami Gardens, FL
  44. Angelo State University – San Angelo, TX
  45. California State University-Dominguez Hills – Carson, CA
  46. Adams State University – Alamosa, CO
  47. Texas A&M University-Kingsville – Kingsville, TX
  48. The University of Texas Health Science Center at San Antonio – San Antonio, TX
  49. Boricua College – New York, NY
  50. Our Lady of the Lake University – San Antonio, TX

I’ve not included the information for the colleges other than UCSC—you’ll have to click through to the original web page to get that information. I note that 23 of the 50 colleges are in California, and 17 of them are public universities. The next biggest group is 16 colleges from Texas. The “name” universities (UCB, UCLA, Stanford,…) don’t appear on the list, because too few of their students are Hispanic—they serve mainly white and Asian students.  UCSC has been aggressively recruiting Hispanic students and has only recently gotten over the 25% enrollment threshold to become an official Hispanic-serving institution (officially given HSI status in 2013), but the figures here are a little old, as we were up to 30% Hispanic by Fall 2013, and are probably above 31% now (Fall 2014 figures aren’t available yet).

Correction 2014 Oct 15: UCSC is not officially an HCI by the US government definition—according to http://officeofresearch.ucsc.edu/broader-impacts/resources/diversity/index.html:

  • The Hispanic Association of Colleges and Universities (HACU) designated UCSC as a HSI member in 2013 because UCSC has >25% Latino undergraduate enrollment). [Of UC campuses] Only UC Merced, UC Riverside, and UC Santa Cruz are members of HACU, as listed on the HACU website.  
  • UCSC is planning to submit a Title V Part A (Developing HSI) grant application in 2015. This is one of EVC Galloway’s “Five by 2015” priorities. We are now conducting an in-depth self-study in preparation for the application—the steering committee in charge of this self-study includes top administrators and senior faculty at UCSC as this is very much a UCSC priority. Once we receive a Title V Part A grant, we will be officially an HSI according to the Dept of Education.

2014 October 13

Say this, not that

Filed under: Uncategorized — gasstationwithoutpumps @ 17:00
Tags: , , , , ,

This summer I bought my son a book to prepare him for college: Say This, NOT That to Your Professor: 36 Talking Tips for College Success. He read most of it, and found it to be reasonably well-written, somewhat poorly copy edited, and worth reading once. Most of the advice in the book he felt was just common sense, but that only means that he has been raised in an academic culture.  What the child of a professor sees as common sense in dealing with professors may seem arcane for someone coming from a different culture—perhaps the first in their family to go to college.

For the past 3 years, over half of our admitted students are first in their family to go to college. So what my son finds “common sense” may be the cultural knowledge of academia that many of the students at UCSC are missing.

After my son left for college, I decided to read the book for myself, to see if it was worth recommending to students at UCSC.

The author, Ellen Bremen, apparently teaches communication at a two-year college (Highline Community College in Des Moines, WA, about an hour and a half south of University of Washington by public transit), and some of the advice she gives seems to be more directed at two-year college students than research university students.  For example, she provides no advice on how to ask a faculty member if you can join their research group, because most 2-year college faculty have no time to do research, but she provides a lot of information about what to do when you miss half a quarter’s classes.

Her example students also seem to be a bit more clueless than the students I see at the University of California.  Perhaps this is because of the stricter admission criteria to UC, or perhaps she has selected the most extreme cases to use as illustrations. Or maybe I just haven’t dealt with enough freshmen—I generally see students in their sophomore through senior years, after they’ve had a chance to get acculturated to academia.

About 3/4 of Bremen’s book is dedicated to what students do wrong, and the last quarter to how students can deal with professors who screw up—about the right ratio for a book like this. Although the actual incidence of student mistakes and faculty mistakes is a larger ratio (more like 10:1 or 20:1), the student mistakes tend to fall into the same sorts of things over and over, with only the players changing names, so a 3:1 ratio is reasonable.

The advice she gives is generally good, though she recognizes only the teaching role for faculty, and assumes that all faculty have as much time and desire to meet one-on-one with students as she does.  At UC, many of the professors see their research role as more important than their teaching role (and the promotion process, summer salary, and publicity about faculty activity clearly favor this belief), so faculty are a little less willing to dedicate 10 hours a week to office hours or meet with students at random times outside office hours. I’m doing a lot of additional appointments this quarter, and it really does break up the day so that I can’t carve out a chunk of time for writing papers or programming.  In previous years I’ve kept one day a week free for working from home, free from student interruptions and meetings all over campus, but this quarter I’ve not been able to do that, so my research time and book-writing time has dropped to almost nothing.  Just coping with the pile of email from students every few hours eats up my day.  I find that a lot of student requests can be handled more efficiently by e-mail than by scheduling meetings—the extra non-verbal communication that Ellen Bremen is so fond of often gets in the way of the actual business that needs to be transacted.

Overall, I think that Bremen’s book is a good one, even if some of the advice is slightly different from I would give.  I think that she would do well to work with a second author (from a research university) for a subsequent edition, to cover those situations that don’t come up much at 2-year colleges.  Despite those holes, I still recommend the book for UC students, particularly first-in-family students.

 

 

2014 September 9

Why are students going to for-profit colleges?

Filed under: Uncategorized — gasstationwithoutpumps @ 18:03
Tags: , , , ,

Faculty in public colleges are often mystified why students would choose to take out enormous loans to attend for-profit colleges whose degrees are mostly not respected either by industry or other colleges. In Confessions of a Community College Dean: Corinthian Learner “Dean Dad” explains why for-profit colleges managed to attract students, despite the low quality and ripoff financing:

Put simply, for-profits rushed in to fill the void left by the publics.  Decades of relative neglect of public higher education, combined with a certain (ahem) narcissism within the sector itself, left entire populations underserved.  Perhaps for impure reasons, for-profits figured out how to reach students nobody else bothered to reach.  They pioneered evening, weekend, and online delivery.  They built schedules around student needs.  They focused on a few distinct majors that both students and employers could understand.  And for a while, in some sectors, some of them got decent results.  In the late 90’s, you could do a lot worse than graduating with a degree in CIS.

For-profits filled a void. If you want to prevent the next catastrophe, tend to the void.

That means consciously and aggressively using the public sector—both community colleges and four-year regional campuses—as hedges against future disaster. It means making a dramatic and sustained turn away from the long-term trend of austerity for the publics and an open spigot for for-profits. When you include the cost of bailouts, the “efficient” for-profits wind up inflicting a far greater fiscal burden on the public than more generously funded publics would have. That’s even more true when you factor in student loan debt from students who never graduated, or who graduated but never earned salaries commensurate with their debts.

If dampening demand isn’t really an option, and diverting demand to the private sector leads to financial catastrophe, maybe…stay with me…we could fund the public sector well enough to meet the demand itself. Keep student cost down, get quality up, and learn some valuable lessons from for-profits about meeting students where they actually are. Prevent the next wave of for-profit megagrowth by choking off its air supply.

That means getting away from flat or declining operating budgets supplemented by “targeted” grants that fade away in three years, and instead pouring a fraction of what a for-profit bailout would cost into the public sector. When I was at DeVry, I didn’t see fear of the government or fear of lawyers. I saw fear of the nearby community college. There was a reason for that.

As long as for-profits are considered in isolation, we’ll continue to miss the point. Yes, close loopholes, prosecute liars, and enforce regulations, but those amount to fighting the last war. If you want to prevent the next bailout instead of the last one, you have to address the demand side. Give community and state colleges the resources—and, yes, the flexibility—to flood the zone. It’ll cost some money upfront, but it’s cheaper, more humane, and far more productive than bailouts and legal fees after the next collapse. We don’t have a great record of learning from catastrophes, but this one should be easy.

His post was made in response to the failure of the Corinthian Colleges chain, where the investors had siphoned off all they could get away with and the federal government was not allowing them to be the recipients of student loan money any more.  Other for-profit chains (Anthem Education, for example) are also on the ropes, now that the Federal spigot of infinite loans is being turned back a tiny bit. No attempt is expected to claw back the money from the “investors” who ripped off the students tricking them into taking out enormous loans for a fake education.

I think that his message is indeed correct. It is a lot cheaper to fund public universities well than to clean up the mess left by a completely unregulated for-profit education “industry”.  Students should not need to take out enormous loans to get an education, particularly not the inferior one that the for-profits provided.  Society is better served by having an educated populace that is not hugely in debt.

 

First-generation students

Filed under: Uncategorized — gasstationwithoutpumps @ 14:27
Tags: , , , ,

There were a couple of blog posts by in The Academe Blog in August 2014 pointing to articles about first-generation college students: students whose parents did not graduate from college with 4-year degrees.  This is a particularly important topic for University of California campuses, as UC admissions puts a high premium on being a first-generation student, resulting in large numbers of first-generation students.  (In Fall 2013, 44% of the freshmen at UCSC were first-generation students.)

In the first post First Generation Students Part I: Difference-Education, Walter Breau quotes from an  interview published in the Stanford Report:

“Closing the Social-Class Achievement Gap,” offers a new approach to help [first-generation students] advance in college: discuss class differences rather than ignore them. The research showed that when incoming first-generation students saw and heard stories from junior and senior students with different social-class backgrounds tell stories about their struggles and successes in college, they gained a framework to understand how their backgrounds shaped their own experiences and how to see this as an asset,”

(How’s that for 4 levels of indirection: me quoting Breau quoting an interview with Hamidani about a paper by Hamidani, Stephens, and Westin.)

The abstract of the paper itself sums up the research fairly well:

College students who do not have parents with 4-year degrees (first-generation students) earn lower grades and encounter more obstacles to success than do students who have at least one parent with a 4-year degree (continuing-generation students). In the study reported here, we tested a novel intervention designed to reduce this social-class achievement gap with a randomized controlled trial (N = 168). Using senior college students’ real-life stories, we conducted a difference-education intervention with incoming students about how their diverse backgrounds can shape what they experience in college. Compared with a standard intervention that provided similar stories of college adjustment without highlighting students’ different backgrounds, the difference-education intervention eliminated the social-class achievement gap by increasing first-generation students’ tendency to seek out college resources (e.g., meeting with professors) and, in turn, improving their end-of-year grade point averages. The difference-education intervention also improved the college transition for all students on numerous psychosocial outcomes (e.g., mental health and engagement).

The do not identify the private university at which they did the study, and I believe that the details of the university make a huge difference.  The adjustments needed when first-generation students are an insignificant minority (as they are at most elite private colleges) and when they are 44% of the incoming class are likely quite different, and so the interventions needed may differ not just in scale but in kind.  Since all the authors are at private research universities (Stanford and Northwestern), they likely did their study at either Stanford or Northwestern, neither of which has many first-generation students, and both of which have large numbers of rather wealthy students.  I question somewhat how well the results of studies on such campuses generalizes to the public universities which may soon be majority first-generation students (or already are in some cases), and where social class is not so skewed toward the wealthy.

In First Generation Students Part II: Cultural Fit, Walter Breau points to an article about cultural fit:

“Unseen Disadvantage: How American Universities’ Focus on Independence Undermines the Academic Performance of First-Generation College Students,” published in 2012 in the Journal of Personality and Social Psychology, concluded:

“That the seemingly positive middle-and upper-class cultural norms that pervade traditional American universities—norms that emphasize independent values such as ‘do your own thing,’ ‘pave your own path,’ and ‘express yourself’—can undermine the academic performance of first-generation students.”

The distinction that the article makes between “middle-class” and “working-class” values is independence vs. interdependence.  They found that college administrators at top-ranked colleges valued independent decision making much more than they valued interdependence, but that there were not significant differences between responses to questions about about independent vs. collaborative work.  Lower ranked colleges had less of a skew toward valuing independent decision making, but were otherwise fairly similar.

The study looked at a modest sample of student surveys and grades (245 first-generation, 1179 continuing-generation) to see whether independent/interdependent motive predicted cumulative GPA at the end of 2 years.  The effect sizes they saw were tiny—only coming out larger than race because they lumped all non-white race categories together, which makes some sense for looking at independent/interdependent, but not for looking at GPA, since Asian student typically perform quite differently from black and Latino students.  SAT scores alone were a much better predictor of grades than any of the sociological variables they looked at, and the only really good predictor of 2-year GPA was the 1-year GPA. So, while their results were statistically significant, it is not clear that they were really large enough to justify any conclusions about whether the “American Universities’ Focus on Independence Undermines the Academic Performance of First-Generation College Students”.  I certainly would not want to base policy decisions on such small effects.

Their study was done at schools with only about 17% first-generation students (based on the participant numbers in their study), which may be typical of US colleges as a whole, but is nothing like the UC system.  Even UCB, with the lowest ratio of first-generation students of any UC campus, had 23% first-generation students in 2010 (the latest for which I could find UC-wide data), and some campus were majority first-generation even in 2010 (Merced and Riverside).

Although it is important for UC to figure out how to help first-generation college students succeed, it is not clear to me that either of the studies that Breau reported on have much relevance for UC.

 

Next Page »

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 276 other followers

%d bloggers like this: