Day diaries

There is an interesting study going on for the next year at the Share Project, led by Sally Fincher from the University of Kent School of Computing:

For one year, we’re going to undertake a “day survey” of teaching activities, asking educators to keep a diary of their teaching on a given day every month. Inspired by Mass Observation we hope this extensive collection will add contrast to the more intense data from the longitudinal study.

If you want to take part, you can sign up on the website for the day diaries.  What they are looking for is a roughly hourly record of what you do and think about on the 15th of each month.  There is, so far, no specific format they are requesting, and no prompts to guide you.  I believe that they plan to use the information they gather to try to figure out what teachers think is important.  I’m not sure how they will digest all the material, but that is their research problem, not mine.

Science conference for high-school girls

On Saturday 9 Oct 2010, UCSC is hosting a science conference for high-school girls from the Monterey Bay Area.

Danielle Feinberg, director of photography at Pixar Animation Studios, will give a keynote talk: “To Infinity and Beyond! The Math and Science Behind Movie Making.”  There will also be a hands-on workshop.

This sort of outreach seems more likely to be helpful in creating more female engineers than efforts like Nerd Girls, which I’ve posted about before.

Sustained performance and standards-based grading

I have been looking at the latest fad sweeping education, standards-based grading (SBG), and trying to see if it something I should incorporate in my own grading practices.

My first post on SBG looked at some of the assumptions and guiding principles of SBG, concluding that it looked like a good idea if you took a reductionist view of education, where you could split your objectives into separately assessable standards.

My second post on SBG looked at the unspoken assumption that assessment is cheap, something that is not the case in many of my classes.

Another problem I have with SBG is that for a lot of standards, the goal is sustained performance, not one-shot success. It isn’t enough to get a comma right once—you have to get have to get almost every comma right, every time you write. Similarly, it isn’t enough to use evidence from primary and secondary sources and cite them correctly once—you have to do it in every research paper you write.

If you forget to include the “sustained performance” or “automaticity” (to use a buzzword that elementary math teachers seem fond of) components of the standards, you get a sloppy implementation that reinforces the do-it-once-and-forget-it phenomenon that makes students unable to do more advanced work.

SBG aficionados believe in instantaneous noise-free measures of achievement.  If a student takes a long time before they “get it”, but then demonstrate mastery, that’s fine.  This results in the practice of replacing grades for a standard with the most recent one.  I think that is ok, as long as the standard keeps being assessed, but if you stop assessing a standard as soon as students have gotten a good enough score (which seems to be the usual way to handle it), then you have recorded their peak performance, not the best estimate of their current mastery.  Think about the fluctuations in stock prices:  the high for the year is rarely a good estimate of the current price, even if the prices have been generally going up.

If you want to measure sustained performance, you must assess the same standard repeatedly over the time scale for which you want the performance sustained (or as close as you can come, given the duration of the course and the opportunity costs of assessment).  The much-derided average is intended precisely for this purpose: to get an accurate estimate of the sustained performance of a skill.

SBG tries to measure whether students have mastered each of a number of standards, under the assumption that mastery is essentially a step function (or, at least, a non-decreasing function of time). Under this assumption, the maximum skill ever shown in an assessment is a good estimate of their current skill level. There is substantial anecdotal evidence that this is a bad assumption: students cram and forget.  Indeed, the biggest complaint of university faculty is that students often seem to have learned nothing from their prerequisite courses.

Conventional average-score grading makes a very different assumption: that mastery is essentially a constant (that students learn nothing).  While cynics may view this as a more realistic assumption, it does make measuring learning difficult.  One of the main advantages of averages is that they reduce the random noise from the assessments, but at the cost of removing any signal that does vary with time.

The approach used in financial analysis is the moving-window average, in which averages are taken over fixed-duration intervals.  This smooths out the noisy fluctuations without eliminating the time-dependent variation.  (There are better smoothing kernels than the rectangular window, but the rectangular window is adequate for many purposes.)  If you look at a student’s transcript, you get something like this information, with windows of about a semester length.  Each individual course grade may be making the assumption that student mastery was roughly constant for the duration of the course, but upward and downward trends are observable over time.

Can SBG be modified to measure sustained performance?  Certainly the notion of having many separate standards that are individually tracked is orthogonal to the latest-assessment/average-assessment decision.  Even student-initiated reassessment, which seems to be a cornerstone of SBG practice, is separate from the latest/average decision, though students are more likely to ask for reassessment if it will move their grade a lot, and the stakes are higher with a “latest” record. Student-initiated reassessment introduces a bias into the measurement, as noise that introduces a negative fluctuation triggers a reassessment, but noise that introduces a positive fluctuation does not.

Perhaps a hybrid approach, in which every standard is assessed many times and the most recent n assessments (for some n>1) for each standard are averaged, would allow measuring sustained performance without the assumption that it is constant over the duration of the course.  If the last few assessments in the average are scheduled, teacher-initiated assessments, not triggered by low scores, then the bias of reassessing only the low-scorers is reduced.

Test prep for kindergarten?

Every year at the beginning of the school year, there are a round of stories about the cut-throat preparation of kids for entry tests for kindergarten (like this one, a few weeks ago).

Test prep for kindergarten seems to be limited mostly to New York City, thanks to a dysfunctional way of assigning kids to schools and the sincere but misguided belief by many New York parents that where a child goes to kindergarten determines their entire scholastic career through graduate school.  Those of us in the rest of the country are rather bemused by the media frenzy over what is really a very local phenomenon. The teacher matters much more than the school does, especially in the early grades, so the scramble to get into a “good school” is a bit silly even if understandable.

I’ve always been rather bothered by the “test preparation” industry.  Mostly they are selling snake oil, so they don’t do too much harm other than taking people’s money.  Occasionally they are selling ways to cheat (like access to the questions that are asked on particular IQ or school entrance exams, which are not supposed to be available to test-takers ahead of time), and then they really are harmful.

Test preparation for kindergarteners is particularly ridiculous, as testing kindergarteners is ridiculous.  There are very few psychometric tests that are reliable for kids under the age of 6.  The things a child needs to be ready for kindergarten are fairly simple and do not need testing to determine.  Furthermore, there is no evidence that a heavily academic kindergarten really offers much advantage to a bright student.

When my son was entering kindergarten (very bright and already reading), I looked for the best teachers I could find.  Although we looked a private school for gifted kids, we ended up with a bilingual program at a public school, because they had some superb teachers, lots of books in the classroom, plenty of library time, and the extra instruction in Spanish would provide something novel for him to learn.  That year I took a sabbatical in the spring in a different city, so we had to do another search for kindergartens.  We looked at several, including both public and private schools, and ended up with a public school that served a very poor neighborhood (though it was temporarily located in another building halfway across the city, due to renovation work at the school).  The reason again was an awesome teacher, a very rich environment in the classroom, and lots of books for him to read.  The range of books to read was important, as he was reading at a high 2nd-grade level by the middle of kindergarten.

School starts next week

My son starts high school on September 1.  We have a bit of a late start this year, because budget cuts have reduced the school year to 175 days.

I counted only 84 days of instruction in the first semester, and 4 of those are “minimum days”, which means that each class gets 80*1.5+4*1=124 hours of instruction per course.  The second semester looks like 87 instructional days, with 5 minimum days, for 128 hours of instruction, but some days may be lost to state testing (also AP testing in the higher grades).  For those with fast mental arithmetic, the missing 4 days are the final exam days at the end of each semester.

My son’s high school uses an “Excel Block Schedule” in which students take only 3 (or 4) courses each semester, but get 1.5 hours a day of each course.  This approach is good for science labs, theater, and art, where long blocks of contiguous time are needed for some of the activities.  It also cuts down on some of the time management and executive skills problems of managing due dates for 6 or 7 classes. The high school claims that the only problem they have seen is in some math classes, and they provide a slower track in algebra for students who need it.  I’m not worried about math for my son, but I think that there could be problems with retention of his Spanish, with 9-month intervals between courses.

I think the block schedule will work well for my son, who prefers being immersed in a few projects, rather than flitting from subject to subject, though I’ve heard from parents of ADD kids that it is very difficult for their kids to deal with the long class periods.  There is a reasonable discussion of the advantages and disadvantages of block scheduling by Lisa Doherty.