SBG and partner work in circuits class

I got the course approval forms for the Applied Circuits for Bioengineers course and lab signed by my chair today, and the department manager has already sent them up the pipeline to the next stage of the process.  Basically, the forms disappear into a black hole at this point, and decisions or questions come back in a few weeks, either from the associate dean or from the Committee on Educational Policy.  Unless politics interferes, the courses should be approved in time to advertise them before Winter quarter.  Right now, I put the probability of the courses being offered this winter at over 90% (way up from a couple of weeks ago, when my estimate of the probability had dropped to 20%).

Of course, I have to finish the course design in time, and I just realized that my weekends the quarter are going to be consumed by grading for my bioinformatics grad class, which has 25 students and no TA.  I think that this is going to mean late nights working on the course design.  I’m wondering whether I’m going to be able to keep up on the physics homework that I’m assigning my son and me for the home-school physics class.

One thing I’ve been thinking about is (perhaps) trying standards-based grading (SBG) in the circuits class.  I think that is doable in the lab course, where we will be putting together a list of lab skills that students need to demonstrate anyway, but I’m not sure we’ll be able to do it for the lecture course.  I’m going to have a hard enough time coming up with enough “assessments” (test/quiz questions) for conventional grading, and standards-based grading seems to rely on having a much larger stock of assessments.  We may end up with a hybrid scheme of checklists of skills for the lab and conventional grading for the book learning.

Another idea I had was about doing group work.  The only lab available with enough equipment (and its availability is far from guaranteed) is set up as 12 stations with 24 seats (2 students per station). Since we are expecting far more than 12 students, we will have to pair students for the lab.  But I have an aversion to forced group work on tasks that are more easily done by one person.  I want to be sure that the pairs are not just one student working and one watching.

Given that most of the students will be in an electronics lab for the first time, pairing students at stations is not a bad idea. A lot of the labs involve manipulating something (voltage, frequency, …) and recording measurements.  Having two people working together (one manipulating, the other recording) is likely to result in better records, though one person could do it alone.

I’m considering requiring that each lab be done with a different partner.  This means that most students would get the opportunity to be both the stronger partner and the weaker one and would practice being helpful in both situations.  No one would be stuck with a freeloader for the whole term nor with a partner who whizzes through everything without involving them.  It also means that students will learn who makes a good work partner and who doesn’t, so that if they need to form a team for a senior project, they know some people to try to include on the team (and some to avoid).  I think that this scheme would have to be done by assigning partners, as allowing free selection could result in some people never getting chosen as partners.  Setting up the partner scheme, adapting it to students dropping the course, and making sure that all students know who their partner is for the next lab so that they can do the prelab work together all add a little extra logistics to this scheme.

One question we need to think about is whether lab reports should be written by the pair, with both names on one report, or whether there should be separate lab reports.  Currently, I’m leaning towards one report per pair, since they will be collecting the data together, doing the design work together, and demonstrating the working design together.  The weaker writers in the class would probably not get enough practice and feedback, since their partners will end up doing most of the writing, but requiring separate reports would probably not fix this problem, since they need to share a lot of content.

Designing tests

A post on Overthinking my teaching by Christopher Danielson, Those aren’t numbers, so don’t treat them as though they were!, lead me to do some thinking about grading systems.  The post itself was about converting standards-based grading (SBG) schemes into point-based schemes so that you got the “right” results when you took percentages and converted back to A, B, C, D, F grades.

Of course, I don’t understand why so many users of SBG systems insist on using numbers for their categories in the first place.  I guess that the “grades-are-evil” meme has so taken root in teacher training that teachers can’t envision using letters for their category names, even when their categories correspond very closely to the original uses of the grade letters. Mr. Danielson says

Maybe you think of it this way:
4 Accomplished
3 Proficient
2 Beginning
1 Struggling
0 Missing

which looks to me exactly the same as saying
A Accomplished
B Proficient
C Beginning
D Struggling
F Missing

He then goes on to assign points for each category (A=40, B=34, C=30, D=26, F=0), to get the percentage-based gradebook he is forced to use to report grades the way he wants (he wants to make sure that students make up any missing assessments, and will fail if they don’t).

I rarely give tests these days, since the skills I’m interested in assessing are ones that take time to display (being able to write and revise detailed papers or to design and code moderately complicated programs).  When I do assess students, I end up using letter grades, which have meanings more like the standards-based grading categories than percentages.  When I need to average, I see no problem with assigning A+=4.3, A=4.0, A-=3.7, B+=3.3, … and averaging those values.  But I teach at a university, where the individual faculty have full control over the grades.  (If a student feels a grade has been unfairly given, the most anyone other than the original faculty member can do is change the course to a pass/fail or remove the course from the record—only the person who issued the grade can change it.)

The problem that I see is not with the SBG categories nor with assigning points to get the desired outcome, but with the notion of fixed meanings for percentages.  I have never understood why so many educators insist that all assessments (homework, tests, quizzes, exams, … ) must be designed so that 90% right is an A, 80–90% is a B, 70–80% is a C, and 60–70% is a D.

From an information-theoretic standpoint, such an assessment provides much less information than one in which the scores are uniformly distributed over 0–100%.  In fact, a test in which almost all students get 50% or more could be half as long (eliminating the easiest questions) with almost no loss of information. Or, if the test were kept the same length, it could be made less sensitive to random measurement error, by testing the material more thoroughly. The standardized tests like the SAT, ACT, and AP exams are not designed so that everyone gets at least 50% right—every question is written so that some fraction of the test takers get it wrong.

Item-response theory, one of the standard methods for designing tests, suggests that the probability of a student getting a question right follows a sigmoidal function of the student’s ability being measured, and that each question can be characterized by the difficulty of the question (the student ability level which leads to a 50% probability of getting the question right) and the slope of the probability function at that point.  A test consists of a mixture of questions with different characteristics.

If all the questions are about the same difficulty, then the test will be very good at distinguishing students below that level from students above that level, but tell us very little about students who are not near the threshold.  (This is good design for a certification test, which is scored on a strict pass/fail basis.) Tests designed to tell us something about all the students, however, need to have a mixture of questions of different difficulty, covering the range of abilities of the students.  Requiring a median score around 85%, however, ensures that most of the questions have to be easy, so that there is a lot of information provided about how the bottom of the class is doing, and very little about how the top is doing. This fits in well with the current political insistence on focusing all educational resources on the students at the bottom (see Debate about how schools treat gifted students), but not with an educational model that insists that all students be taught.

I have seen one family of tests that does a good job of extracting nearly maximal information: the AMC math contests (AMC-8, AMC-10, AMC-12). Those tests are designed with very few easy questions and a few very hard ones.  The average score on the AMC-8 is typically around 40%, and the median for the AMC-10 is around 55%.  The tails of the distribution go out to the edges of scoring on the AMC-8, with a small number of students getting all the questions right and a smaller number getting them all wrong.  (The AMC-10 has too many easy questions, and so has too few students getting the lowest scores.) Since one purpose of the test is to identify those students on the far right tail who are worth coaching for more difficult math contests, a lower mean score might be useful, to get more accurate measurement at the top end.

One possible explanation for requiring assessments to have such a high percentage right for passing is that the point of an exam for the people who make these rules it not to get useful information about the students, but to reassure students about how much they know—to bolster their self-esteem and confidence. Personally, I don’t see a lot of pedagogic value in that.  Students pay more attention to their mistakes than to success on trivial problems, so they are more likely to learn from a difficult test than an easy one.  (I’ve also met too many students who have a very inflated view of their abilities, partly as a result of having never been given anything remotely challenging.)

Skills at the center

This month, a number of the teacher bloggers whose posts I read are participating in a virtual convention, started by Riley Lark.  He introduced a theme for bloggers and is collecting pointers to their posts at the “Convention Center“.  The prompt boils down to “What is at the center of your classroom?”

The current dogma is that classrooms should be “student-centered” (as opposed to “teacher-centered”, which is decried as ultimately evil) and predictably many of the posts reinforce that message.

I’ve been thinking about my courses and I can’t honestly say that they are centered on people at all.  I’m not a “people person”: I have extreme difficulty remembering names or what people look like, and I don’t enjoy small talk or listening to stories about relatives or acquaintances, which seems to be a major pleasure for some people.  So claiming that my courses are either “student-centered” or “teacher-centered” just seems wrong to me—neither is the center of the course.

Mostly I’ve taught adults (grad students and college seniors), though some classes have been for college freshmen and I’ve done a few after-school and summer things for middle-schoolers.  This focus on older students means that development and growth in the traditional sense are not a major goal of my courses, the way they might be for an elementary school teacher. The students coming out of my classes are not transformed in fundamental ways.  Many of them have had 30, 40, or even 50 years to form their personalities and their approaches to life and learning—I may be able to make some small changes in the details, but a 10-week class is not going to turn their lives around.

But I can’t honestly put the content or the curriculum at the center either.  I’ve created and taught too many different courses on a wide range of different topics (VLSI design, technical writing, digital synthesis of music, genome assembly, desktop publishing, bioinformatics, protein design, digital logic, applied discrete math, resource-efficient programming, bicycle transportation engineering, …). Although the content is important to me and I can’t teach a subject unless I know it cold, there are some common threads that run through many of my classes that transcends the specific content of the course.

Looking for those common threads in my 29 years of teaching, I see that I’m mainly interested in students developing skills.  The specific skills vary slightly between courses, but tend to be problem-solving, designing, or writing skills.  I’m not particularly interested in how many facts students learn or how quickly they can recall them, but in how well they apply what they know to new problems.

Different courses have different mixes of skills.  I was trained in mathematics and computer science, so two of the skills that come up again and again are math problem solving and computer programming.  Lectures are often devoted to introducing a skill or particular tools and techniques that the students need learn, followed by students practicing that skill and my providing feedback on their work.

For example, I want students in my genome assembly classes  to apply simple combinatorics to problems like estimating the genome size and figuring out how much sequencing is needed to get the contig lengths they need for finding genes. I don’t want to teach them a formula, but a way of looking at the data and doing back-of-the-envelope calculations (or writing a small program) to estimate what they need to know.

In my bioinformatics class, I want students to be able to write a Python program quickly to find over- and under-represented DNA palindromes in a genome (one of many ways to look for biologically relevant signals).  The specific tasks are not that important (I could replace the DNA palindrome exercise with one that looked for a set of known transcription-factor binding sites, for example), but the skills in writing programs and applying Bayesian probability to biological problems are.

The senior design projects emphasize writing, oral presentation, and debugging of designs (with some time management and group management thrown in, though I’m uncomfortable teaching those, since they are not skills I have mastered).  The bioinformatics course emphasizes programming and applying statistics.  The applied discrete math course emphasized just mathematical problem solving. Many of my grad courses combine various research skills with practice in written and oral presentation.

For senior design and project-based research classes, I end up meeting weekly with individual students (or teams),  working to help them learn to debug their designs or research protocols, as well as providing feedback on written and verbal presentations.  In one senior design course, I made each team start each meeting with a 2-minute summary of what their project was about, so that by the end of the quarter, everyone in the class could give a coherent 2-minute précis of their project without faltering.  It’s a little skill, but a useful one for job and grad school interviews, which most of the students were about to do—we’ve also introduced the 2-minute talk as an annual requirement for all our grad students.

Undoubtedly, an education professor observing my classes could analyze them for adherence to dogma, deciding that they are “student-centered” (for the time spent on feedback on student work) or “teacher-centered” (for the lecture time spent teaching tools), depending what slice of the course they choose to view.  But I think they’d be missing the point—the courses are about developing skills, which only happens with a combination of instruction, practice, and feedback on the practice.

So, my hope is that after one of my courses, students have acquired or improved some of the essential skills that will serve them well in future work.  My assessments of students are primarily intended to determine how well they have developed these skills.  This is not a quick, cheap thing to do.  There are no standardized multiple-choice tests or clicker questions.  To find out if someone can write a 20-page paper or design and implement a program to solve a problem, there is no substitute for having them write the paper or the program, and no substitute for my reading the whole thing closely.  (I have occasionally had a TA to off-load grading to, but it doesn’t help all that much, as they rarely provide a sufficiently detailed critique of writing or programming skills—sometimes because they lack these skills themselves.)

Because I’ve been part of this particular community of teacher bloggers, I’ve done a lot of thinking about standards-based grading (SBG). Unfortunately, that approach to assessment does not seem to work well for courses centered on skills that are not easily decomposed into reductionist standards.  The lack of cheap reassessment undercuts one of the main stays of the SBG approach.

My courses take a lot of my time and a lot of student time, so I can’t recommend my approach to teachers who teach a huge lecture class, who teach many classes at once, or whose students have many other courses, and I’m very afraid that this sort of intensive teaching will be thrown out in the massive budget cuts sweeping through higher education.

 

Laws of grading

Violating the Laws of Grading: A Repeat Offender’s Story is a blog post by a high-school science teacher about the “Laws of Grading” and how her students are resisting standards-based grading, which I have abbreviated here:

Law #1: Law of Right Answers. This law states that, for every right answer, an A shall be given.

Law #2: Law of I Can’t Get a B Because I Always Get an A.

Law #3: Law of GPAs
This one is more of a commandment to teachers rather than a law, and it states that, “Thou shalt not mess with my GPA and ruin my chances of getting into a good college, having a lucrative career and having a fabulous life forever and ever, Amen.”

The post is well written and captures well the frustration of a teacher trying to get students past rote learning.

As a college professor, I was a little offended by Richard Reilly’s comment that “The US college system appears to be a bit of a disaster zone.”  But then I realized that he was referring to admissions process and the circus that has become for high-school juniors and seniors, and I’m afraid I have to agree—though the blame seems to me to sit more with high-school counselors and teachers than with the colleges.  If high schools keep telling all the kids that they should be challenging themselves to apply to Harvard, it is no wonder that Harvard admissions are a circus (in the Roman gladiatorial sense).

Recent posts on standards-based grading

There are  couple of posts on standards-based grading that I’ve been meaning to talk about.

One is a New York Times article by Peg Tyre that presents standards-based grading (in 8th grade math at Ellis Middle School) as a panacea.  “Knowledge grades” are based solely on averaging end-of-unit tests.  There is a separate “life grade” for things like work habits, effort, and citizenship.   While this is a positive change in many ways (no more extra credit for bringing the teacher school supplies), it misses the point of standards-based grading, which usually means splitting the “knowledge” into separately evaluable topics or skills and allowing students to keep working until they have mastered each topic, not just averaging their one-shot end-of-unit tests.

I don’t know whether Ellis Middle School or  Ms. Tyre is abusing the terminology, or whether Ms. Tyre simply left out the important part of Ellis Middle School’s change in order to meet a deadline.  Either way the article is not about standards-based grading, despite the prominent mention of “a new, standards-based grading system.”

More recently, quantumprogress had a post on “Perfectionism and SBG“, in which he (she? I’ll assume male for the rest of this post, as the odds are higher, given that the blogger is a physics teacher) comments on the behavior of perfectionists under SBG scoring schemes in physics classes.

He wonders why a student wants to have perfect scores on all standards going into the test, particularly when the student express it as “I would like to enter the exam having showed understanding on absolutely everything.”  He doesn’t believe it is because the student really wants to understand everything, but relates it to “conceptual consumption” in which ideas are treated like things to be acquired as status symbols (like some bird watchers’ life lists of species sighted).  He also relates the idea to video game achievement lists as described in “Conceptual Consumption and Kicks to the Head” (see also my posts Experience points for classes and Just scoring points).

He idealistically says

I want them to see to see learning as a process, I want them to see that just when you think you understand a concept, you realize how much more there is to understand, and that is a good thing. Precisely for this reason, it’s pointless to put together a list of “intellectual achievements” for you to achieve, …

Personally, I think that the video game designers have tapped into a good way to get engagement, and that we are better off as educators trying to exploit the gaining-points mentality than trying to fight it.  There is significant risk if the point system is poorly designed, both of students forgetting the material as soon as the points have been earned and of not valuing anything that does not have points.  But I believe that addressing those challenges head-on in designing the grading system works better than wringing one’s hands and wishing the students valued learning for its own sake (of course, a few will, and we don’t want to lose them in the point-chasing crowd).

So, as I see the challenge, it is not to magically instill a love of learning in students, but to set up a system in which continuous small rewards keep students attention on the learning and get them to push themselves to do things that previously they would have regarded as either impossible or not worth the effort.  The small rewards need not be “points” or “grades”, but they have to be observable by the students (changes in their knowledge and skills, which outsiders can easily observe, are not really accessible to student perception). Some larger rewards (like the “achievements” lists in some video games) can also be motivating for longer-term commitment.

I think that another point that came up in the comments on Conceptual Consumption and Kicks to the Head is important:

However given the short checkpoints, the challenge never felt insurmountable, and I generally (with a few exceptions) could tell where it was a flaw in my skills or tactics that had let me down—and so the fun was in trying to overcome that.

What do we have as educators that corresponds to the “short checkpoints”? What to do we have that tells students where the flaw in their skills or tactics that let them down, automatically, so that they immediately want to try again?