Teaching by hand

I’ve read a tremendous number of blog posts and articles about teaching in the past few months (I have a list of 27 just on MOOCs that I want to blog about).  One article that particularly resonated with me was from the Chronicle of Higher Education, in the column “The Conversation”: Teaching ‘By Hand’ in a Digital Age by Joseph Harris, who teaches writing at Duke.  He gives an anecdote about memorable feedback he got in freshman English, and points out the importance of this direct interaction.  He ends with the following remark:

The key right of any learner is to the attention of his or her teacher. As my friend Eli Goldblatt says, “We teach by hand”—by which I take him to mean that we teach not subjects or courses but individuals. I suspect we still need to figure out how to offer online learners that sort of care and responsiveness.

In my own thinking about the value of university education and what R-1 research universities like the University of California have to offer undergrads, I’ve come to the conclusion that MOOCs and mega-lecture courses are not what a UC education is about.  The value we have to offer is in high-contact courses, where undergrads work closely in labs, seminars, and individual research with faculty and active researchers.  I have been shaping what I do at the University to increase the availability of such high-contact education—the very opposite of the administrative push for more on-line courses.

The applied circuits course I created and taught this year is a good example—I designed and taught the course to center around design experience in the labs, not lecture delivery, exams, or problem sets.  It required me to spend time with the students in the lab to find out what their misconceptions were and what concepts they were having a hard time converting into practice.  It also required me to model debugging and engineering thinking—”try it and see” became a mantra for the class.  I don’t think that even very good MOOCs like MIT’s circuits course provide the necessary interaction with the students that changes the way they think.  The MOOCs may work well for students who already have the desired mindset and just need to learn the theory and do some problem sets, but I don’t see them as having a transformative effect on the students.

The senior thesis seminar I’ve been doing this quarter has a similar high-contact component: I am reading 5 drafts each of 13 senior theses, providing detailed written feedback on each one, plus meeting with each student individually for half an hour each week (to practice 2-minute elevator talks, discuss their writing and their research, and just chat about the job market, grad schools, the undergrad curriculum or other topics that interest them).

While these two courses are valuable, I was reaching students only in their senior year (though the circuits course should get juniors and some sophomores in future years).  That is one reason I helped the Biomedical Engineering Society create a freshman design seminar for next year, so students in their first year would not get lost in the 200–400-student mega-lecture courses, and come away thinking that MOOCs would be a better deal than the University.

Teaching by hand is hard work, time-consuming, and expensive. It does not scale up to 1000s of students the way passive transmission of information through online video does. I doubt that I can convince the UC administrators and my colleagues that this hand-crafted education is the very soul of the university, and that their attempts to gain greater efficiency through larger and larger courses is dooming the University to irrelevance. As long as they are focused more on cash flow and degree completion than on the educational effect on the students, I don’t think that we can even have a meaningful conversation.

I’ll keep teaching these “boutique” courses, though, and will encourage others  to create unique courses of their own.

Reading student writing

I’m spending this quarter reading senior theses: five drafts each of 13 theses.  None of these students are working for me—I’m just running the class in which they are trying to convert what they’ve done for the past year into something resembling a thesis.  About half the class had not written anything on their projects before taking this senior-thesis seminar—a serious dereliction of duty on the part of their faculty supervisors, who should have been requiring a draft at the end of each quarter of work.

I meet with each student weekly (for about half a hour, though I seem to run over more often than not) in addition to the 1 ¾ hour weekly class meeting and all the reading and scrawling on drafts.  I’m currently spending over 14 hours a week on this 2-unit course (my light teaching quarter, as I had 8 units in the Fall and 7 in the Winter), and the amount of time will probably go up as the drafts get longer and more complete.

I know that a lot of MOOC-proponents are pushing automatic grading of papers as a cost-effective way to handle classes with over 1000 students.  Quite frankly, the idea appalls me—I can’t see any way that computer programs could provide anything like useful feedback to students on any sort of writing above the 1st-grade level.  Even spelling checkers (which I insist on students using) do a terrible job, and what passes for grammar checking is ludicrous nonsense.  And spelling and grammar are just the minor surface problems, where the computer has some hope of providing non-negative advice.  But the feedback I’m providing covers lots of other things like the structure of the document, audience assessment, ordering of ideas, flow of sentences within a paragraph, proper topic sentences, design of graphical representation of data, feedback on citations, even suggestions on experiments to try—none of which would be remotely feasible with the very best of artificial intelligence available in the next 10 years.

Providing good feedback on the student theses requires a good understanding of what the students are talking about (which I have gotten mainly from hearing years of research talks by their supervisors, since none are working on subjects within my areas of expertise) plus an understanding of what makes good technical writing.  Either one without the other is nearly useless, which is why students who worked on their thesis drafts as part of a tech writing course last quarter are not much better off than those who didn’t—the tech writing instructor knew none of the content, and so could not see when the ideas were in the wrong order, misstated, or otherwise badly presented. Misuse of jargon and incorrect presentation of data were also missed. The main advantage for the students who wrote a draft for the tech writing course is that they have more complete draft to start from, with a few of the surface errors already removed.

If even expert tech writing instructors with decades of experience can’t produce good enough feedback on student writing, what hope is there that automated programs can do anything useful?

I’m not alone in my thinking that automated feedback on student writing is an incredibly stupid idea. John Warner, in his post 22 Thoughts on Automated Grading of Student Writing, wrote

…
5. I don’t know a single instructor of writing who enjoys grading.

6. At the same time, the only way, and I mean the only way, to develop a relationship with one’s students is to read and respond to their work. Automated grading is supposed to “free” the instructor for other tasks, except there is no more important task. Grading writing, while time-consuming and occasionally unpleasant, is simply the price of doing business.

7. The only motivations for even experimenting [with], let alone embracing, automated grading of student writing are business-related.

…
12. The second most misguided statement in the New York Times article covering the EdX announcement is this from Anant Argawal, “There is a huge value in learning with instant feedback. Students are telling us they learn much better with instant feedback.” This statement is misguided because instant feedback immediately followed by additional student attempts is actually antithetical to everything we know about the writing process. Good writing is almost always the product of reflection and revision. The feedback must be processed, and only then can it be implemented. Writing is not a video game.

…

14. The most misguided statement in the Times article is from Daphne Koller, the founder of Coursera: “It allows students to get immediate feedback on their work, so that learning turns into a game, with students naturally gravitating toward resubmitting the work until they get it right.”

15. I’m sorry, that’s not misguided, it’s just silly.

…22. The purpose of writing is to communicate with an audience. In good conscience, we cannot ask students to write something that will not be read. If we cross this threshold, we may as well simply give up on education. I know that I won’t be involved. Let the software “talk” to software. Leave me out of it.

I pulled out the points that resonated most for me, but I recommend reading the whole of John Warner’s post.

Petition for human readers

I just found out about a petition against the machine scoring of essays at Human Readers (thanks to a blog post on the AAUP blog):

We call for schools, colleges, and educational assessment programs to stop using computer scoring of student essays written during high-stakes tests.

Every year hundreds of thousands of students write essays for large-scale standardized tests. The scores are used in life-changing decisions. Students are accepted into, placed within, and rejected from educational programs. Graduates are hired or not hired. Teachers are qualified, evaluated, promoted, and fired. Learning institutions are compared, accredited, and punished. Yet in a major disservice to all involved, more and more of these essays are scored not by human readers but by machines.

Let’s face the realities of automatic essay scoring. Computers cannot “read.” They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others. Independent and industry studies show that by its nature computerized essay rating is

• trivial, rating essays only on surface features such as word size, topic vocabulary, and essay length
• reductive, handling extended prose written only at a grade-school level
• inaccurate, missing much error in student writing and finding much error where it does not exist
• undiagnostic, correlating hardly at all with subsequent writing performance
• unfair, discriminating against minority groups and second-language writers
• secretive, with testing companies blocking independent research into their products

…

The basic premise of the petition is good: computers can’t score those aspects of writing that people actually care about, and so should not be used for scoring any essays that matter. Of course, some of the alternatives are just about as bad (like the MOOCs that use peer grading by incompetent “peers”), but I signed the petition anyway.

The bottom line is that assessment of writing is difficult, and so is expensive. If you can’t afford to pay for proper evaluation by well-trained human readers, then you can’t afford to use essays as part of an assessment. There are no shortcuts.

We need low-cost colleges

In an opinion piece for the NY Times, My Valuable, Cheap College Degree, Arthur C. Brooks relates an anecdote of his own education: failing out of college his first year, taking a gap decade, going back to school through low-cost correspondence courses from various state universities,  eventually getting a PhD, and becoming a professor and president of a “Washington research organization”.  (I’d be more impressed if I thought he meant a biotech or computer research job in Seattle, but I fear he means a political hack job in DC.)

His anecdote sounds like a classic “self-made man” story, but is actually about the substantial state subsidies for college education that were available 40 years ago, and that are fast disappearing under the current political belief that government should pay for nothing but guns and people to use them (the state budget in California is heavily weighted toward prison guards and police).

I’m in basic agreement that college has gotten ridiculously expensive—I would welcome a low-cost college education for my son like the one I got back in the 1970s.  Unfortunately, I don’t expect to see state governments coming to their senses in time, and I may have to send my son to a private college to get an adequate education.  (Currently his short list of about 20 schools is almost equally divided between public and private colleges and universities, but most of the public universities are out of state, raising their costs to the same level as the private ones.)

I expect that my son’s education is likely to cost me 2 or 3 years of my salary as a full professor (and that’s pre-tax salary). I’ve been saving 10% of my salary each year for the past 16 years in order to be able to pay for that education for my son, so we should be able to do it without his going into debt, which is good, since I’ll probably be retiring while he’s still in grad school.

The fact that the ratio of college tuition and fees to professorial salary is so high tells me that the cost is not due to high salaries for faculty (and I’m one of the lucky tenured faculty—most colleges are dominantly hiring “contingent” faculty now, who have no job security, low pay, and no voice in the running of the institution).  Other studies have show that the cost of public universities has not changed much—it’s just been shifted from the state to the student. In particular, the socially desirable subsidy of tuition for low-income students is almost entirely born by “return to aid” funding from other students, not from endowments or state subsidies.

Private colleges have had enormous increases in costs: largely related to the debt acquired in the amenities wars and the obscene salaries they pay their administrators and football coaches (though not their professors, who are only slightly better paid than public-university professors like me).

I don’t see online education as being a major cost saver in providing decent quality education.  Massive lecture classes are the least effective courses at the university (though relatively cheap to offer) and MOOCs just amplify the problem.  Those completing MOOCs are mostly the older, highly motivated students who have always been fairly easy to teach—there’s not much evidence that MOOCs are at all effective at educating the run-of-the-mill students who are the current main target of college education efforts.

Online education for continuing education

Tom Katsouleas, Dean of Duke University’s Pratt School of Engineering, provided a very upbeat view of online courses (particularly MOOCS) in a blog post Who says online courses will cause the death of universities?.  He makes some assertions without evidence that I find a bit dubious:

The bottom line, though, is that while online education poses a challenge for universities, they will ultimately improve them.

…

And by moving lecturing online, MOOCs allow in-person time to be more interactive, dynamic, and valuable.

…

Despite the challenges to universities posed by MOOCs, there are great advantages to them as well. And the best universities will be able to capitalize on those advantages to provide the best value for their students – whether that value is online or in person. Despite what some see as a threat to higher education, MOOCs will only help it get better.

This is essential the administrative view of online courses: that they are new and cheap and must therefore be for the good of higher education.

I think that Dr. Katsouleas is right in asserting that the continuing education market for engineers is one of the places where universities will first see the effect of MOOCS, both because there are more MOOCs and more advanced MOOCs available in engineering than in other fields, and because many engineers seek continuing education even without certification.  I don’t think that the MOOCs will reduce MS degree seekers much, because there is still a very large premium for the MS certification in engineering, and the engineering MOOCs available still only go up to junior-level courses.  There are already many MS-level courses offered online, and I’m sure we’ll see an increase in them, but the market for them is too small for a MOOC, so there isn’t the advertising-for-the-provider benefit which drives most of the MOOCs currently, and we’re more likely to see the standard pay-per-course fee structure for them.

I think that we will see some colleges and universities starting to accept credits from MOOCs for undergrad courses, once the problems with cheating and low-quality assessment are fixed.