I spent all day yesterday trying to craft a quiz that would cover all the important topics in the circuits course so far. I feared that it was too long, but I was hoping that students would get half the questions, and that the pattern of errors would show me where students had common misconceptions.

To make the key, my co-instructor, the undergrad tutor, and I each took the quiz ourselves, and I double-checked any answers where we had a discrepancy. I got 29.5/31, my co-instructor got 21/31, and the tutor got 16/31. Their scores tell me that the quiz was too hard (or too time-pressured), as even a professional EE made more than just a couple of arithmetic mistakes. The distribution from the class confirms the problem, with scores ranging from 3/31 to 12/31. I wanted to see a distribution from 10/31 to 25/31.

OK, so I’ve not taught the students some of the things I thought I had taught, and my quiz was both too long and too difficult. Where do I go from here? I can’t fix things right away, as there is one student who was ill and still needs to take the quiz. I’ll probably be able to start correcting the problem on Friday, after this week’s lesson on op amps and the lab.

I can see a few ways forward:

- I could tell them to study and give them another quiz. That would be totally useless, as it would just repeat the problems on this quiz. They don’t know what it is that they need to know, and vague exhortations to study are pointless. I don’t think the problem is lack of effort on their part, and that’s the only problem for which pep talks are a potential solution.
- I could go over the quiz question by question, explaining how I expected students to solve them. This is classic lecture mode and the approach I used to use. It would be easy to do, but I doubt that it would help much. I already did an interactive lecture on the material, and another approach is now needed.
- The students could get the quiz back and be told to go home and look up in their notes and on-line anything they did not get right. They would find and write down the right answers, as if this were homework. (This “quiz correction” is a standard strategy in high school teaching, but not common in college teaching.) One difficulty here is that they might be able to find answers (say by copying from other students in the class) without understanding how to do the problems. It is probably a better approach than yet another lecture, but I’m not sure it will work well enough. If the students were trying to get from 80% understanding to 95%, it might be fine, but to get from 30% to 80%, something more directed is needed. More time and open notes would help, but maybe not enough.
- I could break them into groups and give each group a couple of the problems to work on together in class. This peer instruction technique would be a good one if about 1/2 the students were getting the problems right, but with the top of the class getting only 1/3 right, I may need to give them more guidance than just setting them loose. For example, on some of the problems there was a fundamental misreading of the circuit schematics that was very common. I could clear up that misunderstanding in a minute or so and have them rework the problems that depended on it.
**Then**I could send them home to write correct solutions. - I could give out lots of problem sets to drill them on the material. Of course, since it took me more than all day Sunday to make an 8-question quiz, it would take me forever to generate enough drill problems to be of any use.

I welcome other suggestions that I may not have considered.

On Wednesday I plan to spend about 20 minutes on inverting and non-inverting amplifiers out of op amps, then 20 minutes on a system block diagram for their audio amp lab, then break them into groups (with whoever their lab partner is for Thursday) to do the audio amp design that they will try to wire up and debug in the lab.

I’ll dedicate Friday’s class to group work on a couple of the quiz problems, with 3 groups of 3–4 students either all groups working on the same problem or each group working on a different problem. I’ll have each group report back to the class as a whole after they’ve solved their problem. This is not a teaching style I’ve used before, and I’m curious to find out whether it will work. Next week I’ll try to get in a gnuplot problem-solving session along the same lines, if I can get the students to bring laptops to class. If these methods work, I’ll try introducing them much earlier next year.

Whatever I do for getting them to master the material, I’ll have to give them another quiz in a couple of weeks to show that they really have mastered it.

I was going to suggest a couple of anti-pseudoteaching techniques that I have found helpful, but they were getting unwieldy, so I posted them at my place instead.

Analyzing Mistakes

How to Generate and Grade Differentiated Problem Sets Quickly

Maybe one of these will be of some help.

Comment by Mylène — 2013 February 5 @ 13:51 |

Both of those methods look like good ones, and neither looks immediately usable to me.

Your “differentiated problem sets” relies on having a textbook with solved problems. We’re using purely on-line materials instead of a text, and so I don’t have an easy-to-find problem set. I’m also teaching the material in a different order and with a different emphasis than most circuits courses (putting voltage dividers front and center, rather than Thévenin equivalents), so a lot of the circuits texts would be a bad fit anyway. Finding problem sets for the students would be a good thing for me to do, but I fear I don’t have the time to do that this quarter. I may be able to find or generate some before the offering next Spring, and I might put the time into doing so.

Analyzing mistakes requires that the students be pretty close to getting the concepts, and I think that the quiz results were showing me that even some concepts that I thought all the students had were not nearly as widespread as I had believed. I think I will try something like that, though, after the next quiz, at which time I hope that they’ll be closer to where I thought they should be by now.

Comment by gasstationwithoutpumps — 2013 February 5 @ 17:53 |

I see what you mean. Hm. I’m thinking of the text I use (an algebra-based one) — it introduces voltage dividers long before Thevenin. Possibilities: getting 5-6 of them from the library for students to pass around (or requesting desk copies, or just photocopying the pages that have the voltage-divider problems and their solutions) would allow students to pick out some practice problems. I wonder if access to a few algebra-based (technician level) textbooks would be helpful. I keep a couple of dozen outdated editions and desk copies in my study room for this purpose — available for students to borrow.

The problem sets are helpful mostly for mechanics, increasing fluency, speed, confidence, number sense, etc. But having a grip on that stuff can lower the cognitive load so that they have more processing power available for sense-making, sanity-checking, etc. As for the quiz correction assignment, even when I’ve asked students to do that with problems that are far beyond their “zone of proximal development,” they have been able to glean some insight from it. They may not even learn how to solve the problem — possibly only acquire one new tool or tangle with one set of conflicting ideas, but it’s surprisingly useful even when they are not close to the goal.

Comment by Mylène — 2013 February 5 @ 18:16 |

If quiz correction is useful even when they are far from the goal, I might try it. Not tomorrow (I have to get them to put together a design for Thursday’s lab), but perhaps after I return the quizzes on Friday I can give them a sheet like the one you use (with different questions, since some are quite specific to the way you’ve organized your course). More likely, I’ll handout such a sheet on Monday, so that I have the weekend to work on it, as I need to get the next lab handout written before Friday, and with two science fairs to judge this week I’m a bit short on time.

Comment by gasstationwithoutpumps — 2013 February 5 @ 19:29 |

Can you tell what the failure modes are — did any of them show partial solutions that give some clues about where they’re going astray? It sounds like they are often able to figure things out in the lab, and that makes me think that they have at least some of the pieces in place. I’m curious to know which ones they do and don’t have.

Comment by Mylène — 2013 February 5 @ 19:33 |

I can tell a couple of the failure modes, but I can’t discuss them here until the last student (who has been ill) has taken the quiz. One common failure is a quick fix, but some of the others are either holes in the understanding of the fundamental material of the course or extreme difficulty in reasoning. I’m hoping that peer instruction may help with some of the blocks here, so that the students will be able to construct the chains of reasoning in small groups, even if they can’t yet individually. They seem to have no trouble following the reasoning when it is shown to them, even being able to figure out the next step at each stage, but not when the same problem is presented without a coach.

Comment by gasstationwithoutpumps — 2013 February 5 @ 20:16 |

Right — forgot about the student who hadn’t written yet. Sounds very interesting — especially students who can predict the next step at each stage. Hope you’ll write more when you can.

Comment by Mylène — 2013 February 5 @ 20:22 |

I’m not sure that any student can predict all the “next steps”—but the class as a whole can come up with one student who can do the next step. I think that the problem is that the chains of reasoning are larger than they can hold in their heads, and they haven’t developed either the “chunking” to reduce working memory demands or the “write it down” exploration techniques of math students.

Comment by gasstationwithoutpumps — 2013 February 5 @ 20:38 |

[…] you’re wondering what happened to the 12th day of circuits class, that was discussed in Quiz too long and too hard. On Wednesday, I did some do-now problems to check on some basics and to provide a hook for the […]

Pingback by Thirteenth day of circuit class and first op-amp lab « Gas station without pumps — 2013 February 7 @ 21:02 |

[…] Cross Posted from Gas Stations Without Pumps […]

Pingback by Quiz Too Long and Too Hard | Productive Struggle — 2013 February 11 @ 10:24 |

[…] Quiz too long and too hard, I reported on the first quiz of the circuits course, and how I had made it much too tricky for the […]

Pingback by Quiz 2, better than quiz 1 | Gas station without pumps — 2013 March 2 @ 18:19 |

[…] was really depressed after last year’s quiz, because I had not been expecting such dismal performance. This year I was braced for it, but still […]

Pingback by As expected, students did poorly on the quiz | Gas station without pumps — 2014 April 26 @ 15:43 |