Gas station without pumps

2017 May 18

Midterm quiz doesn’t tell me much new

Filed under: Circuits course — gasstationwithoutpumps @ 09:53
Tags: , ,

I don’t usually give exams in my courses any more, because I’m more interested in what students can do when they have time and resources than what they can do on toy problems under resource limitations.  But if students don’t do the homework, then they don’t learn the material, so I threaten each class that if too many students don’t turn in the homework, I’ll have to add a quiz (worth as much as one of the lab reports, each of which is equal to all the homework) to the course.

This quarter I had to follow through on that threat, because 12% of the class had turned in half or less of the homework (and by that, I don’t mean answered half the questions—I mean turned in nothing at all for half the assignments).  A quarter of the class had not turned in 25% or more of the assignments.

I gave the quiz yesterday, with 6 easy questions that only tested the very basic material: single-pole RC filters (passive and active) and negative-feedback amplifiers.  I told students ahead of time (and on the exam) that they could use the Bode approximations (the straight-line approximations to the gain of the RC filters) and we even reviewed them in class last week.  There were 60 points possible on the test, and none of the questions were design questions—they were almost all of the form “what is the corner frequency?” or “what is the gain of this circuit?”.

There are a small number of students in the class whose probity I have reason to question, so I took steps to reduce cheating that I would not normally bother with: I made up two versions of the test (same schematics, but different component values) and alternated them in the piles passed along each row.  I also had the students sit in different rows from usual, reversing front and back of the room, with the front row reserved for latecomers. I’ve noticed a high correlation between good homework grades and people being on-time and in the first two rows, so I had those students sit in the back row, where no one would be able to copy from them.

I normally figure that a test is appropriately long if an expert can do it in about a quarter of the time allotted.  So I made up the keys for the test while the students were taking it.  Working through one form with the Bode approximations took about 5 minutes.  Doing exact computation with the formulas for series and parallel impedances and complex numbers using only real-number arithmetic on my calculator extended that by another 15 minutes.  The students had 63 minutes, so the exam was too easy if the students used the Bode approximations (as they were told) but a little too hard if they worked just from the fundamentals of complex impedance and negative-feedback amplifiers.  As a consequence, I decided to give bonus points for exact computations of the gains that didn’t use the Bode approximations, though the class was not informed of this bonus, because I didn’t want them to waste time on the tiny bonus.  (The differences in answers were small, because I had deliberately asked for gains only at points well away from the corner frequency, so that the Bode approximations would be good.)

Even if students really didn’t understand complex impedance or RC filters, 39 of the 60 points could be earned with just DC analysis of the negative-feedback amplifiers and knowing that capacitors don’t conduct DC.   So I was hoping that students would do better on these very easy questions than they did on the harder design questions of the homework.  As a confirmed pessimist, though, I expected that students would show almost exactly the same distribution on the test that they showed on the homework, with the middle of the class being around 20 out of 60 points and showing serious misunderstandings of almost everything, with a long tail out to one or two students who would get almost everything right.  I also expected that the correlation between the homework scores and the quiz scores would be high.

So what happened?  First, I saw no evidence of any cheating (not that I had expected any), so that is one worry removed.  Second, my pessimistic assumption that students really were not learning stuff that they had done many times in homework and in lab was confirmed:

Here is a stem-and-leaf plot of the scores:

OO: 3
05: 6889
10: 011112444444
15: 555667777899
20: 00111112223344
25: 677999
30: 12224
35: 5678
40: 00444
45: 67
50: 01
55: 
60: 2

The median is indeed 21 out of 60, as I feared. At least no one got a zero, though the scores at bottom indicated complete failure to apply the basics of the course.

Most students could compute a corner frequency from a resistor and capacitor, but few had any idea what to do with that corner frequency. Many students could compute the DC gain of a non-inverting amplifier, though many could not then apply this knowledge to the DC gain of an active filter (which only requires replacing the capacitors with open circuits). A lot of students forgot the “+1” in the formula of the gain for the non-inverting amplifier.

Inverting amplifiers were even less understood than non-inverting ones, with students forgetting the minus sign or trying to use the formula for non-inverting amplifiers.

A lot of student answers failed simple sanity checks (students were having passive RC filters with gain greater than 1, for example).

Very few students used the Bode approximations correctly, and many tried the exact solution but either couldn’t set up the formulas correctly or couldn’t figure out how to use their calculators, often getting numbers that were way, way off.  Others seem to have ignored the complex numbers and treat x+jy as if it were x+y.

One disturbing result was how many students failed to recognize or understand a circuit that they have designed in three different labs: a voltage divider and unity-gain buffer to generate Vref, combined with a non-inverting amplifier. I asked for the output voltage as a function of the input voltage (both clearly labeled on the schematic). This was intended to be almost free points for them, since they had used that circuit so many times, and the formula they needed was one of the few formulas on the study sheet: \frac{V_{out}-V_{ref}}{V_{in}-V_{ref}} = 1 + Z_{f}/Z_{i} . The frequent failure to be able to fill in the blanks of this formula for a circuit that they have used several times in design makes me question whether the students are actually learning anything in the course, or if they are simply copying designs from other students without understanding a thing. (Note: the extremely poor performance and group-think duplication of ludicrously wrong answers on pre-lab homework this year has also lead me to the same question.)

Did the quiz tell me anything that the homework had not already told me? Here is the scatter diagram:

Pearson’s r correlation is 0.539 and Kendall’s tau is 0.306, so the homework and quiz scores are highly correlated. There are a few outliers: a diligent student who bombed the quiz and a student who has turned in few of the homeworks who actually understands at least the easy material. The points have a small amount of noise added, so that duplicate points are visible.

The high correlation between the quiz and the homework mostly confirmed my prior belief that the quiz would not tell me much that is new, and that the homework grades were pretty reflective of what students had learned. I will want to talk with a few of the most extreme outliers, to find out what happened (why were students who mostly understood the material blowing off the homework? and why did diligent students who had been doing moderately well on the homework bomb the quiz—is there undiagnosed test anxiety that should be getting accommodations, for example?).

Most of the points that were earned were from students randomly plugging numbers into a memorized formula and (perhaps accidentally) having chosen the right formula and the right numbers. Only a few students showed real understanding of what they were doing, and only one student saw the quiz as the trivial exercise it was intended to be.

It seems that the hands-on active learning that I have been so enthusiastic about is not working any better at getting students to learn the basics than the traditional (and much cheaper) droning lecture that EE uses. I’m not in complete despair about the course, as there is some evidence that students have picked up some lab skills (using oscilloscopes, multimeters, soldering irons, …) and some writing skills (though many are still not writing at a college level). But I’m trying to teach the students to be engineers, not technicians, so I was aiming at them understanding how to design and debug things, not just implementing other people’s designs. Picking up lab skills is not enough for the course.

I need help. How do I reach the lower half of the class? How do I get them to think about simple electronics instead of randomly applying half-remembered formulas? We’ve only got 3 weeks left—I don’t know how much I can salvage for this cohort, but I certainly would like better outcomes next year.

2014 May 7

Quiz corrections

Filed under: Circuits course — gasstationwithoutpumps @ 20:36
Tags: , , , , , ,

As I reported last week, students did poorly on the first quiz, which came as no surprise to me.  I had the students redo the quizzes as homework, allowing collaborative work (as long as they acknowledged the collaboration in writing).  They turned in the homework on Monday, a week after the quiz, and I returned them today.  No one aced the redo, with the top score being still only 25/33 (which would have been an A on the first pass, on a redo maybe a B+).

A lot of the students still seem to be having trouble with complex numbers—they got the formulas right when working symbolically, but then the exact same question with numbers instead of letters (which could be done by just plugging into the formulas) came out with real numbers when complex impedances were asked for.  Also, a lot of sanity checked were skipped (several people reported a battery as doubling in voltage when hooked up to a resistor, for example).

These students are not major mathphobes (they’ve all passed a couple of calculus classes and most have done more math past that), but they don’t seem to have any sense for reasoning with or about math—they just want to plug in and grind, even on simple problems like ratios in voltage dividers. This class has almost no memory work (I gave them a one-page handout at the beginning of the year with all the math and physics I was expecting them to memorize), but relies heavily on their being able to recognize how to apply those few facts.  This often requires subdividing a problem, like recognizing that a Wheatstone bridge is the difference between two voltage dividers, or that a 10× oscilloscope probe is a voltage divider with R||C circuits for each of the two impedances.

I spent the entire class today working through each problem in the quiz, to make sure that everyone in the class could understand the solution, and (more importantly) see that they did actually have enough knowledge and math skill to do the questions. Some of the students were feeling overwhelmed on the quiz, because they are not used to doing anything more than 1-step pattern matching for problems, and some of the quiz problems required two steps.  None of the quiz problems were as hard as the prelab they had to do this week, which involved 8 or more steps to get the resistor values to set the gain of the amplifier:

  1. Determine the pressure level of 60dB sound in Pa.
  2. Determine the sensitivity of the microphone in A/Pa:
    1. Convert -44dB from spec sheet to a ratio
    2. Get V/Pa sensitivity for microphone for circuit on spec sheet
    3. Convert to A/Pa given resistance of I-to-V conversion resistor on spec sheet.
  3. Determine voltages needed for op amp power supply.
  4. Determine I-to-V resistor needed to bias microphone in saturation region.
  5. Convert A/Pa sensitivity, RMS pressure level, and I-to-V resistor to RMS voltage out of microphone.
  6. Determine corner frequency and R, C values for DC-blocking filter.
  7. Determine maximum output voltage range of the amplifier as the most limiting of
    1. Voltage range of op amp outputs
    2. Power limits of loudspeaker (10W)
    3. Current limit of op amp (which is a function of the power-supply voltage) into 8Ω loudspeaker
  8. Determine max gain as ratio of RMS voltage into op amp and RMS voltage out of op amp (I’m allowing them to be a bit sloppy about RMS voltage vs amplitude, since we are not looking just at sine waves—the amplitude of a symmetric square wave is the same as the RMS voltage.)
  9. Choose resistor values to give the desired gain.

I’m hoping that pushing them go through these multi-step designs in the lab will give them more practice at decomposing problems into smaller pieces, so that two-step problems on a quiz no longer seem daunting, but routine.

I’m going to be giving them another quiz in about a week, covering op-amp basics and the amplitude response of RC filters.  I’ve got to figure out the best time to do this—possibly a week from Friday, after they’ve done another op-amp lab (using a phototransistor to make a pulse monitor, using this handout).  I think I’ll reorder the labs after that, doing the pressure sensor instrumentation amp lab, then the class D power amp, then the EKG.

 

 

2014 April 29

Inductors and loudspeakers

Filed under: Circuits course — gasstationwithoutpumps @ 19:22
Tags: , , ,

On Monday I gave a little pep talk about the quiz before returning them.  I also assigned the students to redo the entire quiz  as homework due Friday, saying that we’d already gone over most of the material in class, so there wasn’t much point in my showing them again—they needed to do it themselves.  (Engineering is not a spectator sport!)

After returning the stuff I’d graded over the weekend, I talked to them about inductors doing a rather hand-wavy derivation of V = L \frac{dI}{dt}.  There is much more detail in this week’s lab handout, but I’ve found that students in the circuits class do not seem to be able to absorb much information in written form—a shame really, since that is how most of their future learning is going to have to happen.  I fear that most of them are going stop learning the moment they leave college, and then they’ll be stuck with obsolete knowledge and no way to remedy the problem within five years.

I also talked about loudspeakers: how they work and what the impedance vs. frequency curves look like.  They’ll be gathering data for their own loudspeakers today, so I wanted them to be aware of the existence of the resonance peak and the need to gather a lot of data around the peak in order to model it.

I did talk to them about the basic R+ j\omega L curve for any inductor (with R due to the resistance of the wire), and about the resonance peak from the mass+spring harmonic oscillator that is the voice coil, cone, and suspension.  I derived the frequency of an L||C resonant circuit (by computing the impedance and seeing where it went to infinity), and gave  a rather hand-wavy explanation of the effect of adding a resistor in parallel.

2014 April 26

As expected, students did poorly on the quiz

Filed under: Circuits course — gasstationwithoutpumps @ 15:43
Tags: , ,

I gave about the same quiz as I did last year,  changing the numbers, removing one of the harder questions, and making sure that some of the other questions reflected worked examples we had done in class. The quiz was again on the 12th day of instruction. I had intended to move it to the 10th day, but one of the students was called out of town, so I rescheduled it so that everyone could take it at the same time.

I expected similar distribution to last year’s (last year the range was 3/32 to 12/32), but was hoping for slightly better.  I saw a distinct bimodal distribution this year, with half the class getting scores from 0/33 to 6/33 and the other half getting 11/33 or 12/33. This is a little clearer distribution than last year’s, which spread the students out more uniformly. I was still hoping that some of the better students would get over half the points on the quiz, but they seemed to top out at 36%.

I worked this year’s quiz myself in about 24 minutes (which means the quiz was a little too long still—I want about a 3:1 ratio on time, and the students had only 70 minutes).

I was really depressed after last year’s quiz, because I had not been expecting such dismal performance. This year I was braced for it, but still hoping for better.  Still there were some surprises:

  • There were a few questions that should have been free points (like asking for the impedance of a resistor with resistance R)—I was disappointed that some students missed even the trivial questions.
  • I had a pair of questions which were identical, except that one asked for algebraic formulas for impedance and the other gave component values and asked for numbers. I put the algebraic ones first this year, so the numeric ones were just a matter of plugging the numbers into the algebraic ones (and doing a sanity check).  The algebraic ones had a mean score of 2/4 with a standard deviation of 1.2, while the numeric ones had a mean of 1.22/4 and a standard deviation of 1.2.  I had not expected a drop in performance on the numeric ones, since the received wisdom in the physics education community is that students do better with numeric examples than algebraic ones.
  • No one got any points on the oscilloscope probe example, even though it was identical to an example we had worked in class.
  • The average score on a load-line problem was 1/6 with a standard deviation of 1.3.  This did not look like a normal distribution, but an exponential one, with half the class getting no points.
  • I had two low-pass RC filter questions. One asked for algebraic formulas; the other used the same circuit but asked for numeric answers using specific component values, voltages, and frequencies. The algebraic one was bimodal, with 2/3 of the class getting 0 and 1/3 getting the answers completely right. The numeric one was significantly worse, with only 2 out of 9 students getting any points (1/6 and 3/6).
  • I asked a couple of voltage divider questions that required applying the voltage divider formula circuits in which the voltmeter was connected between two nodes, neither of which was ground.  One asked for an algebraic results (a Wheatstone bridge), the other for a numeric result (voltage across the middle resistor of three in series. Students did very poorly on both,  with only one person getting the voltage for the middle resistor (one got half credit for setting it up right, but computing wrong), and no one getting more than 1/5 for the Wheatstone bridge.

Last year I suggested several ways to handle the poor performance on the first quiz:

  1. I could tell them to study and give them another quiz.  That would be totally useless, as it would just repeat the problems on this quiz.  They don’t know what it is that they need to know, and vague exhortations to study are pointless.  I don’t think the problem is lack of effort on their part, and that’s the only problem for which pep talks are a potential solution.
  2. I could go over the quiz question by question, explaining how I expected students to solve them.  This is classic lecture mode and the approach I used to use. It would be easy to do, but I doubt that it would help much.  I already did an interactive lecture on the material, and another approach is now needed.
  3. The students could get the quiz back and be told to go home and look up in their notes and on-line anything they did not get right.  They would find and write down the right answers, as if this were homework.  (This “quiz correction” is a standard strategy in high school teaching, but not common in college teaching.)  One difficulty here is that they might be able to find answers (say by copying from other students in the class) without understanding how to do the problems.  It is probably a better approach than yet another lecture, but I’m not sure it will work well enough.  If the students were trying to get from 80% understanding to 95%, it might be fine, but to get from 30% to 80%, something more directed is needed.  More time and open notes would help, but maybe not enough.
  4. I could break them into groups and give each group a couple of the problems to work on together in class. This peer instruction technique would be a good one if about 1/2 the students were getting the problems right, but with the top of the class getting only 1/3 right, I may need to give them more guidance than just setting them loose.  For example, on some of the problems there was a fundamental misreading of the circuit schematics that was very common. I could clear up that misunderstanding in a minute or so and have them rework the problems that depended on it.  Then I could send them home to write correct solutions.
  5. I could give out lots of problem sets to drill them on the material.  Of course, since it took me more than all day Sunday to make an 8-question quiz, it would take me forever to generate enough drill problems to be of any use.

I feel the same way this year about the possible teaching strategies, but this year I’m going to try a mix of methods 3 and 4, asking them to redo the quizzes at home, working with others until they are satisfied that they can now do the problems and other similar problems when asked.  I’ll have them hand it in this year as a homework, but not go over it in class until after they turn it in.  They need to take a more active role in trying to master the material, and not rely so much on my telling them what to do.

Monday we’ll cover inductors and loudspeakers, in preparation for the Tuesday measurement lab.

On Wednesday I was planning to do gnuplot analysis of the loudspeaker data, but I think I’ll keep that fairly short, so that we can get an intro to sampling and aliasing also before Thursday’s lab.  I have to decide whether to bring in my son’s stroboscope and a moving object to demonstrate aliasing.

Friday, I’ll introduce op amps, with the intent of developing the block diagram in class on Monday for a simple op amp microphone circuit for the Tuesday lab.  This weekend I need to rewrite that lab from last year—I decided last year to use the dual power supply with a center ground for their first op-amp design, rather than having them build a virtual ground (we’ll get that in the next lab assignment).

 

 

2013 March 2

Quiz 2, better than quiz 1

Filed under: Circuits course — gasstationwithoutpumps @ 18:19
Tags: , , , ,

In Quiz too long and too hard, I reported on the first quiz of the circuits course, and how I had made it much too tricky for the course.

Yesterday I gave another quiz covering the material of the first quiz (impedance, voltage dividers, low-pass and high-pass RC filters) plus some new material on inverting and non-inverting amplifiers using op amps.  I also asked a fairly big design question that ended up being worth about 40% of the points:

Design a circuit that takes an AC input signal with a frequency above 1Hz and below 50kHz that is centered at about 1v and has a swing of ±100mV, and provides an in-phase AC output signal that is centered at 2.5V and has a swing of ±1V. (Note: “in-phase” means that the signals move up and down together.) Draw a block diagram of your design and a schematic.  You may assume that you have an external 5V power supply.

I was aiming for a test that would have a mean of 50% and a standard deviation of 20%, for a “typical” class. (Of course, given that this is the first time I’ve taught the class, I still have no calibration for what a typical class should look like, so I’m basing my estimates on where I think the current class should be, given what we have covered). I knew from taking the quiz at the same time as the students that it was a bit too long. It took me 26 minutes to do all the problems, and they had 70 minutes. I usually figure that I should be able to do their 70-minute quizzes or exams in 20 minutes, so the quiz was about 30% too long. The group tutor for the class also took the quiz, and I graded both his quiz and mine before grading the class. I gave myself 95% (I’d done a divide by 2π instead of multiply by 2π in one place, and my schematic was missing some labels and a capacitor to keep the virtual ground low-noise). I gave the group tutor 65%, which was a little lower than I expected from him (though, of course, he has not been studying for this course, but just relying on what he has learned in previous courses). His score was still higher than anyone taking the class got.

I just finished grading the quiz, and the mean was 38.55 with a (sample) standard deviation of 14.6 (out of 100 points), range 13–60, median 35. I had to be fairly generous in my partial credit for the design problem, looking for pieces of the circuit that made sense, as no one got the entire design right. A lot of the block diagrams were nonsense, looking more like random collections of ideas than a block diagram (the people who teach “mind mapping” have a lot to answer for). If we correct for the quiz being 30% too long, it looks like the difficulty level was about where I intended it to be, at least with the generous grading.

I’ll have to assign letter grades to everyone on this quiz (I didn’t for the first quiz—since I decided not to count that quiz after I saw how poorly everyone did on it), and I’m not sure where to make the cut points. The highest scorer in the class was still only what I’d consider a B+ performance—the design problem was not bad (about 2/3 credit), but there were a lot of errors in the earlier, more straight-forward problems. At the low end, I need to decide whether then 13 is a low pass or not passing, and what grade to give the second lowest score—it was clearly passing, but at what level?

[Correction 2013 Mar 3: I regraded some of the quizzes, looking to see if there was any partial credit I had overlooked on the first pass.  This made a big difference at the bottom of the distribution, changing the range to 22–60,  µ=39.45,  σ=13.1.  With those changes, everyone passes.]

I had graded the quizzes a problem at a time, not looking at the names of the students, so I was curious to see how students ranked compared to how they’ve been doing on the lab reports. As it turned out, I was not surprised by the identities of the top and bottom scorers, but some of the other ranks surprised me a bit, with one of the students whose lab reports were generally poor being near the top of the class on the quiz, and one whose lab reports where generally fairly good near the bottom.

Next Page »

Create a free website or blog at WordPress.com.

%d bloggers like this: