Gas station without pumps

2014 June 2

What makes an award-winning senior project?

Filed under: Uncategorized — gasstationwithoutpumps @ 16:39
Tags: , ,

I was on an awards committee for the School of Engineering this year, attempting to give awards to the best undergraduate projects.  The process is interesting, because the projects span a wide range of different disciplines and levels of sophistication.  We had faculty as judges whose fields were computer science, computer engineering,  biomolecular engineering, and technology management (no EE faculty this year). I was the “biomolecular” judge, though my training was in math and computer science and I taught computer engineering for over a decade before switching to biomolecular engineering.  I was probably the only one of the judges who was at least somewhat familiar with all the fields represented by the projects, which is the first main point:

An award-winning project has to be comprehensible to someone outside the group.
Not only does the project description need to state the design goal (or research question) clearly, but it also needs to provide a justification for why anyone would care. Several of the project descriptions seem to have been written solely for the head of the lab with no material to help someone from outside the lab understand what was going on. These were not award-winning—they may have been good research, but the presentation did not make that clear. The more esoteric the research, the clearer the statement to outsiders needs to be.

A good heuristic for student reports (at all levels from freshman essays to PhD theses) is to write to an audience of people who might be interested in joining the research group, but aren’t yet in that field or subfield. So a senior design report should be written with juniors in the same major as the main audience, with a somewhat more general intro and conclusion. If you learned something in the process of doing the project, you can’t assume that your audience has already learned it.

 

For some of the projects, I had a hard time figuring out what the students were trying to do and what they had actually done, which brings us to the next three points:

An award-winning project description starts with a simple statement of the design goal or research question.
I don’t want to read three or four pages of background about a field before finding out what the project is. Start with a simple statement of the problem in one or two paragraphs, then give the background, justification, or work by others needed to put it in context.
A thesis or student project report exists primarily to establish what the students have done.
I get tired of reading reports full of passive voice, in which things happen, but no one does things. I want to know who did what: what the students did, what was done by other lab members, what was purchased, what was done by collaborators elsewhere, and so on. For a senior thesis, which is a single-author work, there should be much more “I” than “we”, and the plural should only be used when the other people involved have been explicitly and unambiguously named.

A thesis is not about science or engineering, but about the particular contributions of a specific person to science or engineering. A senior design report may be a team effort (and so “we” may be more appropriate), but it is still mainly about what the team accomplished, not about the product they produced.

A project report must be specific and detailed.
In general, a thesis provided more detailed information than a journal paper, which in turn was more detailed than a poster, which was often more detailed than a set of slides. We tended to favor more detailed reports (whether single-author or multiple-author) over shorter summaries, and complete reports are better than proposals. Since the deadline for awards is earlier than the end-of-the-year deadline for design reports and theses, there is an enormous advantage to students who write as they do the work, rather than pushing the writing to the end of the project. (Students who write as they go also generally produce better writing and do better engineering, because they are not letting themselves get away with fuzzy thinking, but making sure that they can explain themselves every step of the way.)

 

Of course, no project can win an award unless good work is done, which brings us to the last point:

Award-winning work is carefully done as well as carefully described.
Engineering awards are given for good engineering, not for sloppy work full of obvious errors, nor for proposals that five minutes’ thought would show have no hope of succeeding. The judges may not be in your field, but they are engineers and they pay attention to details. There were several projects that I saw this year which raised red flags, and I spent a little time on Google searches or Wikipedia to check to see whether what the students said was reasonable. Occasionally, I found that I had a misunderstanding of a subject and I re-read the report with a corrected knowledge base. More often, I found that students hadn’t done even simple sanity checks that are apparent to people well outside the field.

Although I said above that a report is “mainly about what the team accomplished, not about the product they produced,” it is still essential to have a good description of the product, with detailed block diagrams, schematics, program pseudocode, or whatever other documentation is needed to communicate the design. The goal should be to have a report that could be handed over to a new team, who could then continue the project without much delay.

2013 March 15

Judging at state again this year

Filed under: Science fair — gasstationwithoutpumps @ 00:17
Tags: , ,

I’ve volunteered to judge at the California State Science Fair again this year, since I need to travel down to Los Angeles anyway to accompany my son, who will once again be going to state (this makes his 6th year in a row).  I didn’t find out from the county judges (even though I was a judge in the elementary and middle-school divisions at county), nor from the awards ceremony, which isn’t until next Monday, but from the public posting of those eligible to register for the state science fair on the California State Science Fair site.  Given that the deadline for Santa Cruz students to register for the state science fair is 2013 March 21, only 3 days after the 2013 March 18 awards ceremony, and that students need to edit their abstracts from the county’s overly rigid format to the state format, I believe that the students going on to state should be informed as soon as possible, not waiting over a week for the awards ceremony.

Santa Cruz County once again got allocated 40 projects at state, based mainly on historical levels of awards.  As I understand the allocation formula, it is now based on the average number of awards at state for the preceding 5 years, with a minimum of 6 for each affiliated fair, even if they have never produced an award-winning project.  The large number of projects allocated for Santa Cruz County (despite our relatively small population) is a testimony to the average quality of the ones we send to state.

I copied the list of names on the state web site, and sorted them into projects at the county level, and I count 44 projects, not 40, so I suspect that this list includes 4 alternates—I don’t know which four. It’s possible that my son is just an alternate (which would be a shame, because I made non-refundable motel reservations earlier today).

Derecho, Devin                Junior    Behavioral and Social Sciences
Hayes, Anika                    Junior    Behavioral and Social Sciences
Marsh, Sam                       Junior    Botany
Gwiazda, Gina                   Junior    Botany
Parsa, Sophie and Rorty, Ruby    Junior    Botany
Clarkson, Maya                   Junior    Chemistry
Failor, Phoebe                    Junior    Chemistry
Hume, Bristol and Sierra, Ava    Junior    Chemistry
Chandiramani, Rishi                Junior    Cognitive Science
Scott-Curtis, Liam              Junior    Cognitive Science
Kvaternik, Jaime                Junior    Earth Science
Mitchell, Jacob                   Junior    Earth Science
Freedman, Max                  Junior    Energy and Power
Ortiz, Chloe                         Junior    Energy and Power
Popilsky, Tovah                  Junior    Energy and Power
Bjorklund, Eli and Hite, Parker    Junior    Environmental Engineering
Kent, Madelyn                    Junior    Environmental Engineering
Donohoe, Jack                    Junior    Environmental Science
Freedman, Emma               Junior    Environmental Science
Taylor, Gemma                    Junior    Environmental Science
Hanlon, Jeremy                   Junior    Math and Software
Patz, Sara                              Junior    Math and Software
Bernstein, Talia                   Junior    Medicine and Health
Tschirky, Chloe                    Junior    Medicine and Health
Silverglate, Steven               Junior    Physics and Astronomy
von Oepen, Marc                 Junior    Physics and Astronomy
Schaefer-Whittall, Emma            Junior    Zoology
Spence, Ferryn                    Junior    Zoology
Webb, Audrey                     Junior    Zoology

Glum, Anthony and Gonzalez, Rene and Ortiz, Daniel    Senior    Botany
Maxwell, Anna and Weigel, Adela    Senior    Botany
Hernandez, Emily                Senior    Botany
Gallagher, Natalie and Lydon, Connor    Senior    Earth Science
Garcia, Cesar                    Senior    Environmental Science
Prambs, Johann              Senior    Environmental Science
Dong, Kevin                     Senior    Math and Software
Karplus, Abraham          Senior    Math and Software
Loehde-Woolard, Hailey            Senior    Chemistry
Miller, Adrian and Sanchez, Michaela    Senior    Microbiology
Torres, Isabel                   Senior    Microbiology
Yerena, Maria                  Senior    Microbiology
Pogson, Angela                Senior    Zoology
Pouls, Jazz                        Senior    Zoology
Manier, Saige                   Senior    Zoology

2013 March 10

Santa Cruz County Science Fair 2013

I spent Friday evening and all day Saturday judging at the Santa Cruz County Science Fair, which is always fun, but a little tiring.  This year I was the lead judge for the “Energy and Power” category, which had 14 projects in grades 4–5 and 14 in grades 6–8.  There were no high school projects in my category, and they decided to have interviews but not judging for K–3, so I ended up only talking briefly with the K–3 students and did not give them written feedback.  I interviewed 26 or 27 of the students in my category, and provided written feedback for each of them.  That written feedback is the most important part of the fair, and the judges in my category were all very diligent about providing detailed feedback, so most of the kids got 4 or 5 feedback forms.  In some other categories, a lot of the judges left without providing feedback, and a few kids ended up with no feedback forms. (I heard about it from some of the parents, because the administrator had left before the public viewing, and I was clearly identifiable as a judge—I wear a lab coat for judging science fair.)

The “energy and power” category is where all the lemon batteries end up, which makes it a rather sad category for judges.  Every category has a few projects that appear (usually very badly done) year after year. The lemon batteries are almost always terrible projects, with the students following rote directions from the web (in at least two cases this year, incorrectly) and having no understanding what they are doing.  I think that Science Buddies has a lot to answer for! The students seem to think that the power is coming from the fruit (rather than from the dissimilar metals) and that voltage is the same thing as power.

We also got the windmills, solar cells, wave generators, and thermoelectric devices. Those were generally a little bit better done—we actually had a pretty good solar cell project and a pretty good Peltier-device project. Because our fair does not have an engineering category (other than “environmental engineering”), we ended up with a number of the engineering projects as well (hovercrafts, ducted propellers, and the like).

There is a big need to train elementary school teachers (and to a lesser extent middle-school teachers) in science and engineering methods.  And I don’t mean the nonsense they teach about the “scientific method”, which bears almost no resemblance to any process of scientific or engineering work I’ve ever seen.  I mean that they need to know how to measure voltage, current, and resistance, and to be able to show kids how to compute power (it is not the same thing as voltage, nor is it the product of open-circuit voltage and short-circuit current).  Teachers should be able to show students how to build a simple calorimeter and measure energy from chemical reactions (like burning fuel). A lot of the students I interviewed were quite bright, but no one had ever taught them the basics they needed to be able to do their projects.  Nor have they been taught how to use the tools they have. I don’t want to see another student wrapping the loop for measuring AC current around a wire and claiming that they are measuring resistance, nor claims that lemon batteries produced 9 Amps at 1v.

Things I learned when I was 8–10 years old should be within reach of their teachers. I think that a few hours of professional development that involved them actually doing some measurements and learning the basics of some of the science and engineering projects would improve the quality of their students projects a lot. Every elementary school teacher should know how to use a hand saw, a drill, wire strippers, and a soldering iron, and they should be teaching the kids how to use them also.  (Yes, I can see the safety problems if you try to do it in a large class—but the safety problems in PE classes are far larger, but we haven’t thrown out all sports in schools because of it.)

Even just telling the teachers some basic ideas might help.  Some of the things I see repeatedly:

  • Know what you are measuring (voltage is not power).
  • Measure the right thing to answer the underlying question.
  • Measure inputs as well as outputs (counting colonies tells you how many culturable bacteria or fungi were in your initial sample, which is useless if you don’t know how big the sample was).
  • Don’t culture unknown micro-organisms (except in a lab with proper protection and sterilization equipment).
  • Read (and cite) some material from the web. High school students should be going well beyond Wikipedia in their literature searches, but even a short Wikipedia seach would be a big step up for most of the middle school and elementary school students.  If Wikipedia is too difficult for an elementary school student (as it may well be), see if there is anything useful on Simple English Wikipedia.
  • Good science fair projects take time, often with many false starts. There are way too many 1-week projects at the county science fair.
  • Mentorship is good, but doing the work for the kid is not—especially not the interpretation of the results. This point is aimed more at the over-involved parents than the teachers—but judges have to be very careful, as there are some highly motivated kids doing things that look like adult work, but really are just the student.  (I remember an incident about a decade ago, of a kid in another category who was severely down graded by the judges in who thought they were judging a parental project, but I talked with the kid for 15 minutes later on and I was convinced that the work really was his alone.  I was angry at the judges for not being more careful in their judgements, but there was nothing I could do about it.)

It’s great to see the enthusiasm and talent of the K–3 group (which has been growing so rapidly that the hall that is rented for the Science Fair is no longer big enough), but that enthusiasm and talent seems to dissipate rapidly around middle school—there are still a lot good middle-school projects, but there are also a number of kids just going through the motions and only a few are continuing to do science fair once they are not required to.  I see more evidence of parental over-involvement at middle school than at elementary school (though that may be due to the selection processes at the different feeder schools, rather than inherent in the age groups).  I didn’t see any evidence of over-involvement in my category this year—if anything, I saw the opposite, with students not getting critical guidance so that they could do a really meaningful project.

One very sad part of the county science fair is how few high school students participate.  There are no school-level fairs in our county at the high school level, and little or no encouragement of individual projects.  This year I think we had 23 projects from high school students, out of a population of about 7500 high school students—about 0.3%.    According to the statistics from the Bureau of Labor Statistics, the various STEM categories add up to about 6% of the workforce (not counting healthcare, which would double the number, and not counting several related occupations, like high-school science teachers, scientific sales, science and engineering managers, …).  So even with very conservative counting, we’re short by a factor of 20 in this county.  I’d be satisfied if even 1–2% of the high school students were entering science fair, but we’re nowhere close to that number, and the participation at the high-school level is shrinking, not growing, each year.

The problem is not strictly a local one—most places see a drop in participation from middle school to high school, but I don’t think many are as extreme as here.  There are some places in the US where high school science fair is big—what have they done differently?

Lots of organizations have seen the problem of high school students losing interest in science fair, and they have put up cash prizes and other incentives for high school students, but (in this county anyway), no one is taking the bait.  We need to find a way to get high-school students excited about doing science or engineering projects, and I don’t know what would stimulate that excitement.

Many (most?) of the good projects in middle school and high school came from home-schooled kids or kids getting a lot of after-school education from mentors or parents.  This may be related to the point that good science fair projects take time and require passion on the part of the students, and the local schools (public, private, and charter) don’t provide a good environment for projects that take time nor for students to show passion—way too much busywork and time wasted preparing for standardized tests.

2012 March 6

How to be a Good Science Fair Judge

Filed under: Science fair — gasstationwithoutpumps @ 20:48
Tags: , , , , ,

It is science fair judging time at the County level, so I collected a few of the resources available for judges:

I was also going to post Resources from a science teacher at teachscienceright.com on the list above, but while there are two rubrics given, they are just point-score lists, with no indication how to do the judging.  I have point-score lists from several different school fairs, and I’ve always found them pretty useless—I much prefer scoring sheets that allow a lot of room for notes about strong and weak aspects of the project, and that don’t bother with point scores.  Other judges have different reactions—some love adding up numbers. The site does have an exercise for students in writing hypothesis statements that may be useful for some teachers.

2011 May 4

California State Science Fair judging

Filed under: Science fair — gasstationwithoutpumps @ 21:41
Tags: , ,

I just got back today from the California State Science Fair, where my son was an entrant in the senior division (high school) and I was a judge in the junior division (grades 6–8).  [My son did not win prizes at state this year.]

I judged one category in the junior division and was the judge from our category who argued for our project to be “project of the year” for the junior division.  The 22 first-place projects (one from each category) all compete for project of the year, which carries a substantial cash prize ($5k for junior division).  So I spent a couple of hours walking around with the other judges looking at the best project in each category.  There were some really amazing projects, as well as a few that were clearly not in the same league.

The process was a bit rushed (we started at 1:30 and the award ceremony was at 4:00), but each of the judges did a good job of presenting what was great for the 1st place in their category and answering probing questions from the other judges. When we finally started voting on the projects, it was clear that judges were not just voting for their own category, but were seriously looking for the best project.  Each judge had 2 votes in the first round, and many of the projects got 0 votes.  Four projects (including the one from my category!) went to the second round of voting, where judges got one vote.  All four were strong projects that could reasonably have won the overall prize, depending on exactly which qualities of the project one focused on as most important.

I was pleased with the discussion of the judges both in the category judging and in the project of the year judging.  We did not all have exactly the same opinions of projects, but the discussion was based on careful observations and weighing all the relevant criteria.  I heard no hint of irrelevant criteria (such as race, age, gender, wealth, or access to equipment) in either discussion, which has marred some discussions I’ve heard in the past.

Every judging process has some randomness and picking one winner often seems more like a lottery than a purely merit-based system.  I think that the discussion and process was successful in keeping this noise down near the irreducible minimum level.  Had I been judging alone, I likely would have picked somewhat different rankings than the group as a whole, but I was happy to sign off on the consensus decisions as being reasonable ones.

Had there been more time, I think that the project of the year voting should have been done in more rounds: say with 4 votes for each judge, then 2 votes each among the top vote getters, then 2 rounds of one vote each.  One of the top 4 projects would have still come out on top, possibly even the same one, but I would have had more confidence that it wasn’t an artifact of the voting scheme.

If I were to dispense advice to students for future years based on my judging experience, I would say that the following things seem to matter most:

  • Novelty.  Judges like best projects that they’ve never seen before.
  • Thoroughness. Judges like for everything to have been thoroughly tested with proper controls and with alternative explanations and methods ruled out.
  • Student-driven projects.  Judges are always on the lookout for projects that have had too much mentor or parent influence.  Since the judges have limited time with the students, it is important for the student to be very clear about their own contributions to the project (did they write their own programs, design their own experiments, build their own equipment, …) The more of the project that is original with the student, the better.
  • Utility. The science fair is really about engineering and applied science. Pure science projects never seem to win, and projects that have a plausible health or environmental application are rated much more highly than the similar projects without those fields of application.  Adding “greenwash” is not a good idea, though.  Projects that pretend to have health or environmental application, but really don’t, get rather harshly downgraded.
  • Quality work. For software, well-written clean code with documentation gets a lot more attention than spaghetti code.  For hardware, neat construction is worthwhile.
  • Good presentation.  The oral presentation and the poster should be clear and easily understood by someone outside the field. Students should be able to answer clearly and politely both difficult, probing questions by experts and clueless questions by people who don’t seem to know anything.
  • Proper categories. Projects that have been miscategorized are sometimes moved to the correct categories, but if not caught in time, they are judged in the category chosen by the student.  Judges will see the work as being weak in the characteristics they are looking for in the category, even if the project would have been strong in the right category. Gaming the system to try to get into a category with fewer projects is rarely a successful strategy.

%d bloggers like this: