Gas station without pumps

2012 February 6

Science fair time again

Filed under: Science fair — gasstationwithoutpumps @ 20:53
Tags: ,

Last week I judged one middle-school (7–8th grades) science fair, tomorrow I’ll judge a science fair at a K–8th school, and Thursday I’ll judge a science fair at a K–5th school.  Each of the three schools gets to send 10 kids up to the county science fair, which is not a completely fair allocation, as some schools are larger than others.

The judging styles are different at each of the school fairs:

  • The middle school has the judging done just on the poster, report, and lab notebook, without the kids being present, but allows the judges lots of time—there are two full days that the posters are up.  They make sure that each of the posters is judged by at least two different judges, and that the top posters are judged by more judges (if possible), so that they can make a reasonable decision about which 10 kids to send on to the county science fair.  From what I saw there (I only judged about 30 or 40 of the over 180 posters), they have about 20 or so posters that would be quite respectable at the county level, but none that were so far above the rest that there was no question about which ones to send on to county.  The teachers will have to pick 10 out of the 20 or so, and the 10 they pick are likely to be a rather arbitrary selection.  Interviewing the students might have added a little more information, but interviewing the 180 kids would have needed a lot more judges, which introduces other sources of randomness.
  • The K–8 school I’m judging at tomorrow is smaller (I expect to judge 60–80 posters), and does a 2-stage judging.  In the first stage, all posters are examined. In the second stage, selected kids are interviewed.  These may be contenders for going on to county, or they may be kids whom the judges have questions about excessive parental involvement, or they could just be a few randomly selected kids.  The number of judges is probably quite small, with every judge viewing every poster, but only for a few minutes each.
  • The K–5 school I’m judging for on Thursday has the closest equivalent of the style of judging done at the county level, with the kids all standing by their posters, waiting for roaming judges to come up and talk with them.  This is the most fun style of judging, but it makes it difficult for the judges to read the posters and reports in any detail.  It also requires a huge number of judges all to be there at the same time, so the kids don’t have to wait too long between judges (I think that the ideal ratio is around one judge for every 3 posters).

Last year, about this time, I posted some notes on judging, which may be worth re-reading.  This year, I want to talk more about what things I see as common problems in science fair projects.

The biggest problem is cookie-cutter projects.  When students don’t have any questions of their own to answer, their parents or teachers tend to suggest standard projects that have been done hundreds of times before.  This includes things like the lemon battery, the Stroop test, the effect of carbonated beverages on teeth, the effect of different chemicals on the growth of plants, and so forth.  Sites like Science Buddies have hundred (or 1000s) of these run-of-the-mill projects.  Although students can learn about science by doing a routine project, most often they are just going through the motions, following carefully scripted instructions from a web site or book.  It is very rare for a student doing one of these routine projects to have dug into the science behind the experiment and understood what they are doing.  It is even rarer for one of these projects to have been sparked by a student question.

Another big problem is one I lay at the feet of the teachers. Few students have any idea what a hypothesis is. Almost all of them think that it is a random guess about the outcome of some predefined experiment, but a hypothesis is not a guess. It is a prediction from a theoretical model of what is happening.  For a hypothesis to be meaningful, there must be at least two competing theories of what is going on, and the experiment has to be designed so that these two theories result in different predictions (different hypotheses) in the context of the experiment.

For example, if we were interested in the effect of temperature on the bounciness of rubber balls, it would not be good enough to randomly guess that warmer balls would bounce higher or less high.  One needs a theory of bounciness that would predict a change in bounciness.  For example, if one were considering tennis balls, one might predict that warmer tennis balls would have a higher air pressure inside, and hence be stiffer, causing less frictional loss from flexing the rubber of the ball, and hence bouncing higher.  But for solid balls, the warmer ones might be less stiff, due to softening the material, and hence bounce less high.  The theory comes before the experiment design!

Even students who have good hypotheses often have poorly designed experiments.  The biggest problem, particularly in behavioral studies and biology, is having too small a sample.  Students often devise experiments testing dozens of different conditions once each.  The results are usually uninterpretable, since there is no way of telling how much of the variation is due to noise in the measurements and how much is due to the conditions being tested.  It is far better to have a very small number of conditions (2 or 3) and dozens of data points for each condition.

One thing most judges are taught to look for is a contemporaneous lab notebook.  Did the student record their experiment as they did it?  Are the dead ends and mistakes recorded as well as the successes?  Are the raw data recorded?  A student who just has a polished final report but no lab notebook is seriously handicapped in convincing judges that they did the work themselves.

Specific flaws appear in some common projects.  For example, in projects about generating power (wind turbines, fuel cells, batteries, tide generators, muscle-powered generators, solar cells, …) it is very common for students to measure and report the voltage of the device they are testing.  The voltage is easy to measure with a multimeter, but it is of little interest for the underlying question of extracting energy from something.  Almost all the time what we are interested in is the energy in joules or the power in watts (the energy per unit time).  It is not much more difficult to measure power instead of voltage—at the middle school level, simply putting in a fixed load (say a 100-ohm resistor) and measuring the voltage is enough to compute the power using Ohm’s Law. At the high-school level, I would expect load matching to maximize the useful power output.

Similarly, for experiments that compare the number of microorganisms in different conditions, it is important to use a method of measuring the microorganisms that reflects what is supposed to be compared.  If you are trying to compare the number of bacteria from one source or another, you want to count the number of bacteria.  One way to do this, one takes a fixed size sample, dilutes it, then inoculates a petri dish.  Each of the culturable bacteria in the initial sample grows into a separate colony and the colonies can be counted.  It is important for this technique that the inputs be measured, and not just the outputs.  Doing a smear plate with an unknown size sample provides nothing that can be meaningfully quantified.  A different technique is used if you are looking at how fast bacteria grow in different conditions.  For that, you want to start with about the same number of bacteria in each condition, then measure how many are present after growth—measuring the opacity of a liquid culture is a good way to do this.  For measuring the strength of antibiotics, it is common to grow a “lawn” of bacteria, and measure a “zone of inhibition” around disk of the antibiotic material—a technique that assumes that the antibiotics being compared diffuse equally well through the agar growth medium.  There is not one universally correct way to measure bacteria—it all depends on what is trying to be measured (number in the initial sample, ratio of different strains, growth rate, …).

It is largely the teacher’s job to assist students in figuring out whether a proposed experiment measures what is relevant for the investigative questions. Too often the teachers are satisfied with the students measuring anything, without any attention to whether what they measure provides a meaningful answer to their question.  I think that it would be valuable to have science-fair training workshops for teachers, in which they are given dozens of descriptions of projects and try to figure out what advice the kids need to improve their projects.

 

10 Comments »

  1. I recall that your son has done computational science fair projects. Could you comment on coming up with a hypothesis for a project where the student writes a program? Also, hints for keeping a notebook, when most of the actual work is done on a computer? My son may be doing some sort of computer-based science fair project next year when he is in 8th grade.

    Comment by Yves — 2012 February 6 @ 21:47 | Reply

    • For the past several years my son has done computer-science projects for science fair. His reports are available at http://users.soe.ucsc.edu/~karplus/abe/science_education.html

      Last year’s was an engineering project, without much of a hypothesis, but others have involved the difference (in speed or quality) between different algorithms. The fractal one was really a math project with a computational component, and it did not have a hypothesis either (it did include him deriving the fractal dimension of the Sierpinski triangle).

      Science fair judges do not, as a rule, understand pure math projects, and they don’t like engineering projects as much as hypothesis-driven research, so it is best to recast engineering projects into some sort of comparison of different approaches, so that the engineering design decisions can be presented as hypotheses.

      Comment by gasstationwithoutpumps — 2012 February 6 @ 21:59 | Reply

  2. You are expecting original experiments, and hypotheses derived from a theory, from elementary school students??? Personally, at that age, a student who can replicate one of those “cookie cutter” experiments is doing quite well. I think the role of science fairs in this age group is just to get the students to become familiar with the language and process of doing science.

    Comment by Bonnie — 2012 February 7 @ 04:49 | Reply

    • In a word, yes. I don’t expect the theories to be sophisticated, nor do I expect enormous originality, but I expect students to be thinking as if they were scientists. I expect teachers to be guiding them in learning how to think like scientists, rather than in how to copy off the web and follow a recipe. I see enough successes to know that it is possible for a substantial fraction of the students currently, which leads me to believe that with teaching directed at that goal, a much larger proportion could be getting it.

      Comment by gasstationwithoutpumps — 2012 February 7 @ 07:53 | Reply

  3. I agree wholeheartedly with you that teachers, especially at lower grades, but sometimes even at the high school level, sometimes seem to encourage “cookie cutter” projects. Sometimes these cookie cutters are basically lab demonstrations of a known concept (i.e. “can I measure the Stroop effect?”, which not an experiment, but a demonstration). In that case, the “experiment” should be called something else, and might be a valuable experience of its own (to learn to do the mechanics of an experiment, of designing equipment, collecting data, keeping data, and analyzing it). But I’d be happier if it was called a science demo. I have been guilty of encouraging a type of project that I do consider to be an expeirment, that has the flaws you raise earlier in this article, though: non-hypothesis driven (in the grant applications, biologists/psychologists, like to call it “hypothesis generating”) science experiments. I know that these experiments don’t sell as well (to grants agencies as well as science fairs), but still think that they have potential value. The problem, though, is when students are unlikely to be able to use them to generate even simple hypotheses when they are as complicated a the Stroop effect (for example, if one examined the relationship between the Stroop effect & some measure of attentional focus or video game playing or spelling ability) because they lack knowledge of the science one might even consider (about the brain, in this case and psychology). In your bouncing ball example, though, it seems to me that a “hypothesis generating” experiment (of bouncing 5 different kinds of balls, and seeing which one’s are affected by temperature) might actually help a child develop a hypothesis that incorporates ideas about materials and stiffness and bounciness.

    I’ve been disturbed enough by the paths I see science fair experiments take (especially at younger levels, or in schools where they are required) that I’ve feared they teach “cargo cult science” (i.e. science that looks like science but isn’t) that I have worried about their use at all. I’d be interested in suggestions for improving science fair projects. I think my own first suggestion at the elementary school level is that the children probably lack the understanding to do a real hypothesis testing or even generating experiment and should instead be doing science “demos” where they are required to test a hypothesis (that they generate themselves) but in a defined problem (for example, the bouncing balls). I would concentrate on making the science real science, rather than giving the children the obligation to choose the question themselves. I would try to generate good cookies by generating better cookie cutters. I don’t know how that would then be judged as a competition, though.

    Comment by zb — 2012 February 7 @ 10:16 | Reply

  4. To leave the possibility that an individual child might be capable of more, I would allow them to get a project not an approved list approved (but, I’d want to have it approved by a practicing scientist in the field they’re topic fits, not an elementary school teacher — unless they are also a practicing scientist in the field).

    Comment by zb — 2012 February 7 @ 10:18 | Reply

  5. […] Science fair time again (gasstationwithoutpumps.wordpress.com) […]

    Pingback by Science Fair advice « Gas station without pumps — 2012 February 14 @ 18:49 | Reply

  6. […] Science fair time again (gasstationwithoutpumps.wordpress.com) […]

    Pingback by More science fair advice « Gas station without pumps — 2012 February 15 @ 14:25 | Reply

  7. […] Science fair time again (gasstationwithoutpumps.wordpress.com) […]

    Pingback by Science fair poster boards « Gas station without pumps — 2012 February 21 @ 21:33 | Reply

  8. […] the prediction of a model in a situation in which different models make different predictions. (See Science fair time again or Google science fair, for […]

    Pingback by Scientists don’t test hypotheses « Gas station without pumps — 2012 February 27 @ 15:33 | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: