In his post Do numbers help students solve physics problems?, John Burk points us to the preprint How numbers help students solve physics problems, by Eugene Torigoe.

The paper describes an interview experiment in which students are given physics problems to solve, and questioned about them. One aspect of the method struck me as strange:

… each student was first given the symbolic version to solve while speaking aloud about their method to reach the solution. Whether correct or incorrect, the subject was asked questions to gauge their understanding of the symbols in the problem. If the subject had difficulties with the symbolic version, they were asked to solve the numeric version of the same question.

Why were they only given the numeric version if they had trouble with the symbolic one? Why was the symbolic one always given first? It seems that this experimental method is designed to elicit a particular conclusion (that symbolic problem solving is harder) by deliberately biasing the way the questions were asked. I would have found the study more interesting if both the symbolic and the numeric versions of the problem were always given and with the order randomized. Does doing a numeric example first make the symbolic form easier? Does doing the symbolic form first make the numeric solution easier? What order of asking the questions maximizes the chance of a student getting them both right? Does allowing a student to go back and fix previous answers make a difference? Did asking the student about the meanings of the symbols change the way they approached the second problem?

In short, I feel that the experimental protocol introduces so many confounding variables that no conclusions can be made based on this study. The tiny sample size (13 students) is less of a problem, since analysis was done on similar questions from a much larger population, to determine what sorts of errors were common, and the interviews with the students were an attempt to get at underlying causes for the common errors.

Despite my qualms about the experimental protocol, some of the observations from the interviews seem useful to teachers. For example, students seem to have difficulty dealing with multiple velocities or multiple accelerations in the same problem. All velocities become “v”, resulting equations that make no sense, because the variables change meaning in mid-equation. This confusion is undoubtedly a common one, but one that teachers can help forestall by insisting on students providing a meaning for every variable they use—and not just a generic meaning like “velocity”, but specific meanings like “the velocity of the hare as it crosses the finish line”. Using computer-programming-style variables that are full words or phrases, like “hare_final_velocity” might help. That is, the confusion results from bad variable naming conventions in standard presentations of physics more than from students’ cognitive problems. In computer programming classes, students would be docked points for using notation as cryptic and non-explanatory as physicists normally do. (That is one reason why “programs like a physicist” is considered an insult by programmers.)

Another observed confusion that students have is distinguishing known and unknown quantities. An example is given of a student combining two equations and removing the known value, being left with two unknowns, rather than removing one of the unknowns to get an equation involving an known and an unknown. This confusion is a more difficult one to address, but is probably largely responsible for the students’ greater success when given numeric problems (assuming that isn’t an artifact of the experimental protocol), since the numbers are clearly known constants, rather than variables. I wonder if there is a pedagogic technique that can be used to help students make the distinction. Obviously, in a book or lecture one could color-code the variables and constants differently, but that would be a very awkward approach for students working on an exam question. Again, the computer programming conventions can help: many languages allow declaring certain names as constants (with the “const” adjective), rather than variables, within the scope of a function. It is also generally expected in good programs that the meanings of variables are explained when the variable is declared. (The lack of a good standard place to put this description is one of the problems with programming languages like Python that do not require declaration of the variables.)

Eugene says in the discussion

A common practice among physics instructors is to assign numeric problems, but to instruct the students to solve the problems symbolically and then only plug in the numbers as one of the last steps. While this is a sensible balance between numeric and symbolic problem solving, I think that it is important to also instruct students in the details of symbolic problem solving.

The “common practice” seems to me a bit backwards: one should solve the numeric problem first, then generalize to other problems, and check the general solution with the already solved numeric problem. Trying to do the general case without the insight from a specific case is likely to lead to a muddle.

Some of the specific suggestions that Eugene makes also strike me as troublesome:

*The first recommendation is to encourage the use of subscripts.*I’m not convinced that subscripts decrease the confusion students have about the meanings of their variables or what the knowns and unknowns are. Of course, physicists love subscripts (and superscripts and all sorts of other notation that allows them to pack meaning as cryptically as possible into as few strokes as possible), so maybe students do need to be inoculated with them. Personally, I prefer the computer science approach of using longer variable names, though I will sometimes use subscripts to save writing time.*Second, encourage students to carefully identify and distinguish known and unknown quantities.*This seems like good advice, but lacks some specificity. How should we encourage the students? Making a divided box labeled with known and unknown? Color coding? underlining or circling (which Eugene does suggest)? What pedagogic approaches actually help?

Since I am teaching a couple of bright high school students using *Matter and Interactions*, I will try suggesting the use of programming-style variables and requiring an explanation when they set up the problem, and see how they react. I think I’ll go with a box of unknowns and box of knowns, rather than the rather clunky “const” notation, for keeping track of what they are trying to do.

Thank you for your careful reading of my paper. I agree that the interview protocol was not well balanced for alternative narratives. The reason for this is that all of our prior research showed that symbolic problems were more difficult for students than numeric problems. Especially for the type of questions used in this study which contained multiple quantities of the same type, multiple equations, and which could be solved sequentially. The protocol was specifically designed to uncover mechanisms for those prior observations.

Random order for the versions seems like a good idea, but my thinking at that time was that it would be a waste of time. We already knew from the earlier studies that the numeric versions of these questions had averages 85-95%. So it was very likely that they would get the numeric version correct, and I thought that that prior correct work would greatly bias how they would attempt to solve the symbolic version. To me the more striking comparison was to show how a change in format could transform inability into ability.

Thinking about it now, it would be interesting if there were students who correctly solved the numeric version but who were then unable to solve the symbolic version. That would also be a very striking distinction. Perhaps something to try in the future.

As for the connection to computer programming, there is already some research that shows that students are more likely to correctly express an algebraic relationship in the context of a computer program than in the context of a math problem.

Soloway, Elliot, Jack Lochhead, and John Clement. Does Computer Programming Enhance Problem Solving Ability? Some Positive Evidence on Algebra Word Problems. In Computer Literacy, edited by Robert J. Seidel, Ronald E. Anderson, and Beverly Hunter, 171-185. New York: Academic Press, 1982.

This study was an extension of John Clement’s famous students and professors problem.

Comment by Eugene Torigoe — 2011 December 23 @ 12:51 |

I see that you have three previous papers on your site https://sites.google.com/site/etorigoe/research I’ll have to look at them also.

Comment by gasstationwithoutpumps — 2011 December 23 @ 17:47 |

I would expect programming-type long variable names to not work as well for by-hand calculations and algebraic manipulations: long variable names have a disadvantage for handwritten work that it’s hard to tell where one variable ends and the next begins (when you’re typing, there’s a space between variable names, but it’s harder to see the spaces when you’re hand writing). In algebra ab typically means a*b, and if you introduce long variable names, then you have the confusion between a*b and a variable named ab–it makes it harder to see the algebraic form of the equation or expression. It’s also time consuming enough to write out long variable names that it’s tempting to skip algebraic steps, which introduces other errors. With their programming background, it may work for the students you’re working with, however. I’ll be interested to hear how it works.

Comment by LSquared — 2011 December 31 @ 15:32 |

I know what you mean about the long names taking too long to write. I suppose that is why the physicists have gotten into the habit of cryptic subscripts, superscripts, left subscripts and so on. There is a tradeoff between speed of writing and clarity of understanding, and I suspect that it is different for novices and for experts.

Comment by gasstationwithoutpumps — 2011 December 31 @ 16:25 |

[…] work, which I discussed in Numbers vs. symbols and Numbers vs. symbols again, is beginning to spread through the educator blogosphere. Dan […]

Pingback by More on numbers and symbols « Gas station without pumps — 2012 January 2 @ 11:39 |

[…] Numbers vs. symbols (gasstationwithoutpumps.wordpress.com) […]

Pingback by Modeling Instruction « Gas station without pumps — 2012 February 29 @ 18:32 |

[…] https://gasstationwithoutpumps.wordpress.com/2011/12/22/numbers-vs-symbols/ […]

Pingback by Sabbatical leave report | Gas station without pumps — 2015 September 8 @ 22:23 |