The paper describes an interview experiment in which students are given physics problems to solve, and questioned about them. One aspect of the method struck me as strange:
… each student was first given the symbolic version to solve while speaking aloud about their method to reach the solution. Whether correct or incorrect, the subject was asked questions to gauge their understanding of the symbols in the problem. If the subject had difficulties with the symbolic version, they were asked to solve the numeric version of the same question.
Why were they only given the numeric version if they had trouble with the symbolic one? Why was the symbolic one always given first? It seems that this experimental method is designed to elicit a particular conclusion (that symbolic problem solving is harder) by deliberately biasing the way the questions were asked. I would have found the study more interesting if both the symbolic and the numeric versions of the problem were always given and with the order randomized. Does doing a numeric example first make the symbolic form easier? Does doing the symbolic form first make the numeric solution easier? What order of asking the questions maximizes the chance of a student getting them both right? Does allowing a student to go back and fix previous answers make a difference? Did asking the student about the meanings of the symbols change the way they approached the second problem?
In short, I feel that the experimental protocol introduces so many confounding variables that no conclusions can be made based on this study. The tiny sample size (13 students) is less of a problem, since analysis was done on similar questions from a much larger population, to determine what sorts of errors were common, and the interviews with the students were an attempt to get at underlying causes for the common errors.
Despite my qualms about the experimental protocol, some of the observations from the interviews seem useful to teachers. For example, students seem to have difficulty dealing with multiple velocities or multiple accelerations in the same problem. All velocities become “v”, resulting equations that make no sense, because the variables change meaning in mid-equation. This confusion is undoubtedly a common one, but one that teachers can help forestall by insisting on students providing a meaning for every variable they use—and not just a generic meaning like “velocity”, but specific meanings like “the velocity of the hare as it crosses the finish line”. Using computer-programming-style variables that are full words or phrases, like “hare_final_velocity” might help. That is, the confusion results from bad variable naming conventions in standard presentations of physics more than from students’ cognitive problems. In computer programming classes, students would be docked points for using notation as cryptic and non-explanatory as physicists normally do. (That is one reason why “programs like a physicist” is considered an insult by programmers.)
Another observed confusion that students have is distinguishing known and unknown quantities. An example is given of a student combining two equations and removing the known value, being left with two unknowns, rather than removing one of the unknowns to get an equation involving an known and an unknown. This confusion is a more difficult one to address, but is probably largely responsible for the students’ greater success when given numeric problems (assuming that isn’t an artifact of the experimental protocol), since the numbers are clearly known constants, rather than variables. I wonder if there is a pedagogic technique that can be used to help students make the distinction. Obviously, in a book or lecture one could color-code the variables and constants differently, but that would be a very awkward approach for students working on an exam question. Again, the computer programming conventions can help: many languages allow declaring certain names as constants (with the “const” adjective), rather than variables, within the scope of a function. It is also generally expected in good programs that the meanings of variables are explained when the variable is declared. (The lack of a good standard place to put this description is one of the problems with programming languages like Python that do not require declaration of the variables.)
Eugene says in the discussion
A common practice among physics instructors is to assign numeric problems, but to instruct the students to solve the problems symbolically and then only plug in the numbers as one of the last steps. While this is a sensible balance between numeric and symbolic problem solving, I think that it is important to also instruct students in the details of symbolic problem solving.
The “common practice” seems to me a bit backwards: one should solve the numeric problem first, then generalize to other problems, and check the general solution with the already solved numeric problem. Trying to do the general case without the insight from a specific case is likely to lead to a muddle.
Some of the specific suggestions that Eugene makes also strike me as troublesome:
- The first recommendation is to encourage the use of subscripts. I’m not convinced that subscripts decrease the confusion students have about the meanings of their variables or what the knowns and unknowns are. Of course, physicists love subscripts (and superscripts and all sorts of other notation that allows them to pack meaning as cryptically as possible into as few strokes as possible), so maybe students do need to be inoculated with them. Personally, I prefer the computer science approach of using longer variable names, though I will sometimes use subscripts to save writing time.
- Second, encourage students to carefully identify and distinguish known and unknown quantities. This seems like good advice, but lacks some specificity. How should we encourage the students? Making a divided box labeled with known and unknown? Color coding? underlining or circling (which Eugene does suggest)? What pedagogic approaches actually help?
Since I am teaching a couple of bright high school students using Matter and Interactions, I will try suggesting the use of programming-style variables and requiring an explanation when they set up the problem, and see how they react. I think I’ll go with a box of unknowns and box of knowns, rather than the rather clunky “const” notation, for keeping track of what they are trying to do.