Gas station without pumps

2011 January 21

Retrieval beats concept maps for studying

Filed under: Uncategorized — gasstationwithoutpumps @ 22:00
Tags: , , ,

Several bloggers have been posting about the recently released paper “Retrieval Practice Produces More Learning than Elaborative Studying with Concept
by Jeffrey D. Karpicke and Janell R. Blunt (in Science).  Actually most are responding to the New York Times summary of the study, since (as usual) Science wants an arm and a leg to read the article itself. (Unless you are lucky like me and work for a University that pays a million arms and legs to get a library subscription—actually, anyone who lives close enough to a University of California campus can go to the library and read the Science article on-line there.) Luckily, the NY Times author, Pam Belluck, has done a decent job of summarizing the Science article.

The main result (that writing down what one can recall produces more learning in a fixed amount of time than concept mapping does) doesn’t surprise me, as I’ve never seen the point of concept mapping—I did all my learning through problem solving in homework exercises, not from reading notes or cramming.Of course, metacognition (awareness of one’s own thinking) is notoriously poor, so I may have been fooling myself.  Indeed the authors observed in their experiment that metacognition is poor even in the context of their experiment:

Students’ judgments of learning were solicited after students had experienced each text in the initial learning phase. In general, students erroneously predicted that elaborative concept mapping would produce better long-term learning than retrieval practice.

Making “concept maps” has always seemed pointless to me, because it bears no relationship to how I recall information.  I believed that it served some people, since I know my memory functions differently from many other people (I have no recall memory for faces and poor recall memory for names, for example).  So I was interested to see in the article that maybe concept mapping is not the gold standard that education professors make it out to be:

Retrieval practice produced the best learning, better than elaborative studying with concept mapping, which itself was not significantly better than spending additional time reading.

Overall, 101 out of 120 students (84%) performed better on the final test after practicing retrieval than after elaborative studying with concept mapping. … Ninety out of 120 students (75%) believed that elaborative concept mapping would be just as effective or even more effective than practicing retrieval.

The study carefully controlled the type of test being done.  Indeed one version of the experiment had the students making a concept map from memory as the test.  Even in this setup, designed to give maximum advantage to those who studied by concept mapping, the retrieval practice worked much better (with about 60% more recall, if I’m reading their result graphs right).

One of the best reflections on this article  I’ve seen is in the Quantum Progress blog: The NYT on learning: studying notes can be deceiving, where comparisons are made to the hard struggle students make to learn physics and weightlifting is used as a metaphor.



  1. Gas…
    I am reminded of reading how Feynman would sit down and try to recreate every thing he knew about some area starting from first principles.

    Comment by E. E. "pat" Ballew — 2011 January 22 @ 01:52 | Reply

  2. Gas…..

    Just bounced to your blog from Jung’s blog and I’ll list the web address for a presentation that I gave a few weeks ago at The Conference of Higher Education Pedagogy that might interest you.

    I asked on Jung’s blog how the formative assessment strategy I used in teaching several Biology courses fit within the current thinking of STEM instruction. My approach described in the presentation seemed to fit well with Karpicke and Blunt’s paper but also seems to run counter current with the focus on STEM education being more aimed at critical thinking and making sure students learn how to think like scientists. I don’t disagree with this approach but wonder as your posting above does about the role of memorization and retrieval practice in the initial entry into science areas where new terms must be memorized before they can be used in understandable sentences much like what must occur in learning a new language.

    Thanks for your thoughts and any help you can provide about my use of formative assessment in the teaching of Biology would be most appreciated.


    Comment by Mike Kolitsky — 2011 February 17 @ 07:33 | Reply

    • I see you have data that shows that “quizlet” scores correlate highly with exam scores. That is hardly surprising, assuming that the questions were generated by the same people to cover the same content. The correlation tells me nothing about retrieval vs. concept maps (which are study techniques independent of assessment strategy), nor memorization vs. critical thinking (which would require analyzing the questions on both the quizlets and the exams). The fact that the number of quizlets done correlated (though less well) with exam grades indicates that you were measuring diligence to some extent.

      Personally, I believe that frequent feedback to the students is useful in encouraging them to learn the material, but I’m not so sure that there is one best way to provide that for all students.

      Comment by gasstationwithoutpumps — 2011 February 17 @ 08:57 | Reply

      • Thanks for your feedback, I’m still trying ascertain whether what I am seeing is useful or not. I guess as one who is used to getting data from cells and enzymes, it was surprising to me that I could get data such as this from the action of people, i.e. students. The results reported at Virginia Tech extended what I first saw in my online courses but the extra piece about linking Quizlets to study time was made more apparent when I was teaching face-to-face at Washington College. I have found that knowing how much time (even minimal time as measured by Quizlets done) has been very useful when talking with students who ask how to improve their performance on exams. I am also very aware that the types of questions being asked are mostly at the lower levels of Bloom’s Taxonomy and I’ve told myself that this work should be viewed as constructing a baseline that I can compare future work with when future Quizlets include more questions of the analytic and evaluative type as opposed to the remembering and understanding type. I’m still trying to understand how the retrieval practice approach fits what I am seeing so I appreciate your response.

        Comment by Mike Kolitsky — 2011 February 18 @ 06:41 | Reply

        • As I understand the retrieval practice experiment, all the questions were near the bottom of Bloom’s Taxonomy. The tests were memory tests, not problem-solving tests.

          Comment by gasstationwithoutpumps — 2011 February 18 @ 07:25 | Reply

          • That’s what I am thinking also and for my observations with Quizlets as well. So, what is the role of memorization in learning in the sciences and more specifically biology. I have noted in my own courses that there are always new terms that students need to know what they specifically mean. I have even said that some portion of learning about biology is to understand the meanings of terms that must to some extent be memorized so that students can use that language as a foundation for speaking about and understanding concepts. This might be likened to learning a new language in which there must be some minimum vocabulary memorized before one can begin to utilize the rules of grammar to speak in sentences. I am not speaking against what might be viewed as traditional STEM pedagogies like concept mapping or inquiry-based learning but simply asking what is the role for memorization in the sciences, especially in Biology. And, if so, then the Quizlet approach and their usage in formative assessment may also have a role in STEM learning.

            Comment by Mike Kolitsky — 2011 February 18 @ 08:03

  3. Mike, I’m not the best person to ask about the role of memory work in STEM pedagogy. I’ve got a terrible memory and always hated memory work. That’s why I started out in math and only gradually drifted into computer science, computer engineering, and bioinformatics. I could never have started in biology, which in the 70s was taught as an almost pure memory exercise. I don’t have much problem with learning vocabulary, but things like memorizing all the reactions of the Krebs cycle would stop me cold.

    Some biologists do revel in using arcane terminology (often making up new words and acronyms, when perfectly good terms already exist), but the good ones can explain what they are doing with a reasonably small vocabulary.

    Biology is a huge field, and some branches of it are memory-intensive, but there are now many branches of biology that rely more on concepts and problem solving than on memory, so it is no longer reasonable for intro bio courses to select students solely on the basis of their capacity for memorizing long lists of things, as used to be the standard.

    Comment by gasstationwithoutpumps — 2011 February 18 @ 08:58 | Reply

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at

%d bloggers like this: