Thomas Wilschut and Myrthe Braam
Retrieval practice is generally seen as one of the best learning methods: testing yourself is among the most effective ways to learn. In this blog post series, we present a guide to making the best content for retrieval practice. In our last post, we discussed that retrieval works well for both multiple choice and open ended, short answer questions (McDermott et al., 2014). Also, we discussed that retrieval practice can be used to study a variety of materials, ranging from purely factual information, such as vocabulary items or toponymy. It can also be effective for concept-based learning, such as glossary items, and finally, testing yourself is a great way to learn more complex questions (eg, see McDaniel et al., 2013)). However, for each of these item formats and types of materials, the effectiveness of retrieval practice critically depends on the design of the question cue and the answer alternatives. Today, we will give you some hands-on advice for creating the best possible items – to make sure that you and your students can make the most of the available study time.
#1: Keep cues short
To start, it is advisable to keep the length of a written cue to a minimum, while keeping it informative. For factual materials, this is quite straightforward. For example, simply present the translation of a vocabulary item or a location on a map for a toponymy question. For glossary items, sticking to a short formulation that covers the essence of a question can be more challenging. A pitfall is to create a lengthy definition for a glossary item to ensure that it covers all components. In multiple repetition retrieval practice sessions, this will quickly result in the learner recognizing and memorizing only the first few words of the cue, instead of the semantic content of the item.
| Don't do | Do |
|---|---|
| Glossary item: cell membrane | Glossary item: cell membrane |
| Cue: A structure that separates the inside of the cell from the outside environment, which consists of a semipermeable lipid bilayer and which regulates the transport of materials entering and exiting the cell | Cue: Protective layer separating cell from its environment |
#2: Keep cues varied
Using different types or formulations of cues prevents the learning from memorizing the syntactic structure of the cue, and allows you to incorporate different aspects of the definition in a learning session. Randomly selecting different formulations of a cue, or important aspects of a definition, will result in better long term retention and transfer of knowledge (eg, see Farr, 2012).
| Don't do | Do |
|---|---|
| Glossary item: cell membrane | Glossary item: cell membrane |
| Cue: Protective layer separating cell from its environment | Cue 1: Protective layer separating cell from its environment Cue 2: Selective boundary isolating a cell from surroundings Cue 3: A barrier, enclosing cell's internal space, regulating transport |
#3: Choose your answer alternatives carefully
If we want to apply multiple choice questions (MCQs) as a learning method, we have to be thoughtful about the answer alternatives (distractors) that we choose. A certain desirable difficulty is required to ensure that MCQ will not solely eventuate recognition processes. Previous studies suggested that the use of highly similar/related distractors could increase the difficulty level of the learning process while improving learning outcomes. For instance, a study conducted by Little and Bjork (2015) demonstrated that learning with related distractors resulted in lower accuracy scores during the learning phase but higher accuracy scores on the final test. The comparison between highly similar MC distractors, which is called contrasting (Baxter et al., 2021), has shown to benefit the storage of new material. Learners have to retrieve more detailed information about why the incorrect alternatives are incorrect, which will in turn facilitate the learning process. Along with the time efficiency and low required effort for learning with MC, this highlights the advantages of implementing this learning method.
| Don't do | Do |
|---|---|
| Question: What is the name of the protective layer that separates the interior of the cell from its external environment? | Question: What is the name of the protective layer that separates the interior of the cell from its external environment? |
| Answer options: A) Skin B) Blood vessel C) Corneal D) Cell membrane | Answer options: A) Nucleus B) Cytoplasm C) Cell wall D) Cell membrane |
In a related study, we asked students to study French vocabulary items. We compared the efficiency of 3 learning methods: MCQ with related distractors, MCQ with unrelated distractors and open-ended questioning. The distractors that were selected for the 'related' condition, were words similar in meaning, and words similar in word form to the correct answer. We found that both the learning method with related distractors and the learning method with open-ended questions led to higher scores on the final tests compared to the learning method that exclusively used unrelated distractors. This emphasizes the importance of composing clever distractors in MCQ.
#4: Consider the coming test
When designing a retrieval practice item, it is good to keep in mind what the final test will look like. For example, both Little, Storm, and Bjork (2011) and Nguyen & McDaniel, 2014 examined the relationship between practice items and later test items and found that if both targeted the same concept, there was the expected advantage of testing. However, if the quiz items were about related concepts or topics but not about the exact test topics, learning by quizzing actually impaired learning. These results suggest that quizzing may strengthen memory for some information at the expense of related information. The take-home message is that when quiz and test items are aimlessly sampled, teachers must be cautious in assuming that testing will confer benefits for exam performance.
#5: Think about the response modality
Finally, it is important to note that the chosen response modality can influence the learning outcomes. Today, many digital retrieval practice applications offer response options other than typing or selecting the correct answer by clicking with the mouse. For example, speech-based learning has been shown to be a time-efficient way to learn vocabulary items. It automatically trains pronunciation and can be used for learning who have difficulty spelling or typing (see Wilschut et al,. 2021).
Overall, careful question design is pivotal for successful retrieval practice, significantly impacting learning outcomes across various question formats. By applying these tips, educators and learners can optimize their study time and enhance the effectiveness of retrieval practice.
References
Baxter, P., Droop, M., Van Den Hurk, M., Bekkering, H., Dijkstra, T., & Leoné, F. (2021). Contrasting Similar Words Facilitates Second Language Vocabulary Learning in Children by Sharpening Lexical
Representations. Frontiers in Psychology, 12, 688160. https://doi.org/10.3389/fpsyg.2021.688160
Little, J.L., & Bjork, E.L. (2015). Optimizing multiple-choice tests as tools for learning. Memory & Cognition, 43(1), 14–26. https://doi.org/10.3758/s13421-014-0452-8
