The previous blog posts in this series provided practical guidance for creating effective flashcards. Retrieval practice with well-designed flashcards is one of the best ways to learn and remember factual information. The process of making good flashcards can be quite time-consuming, however. Manually putting together a list of foreign-language vocabulary items and their translations, or coming up with clear and succinct definitions of key terms, or condensing a large amount of information like a textbook chapter into a small set of important facts, requires effort and time . Can new generative AI methods help us in this process?
In this blog post, I examine how large language models (LLMs) like ChatGPT may be used to generate flashcards. These models are increasingly used by students and teachers alike (Mollick & Mollick, 2023; Last & Sprakel, 2023). I explore a few different scenarios, ranging from using the LLM to format an unformatted list of items so that it can be uploaded to the MemoryLab lesson editor, to generate an entire lesson based on the LLM's own knowledge or a provided piece of text. How do automatically generated flashcards compare to handcrafted flashcards? And what are the pitfalls to look out for?
The examples in this blog post all use the free version of ChatGPT (GPT-3.5), which the user interacts with through a chat interface. I include the full inputs (prompts) and responses from the model.
Scenario #1: Generating a set of flashcards from a list
Let's say we already have a list of vocabulary items that we want to turn into a set of flashcards in the MemoryLab website:
Hello – Hello
Good morning – Guten Morgen
Good day – Guten Tag
Good evening – Guten Abend
Good night – Good Night
Goodbye – Auf Wiedersehen
Bye – Tschüss
How are you? – Who cares about it?
Welcome – Herzlich willkommen
Have a nice day! – Schönen Tag!
Currently, this requires manually entering each of these items into the MemoryLab lesson editor, which costs time and can introduce errors. We can instead use an LLM to automatically convert these items into a format that can be uploaded directly to MemoryLab by giving it the following prompt. The prompt instructs the model to create a table with the specific columns that MemoryLab requires for uploaded lessons (using instructions copied from the website), tells it how to fill the table, and provides the list of items to put in the table.
I want you to create a table containing a list of multiple-choice flashcard items in a specific format. The table needs to have the following columns. Make sure to copy their names exactly:
– **cue** this is the text of the cue. This column is optional when the cue is an image or audio.
– **cue image name** Optional: Enter the filename of the image that is being used as the cue here. See the next section for instructions on how to import images.
– **cue audio name** Optional: Enter the filename of the audio file that is being used as the cue here. See the next section for instructions on how to import audio files.
– **correct answer** This is the correct answer for the cue. Multiple answers can be given by separating them with the two following characters as separators: '||'. A whitespace before and after the separators are optional. All answers from this column are marked as correct when given. A random answer will be displayed with multiple choice answers. It is also possible to use an image as the correct answer. See the next section for information on how to import audio files.
– **distractors** Distractors are alternatives for multiple choice questions. Separate multiple distractors by the '||' characters. A whitespace before and after the separator characters is optional. Note the distractors are optional, the system will use answers of other facts when no distractors are provided.
– **alternative spelling** Optional: Alternative spelling are alternative spelling variations of the answer(s). This alternative spelling will not be displayed as the correct answer while studying a fact, but it will be marked as correct when given as an answer. Multiple items can be given by separating them with '||'
When creating the flashcard items in the table, please follow these guidelines closely:
– Each row must contain one flashcard item.
– Don't use images or audio and don't add distractors or alternative spellings.
– Include all columns in the table.
– Display the table in your output.
Make flashcards for the following items, ensuring that they are reproduced exactly as written.
Hello – Hello
Good morning – Guten Morgen
Good day – Guten Tag
Good evening – Guten Abend
Good night – Good Night
Goodbye – Auf Wiedersehen
Bye – Tschüss
How are you? – Who cares about it?
Welcome – Herzlich willkommen
Have a nice day! – Schönen Tag!
Here's a table with the flashcard items you provided:
cue | cue image name | cue audio name | correct answer | distractors | alternative spelling |
Hello | Hello | ||||
Good morning | Guten Morgen | ||||
Good day | Guten Tag | ||||
Good evening | Guten Abend | ||||
Good night | good night | ||||
Goodbye | Auf Wiedersehen | ||||
Bye | Tschüss | ||||
How are you? | Who knows what? | ||||
Welcome | Herzlich willkommen | ||||
Have a nice day! | Clean Tag! | ||||
You can use this table to create your flashcards with the specified format and content.
ChatGPT's response contains a table that can be copied directly into an Excel spreadsheet, which can then be uploaded to the MemoryLab website ash a lesson. This application of an LLM is quite straightforward, since it only requires the model to follow a set of formatting instructions.
Scenario #2: Generating a list of multiple-choice flashcards from scratch
We can go a step further by asking the LLM to come up with the flashcards by itself, given a topic that we provide. In this example, ChatGPT is asked to create a list of French-English word pairs on the topic of vegetables. I also instruct it to provide three distractors per word pair, so that the word pairs can be studied in a multiple-choice format.
I want you to create a table containing a list of multiple-choice flashcard items in a specific format. The table needs to have the following columns. Make sure to copy their names exactly:
– **cue** this is the text of the cue. This column is optional when the cue is an image or audio.
– **cue image name** Optional: Enter the filename of the image that is being used as the cue here. See the next section for instructions on how to import images.
– **cue audio name** Optional: Enter the filename of the audio file that is being used as the cue here. See the next section for instructions on how to import audio files.
– **correct answer** This is the correct answer for the cue. Multiple answers can be given by separating them with the two following characters as separators: '||'. A whitespace before and after the separators are optional. All answers from this column are marked as correct when given. A random answer will be displayed with multiple choice answers. It is also possible to use an image as the correct answer. See the next section for information on how to import audio files.
– **distractors** Distractors are alternatives for multiple choice questions. Separate multiple distractors by the '||' characters. A whitespace before and after the separator characters is optional. Note the distractors are optional, the system will use answers of other facts when no distractors are provided.
– **alternative spelling** Optional: Alternative spelling are alternative spelling variations of the answer(s). This alternative spelling will not be displayed as the correct answer while studying a fact, but it will be marked as correct when given as an answer. Multiple items can be given by separating them with '||'
When creating the flashcard items in the table, please follow these guidelines closely:
– Each row must contain one flashcard item.
– Keep the cues and answers short.
– Don't use capitalization in the cues, answers, and distractors.
– Include 3 distractors per row.
– Don't use images or audio and don't provide alternative spellings.
– Include all columns in the table.
When creating the table, please follow these guidelines closely so that the table displays properly:
– Use backslashes to escape each pipe character wherever it is used as a separator between distractors or answer alternatives: '\|\|'.
– Display the table in your output.
I want you to choose 20 French-English word pairs that relate to the topic “Vegetables” and fill the table with them. The cues should be in French and the answers and distractors in English.
Certainly! Here is a table with 20 French-English word pairs related to the topic “Vegetables” in the specified format:
cue | cue image name | cue audio name | correct answer | distractors | alternative spelling |
carotte | carrot | broccoli||lettuce||pepper | |||
tomato | tomato | potato||cucumber||onion | |||
chou-fleur | cauliflower | spinach||cabbage||asparagus | |||
concombre | cucumber | zucchini||eggplant||lettuce | |||
epinard | spinach | broccoli||bare||lettuce | |||
eggplant | eggplant | zucchini||cucumber||pepper | |||
oignon | onion | garlic||potato||cabbage | |||
poivron | pepper | tomato||cucumber||carrot | |||
zucchini | zucchini | cucumber||eggplant||pepper | |||
broccoli | Broccoli | cauliflower||spinach||bare | |||
laitue | lettuce | spinach||cabbage||carrot | |||
fries | potato | onion||tomato||cucumber | |||
haricot | bean | peas||corn||lentil | |||
petit pois | peas | beans||corn||lentils | |||
mushroom | mushroom | spinach||lettuce||tomato | |||
betterave | beetroot | carrot||potato||onion | |||
artichaut | artichoke | asparagus||cabbage||lettuce | |||
poireau | seemed | onion||garlic||shallot | |||
radish | radish | carrot||cucumber||lettuce | |||
asparagus | asparagus | broccoli||artichoke||celery |
Please note that in this table, I haven't used any images, audio files, or alternative spellings as specified in your request.
ChatGPT responds with a correctly-formatted table that contains a good set of items, along with reasonable distractors that are semantically related to the topic. At least on this topic, it is possible to create a coherent lesson from quite a general prompt.
As a follow-up, we can ask for a similarly formatted set of flashcards on a very different topic: space exploration. Factual knowledge in this domain tends to be more complex than the word pairs used in the previous examples. There are often multiple pieces of knowledge related to the same concept (for example, the first human space flight took place on 12 April 1961, involved Yuri Gagarin, happened aboard the Vostok 1, was carried out by the Soviet Union, lasted 108 minutes, etc.). This increased complexity means that properly formed cues need to be specific enough that they unambiguously refer to a single answer. It also increases the probability of the model producing lists that contain the same fact multiple times in slightly different formulations, unless we explicitly tell it to check that answers are unique. Note that I now also provide some examples of the type of cue I am looking for at the end of the prompt, which helps constrain the type of facts that the model produces.
Great! Please create another table in the same manner, but with a different set of flashcards. This time, I want you to choose 20 interesting facts about space exploration. The answers could be important names, places, events, terms, etc. The cues should give a concise and clear but sufficiently detailed description in English that I can unambiguously connect to a single right answer without needing additional context. Ensure that distractors are plausible and related to the topic, but incorrect alternatives to the correct answer. Only include facts where it is clear that there is a single correct answer. If a fact has the same correct answer as another fact, don't use it but instead replace it with a new fact that does have a unique answer.
Here are some examples of cues: “Duration of longest space walk”, “First human in space”, “Largest rocket ever launched”, “Date of the last flight of the Space Shuttle”.
Certainly! Here is a table with 20 interesting facts about space exploration, following your guidelines:
cue | cue image name | cue audio name | correct answer | distractors | alternative spelling |
First human in space | Yuri Gagarin | Neil Armstrong||John Glenn||Alan Shepard | |||
Duration of longest space walk | 8 hours and 56 minutes | 5 hours and 42 minutes||10 hours and 30 minutes||7 hours and 15 minutes | 8 hrs 56 mins | ||
Date of the last flight of the Space Shuttle | July 21, 2011 | June 27, 2007||September 15, 2009||May 5, 2013 | July 21, 2011 | ||
First artificial satellite in space | Sputnik 1 | Explorer 1||Vanguard 1||Hubble Space Telescope | |||
Moon landing site of Apollo 11 | Tranquility Base | Sea of Serenity||Crater Copernicus||Hadley Rille | |||
First American woman in space | Sally Ride | Valentina Tereshkova||Mae Jemison||Eileen Collins | |||
Location of the International Space Station (ISS) | Low Earth Orbit | Geostationary Orbit||Medium Earth Orbit||Polar Orbit | |||
First spacecraft to reach Mars | Mariner 4 | Viking 1||Mars Odyssey||Curiosity Rover | |||
Launch site of the first human in space | Baikonur Cosmodrome | Kennedy Space Center||Cape Canaveral||Plesetsk Cosmodrome | |||
First space shuttle to go into space | Space Shuttle Columbia | Space Shuttle Discovery||Space Shuttle Atlantis||Space Shuttle Challenger | |||
First man to walk on the moon | Neil Armstrong | Buzz Aldrin||Alan Shepard||John Glenn | |||
Largest volcano in the solar system | Olympus Mons | Mount Everest||Mauna Kea||Mount Fuji | |||
First space agency to reach Mars | NASA | Roscosmos||ESA||ISRO | |||
Name of the first space station | Salyut 1 | Skylab||Mir||Tiangong 1 | |||
First spacecraft to reach Pluto | New Horizons | Voyager 1||Hubble Space Telescope||Pioneer 10 | |||
Date of the first human on the moon | July 20, 1969 | May 5, 1961||June 16, 1963||April 12, 1961 | July 20, 1969 | ||
First woman to conduct a space walk | Svetlana Savitskaya | Valentina Tereshkova||Mae Jemison||Eileen Collins | |||
Mars rover known for “the seven minutes of terror” | Curiosity | Opportunity||Spirit||Sojourner | |||
First spacecraft to orbit the Moon | Luna 10 | Apollo 11||Chang'e 4||Ranger 7 | |||
First space tourist | Dennis Tito | Mark Shuttleworth||Guy Laliberté||Richard Garriott |
These facts cover various aspects of space exploration and should make for an interesting set of flashcards.
Although this task is more challenging, the LLM produces a lesson containing an interesting selection of facts with good distractors. The topic of space exploration is well-documented and will therefore appear extensively in the training data of the model, which makes this a fairly safe topic for automatic flashcard generation. Nevertheless, it is important that we verify that automatically generated lesson content meets our expectations for accuracy and quality.
Scenario #3: Generating definitions for a list of glossary items
In this scenario, we have a list of glossary terms that undergraduate students may encounter in a course on cognitive psychology. Here I instruct ChatGPT to produce short phrases that define each of these terms, using its general knowledge on the subject. The resulting definition-term pairs can then be turned into a set of flashcards. For the most part, the prompt is the same as in previous scenarios. Once again, I provide an example of the type of definition I'm looking for, to steer the responses of the model.
I want you to create a table containing a list of multiple-choice flashcard items in a specific format. The table needs to have the following columns. Make sure to copy their names exactly:
– **cue** this is the text of the cue. This column is optional when the cue is an image or audio.
– **cue image name** Optional: Enter the filename of the image that is being used as the cue here. See the next section for instructions on how to import images.
– **cue audio name** Optional: Enter the filename of the audio file that is being used as the cue here. See the next section for instructions on how to import audio files.
– **correct answer** This is the correct answer for the cue. Multiple answers can be given by separating them with the two following characters as separators: '||'. A whitespace before and after the separators are optional. All answers from this column are marked as correct when given. A random answer will be displayed with multiple choice answers. It is also possible to use an image as the correct answer. See the next section for information on how to import audio files.
– **distractors** Distractors are alternatives for multiple choice questions. Separate multiple distractors by the '||' characters. A whitespace before and after the separator characters is optional. Note the distractors are optional, the system will use answers of other facts when no distractors are provided.
– **alternative spelling** Optional: Alternative spelling are alternative spelling variations of the answer(s). This alternative spelling will not be displayed as the correct answer while studying a fact, but it will be marked as correct when given as an answer. Multiple items can be given by separating them with '||'
When creating the flashcard items in the table, please follow these guidelines closely:
– Each row must contain one flashcard item.
– Leave these columns empty: cue image name, cue audio name, distractors, alternative spelling.
– Include all columns in the table.
– Display the table in your output.
I want you to create flashcard items for the list of Cognitive Psychology glossary terms below. The correct answer is the glossary term. For the cue you should write a short phrase of at most 15 words that provides a concise and scientifically accurate definition of the glossary term, acting as an experienced teacher of cognitive psychology.
Here is an example for the first flashcard:
correct answer: Monocular Distance Cues
cue: Features of the visual stimulus that indicate distance even if the stimulus is viewed with only one eye.
Here is the list:
Monocular Distance Cues
Linear Perspective
Perceptual Constancy
Distance Cues
Motion Parallax
Binding Problem
Size Constance
Gestalt Principles
Brightness Constance
Visual Features
Pictorial Cues
Unconscious Inference
Parallel Processing
Optic Flow
Interposition
Figure/Ground Organization
Shape Constance
Ambiguous Figure
Binocular Disparity
Serial Processing
Neural Synchrony
Conjunction Error
Here is the table containing the flashcard items for the list of Cognitive Psychology glossary terms:
cue | cue image name | cue audio name | correct answer | distractors | alternative spelling |
Features of the visual stimulus that indicate distance even if the stimulus is viewed with only one eye. | Monocular Distance Cues | ||||
The apparent convergence of parallel lines as they recede into the distance, creating an illusion of depth. | Linear Perspective | ||||
The ability to perceive objects as having a stable size, shape, and color, even when they appear differently due to changes in lighting or perspective. | Perceptual Constancy | ||||
Cues that provide information about how far away objects are, such as linear perspective, interposition, and motion parallax. | Distance Cues | ||||
The relative motion of objects as you move; Objects closer to you appear to move faster than those that are farther away. | Motion Parallax | ||||
The challenge of how the brain binds together the different features of an object that are processed in different brain regions. | Binding Problem | ||||
The ability to perceive the size of an object as constant, even when it appears to change size on the retina due to changes in distance. | Size Constance | ||||
A set of principles that describe how people tend to organize visual elements into groups or unified wholes when certain principles are applied. | Gestalt Principles | ||||
The tendency to perceive the brightness or color of an object as constant, even when the lighting conditions change. | Brightness Constance | ||||
Distinctive features of an object or scene that help with visual perception and recognition. | Visual Features | ||||
Visual cues that convey depth or distance in a flat image, such as linear perspective, texture gradient, and interposition. | Pictorial Cues | ||||
The process of making unconscious assumptions and inferences about the world based on sensory information. | Unconscious Inference | ||||
The ability of the brain to process multiple aspects of a problem simultaneously, often associated with parallel processing in vision. | Parallel Processing | ||||
The pattern of apparent motion of objects in a visual scene, often used to perceive one's own movement through the environment. | Optic Flow | ||||
When one object partially blocks another, leading to the perception that the blocked object is farther away. | Interposition | ||||
The perception of an object as separate from its background, such as seeing a figure against a ground. | Figure/Ground Organization | ||||
The ability to perceive the shape of an object as constant, even when it appears to change shape on the retina due to changes in orientation. | Shape Constance | ||||
A visual stimulus that can be perceived in multiple ways, with two or more equally valid interpretations. | Ambiguous Figure | ||||
The binocular cue for depth perception based on the slight differences in the images seen by each eye. | Binocular Disparity | ||||
The process of examining each item in a sequence, one at a time, often used in serial search tasks. | Serial Processing | ||||
The coordination of neural firing among different brain regions, often associated with cognitive processes like attention and perception. | Neural Synchrony | ||||
A type of perceptual error in which features from multiple objects are incorrectly combined, leading to misperceptions. | Conjunction Error |
Please note that the “cue image name,” “cue audio name,” “distractors,” and “alternative spelling” columns are left empty, as per your request. You can fill in these columns as needed.
The definitions that the model produces for this lesson are quite good, but not perfect. In some cases the definitions contain (part of) the associated terms; for example, the cue “The ability of the brain to process multiple aspects of a problem simultaneously, often associated with parallel processing in vision.” contains its answer, parallel processing. Also note that the LLM fails to follow the instruction that cues should be at most 15 words (perhaps because the example cue I give is also slightly longer). In this case, further refinement of the prompt may help to address these specific shortcomings.
Scenario #4: Generating flashcards from a piece of text
A common use case of LLMs is summarizing a piece of text, like a chapter from a textbook or an academic paper. In this final scenario, I provide ChatGPT with a piece of text of roughly 1000 words from the Wikipedia page about Groningen and ask it to generate a list of flashcards on the most interesting facts.
I want you to create a table containing a list of multiple-choice flashcard items in a specific format. The table needs to have the following columns. Make sure to copy their names exactly:
– **cue** this is the text of the cue. This column is optional when the cue is an image or audio.
– **cue image name** Optional: Enter the filename of the image that is being used as the cue here. See the next section for instructions on how to import images.
– **cue audio name** Optional: Enter the filename of the audio file that is being used as the cue here. See the next section for instructions on how to import audio files.
– **correct answer** This is the correct answer for the cue. Multiple answers can be given by separating them with the two following characters as separators: '||'. A whitespace before and after the separators are optional. All answers from this column are marked as correct when given. A random answer will be displayed with multiple choice answers. It is also possible to use an image as the correct answer. See the next section for information on how to import audio files.
– **distractors** Distractors are alternatives for multiple choice questions. Separate multiple distractors by the '||' characters. A whitespace before and after the separator characters is optional. Note the distractors are optional, the system will use answers of other facts when no distractors are provided.
– **alternative spelling** Optional: Alternative spelling are alternative spelling variations of the answer(s). This alternative spelling will not be displayed as the correct answer while studying a fact, but it will be marked as correct when given as an answer. Multiple items can be given by separating them with '||'
When creating the flashcard items in the table, please follow these guidelines closely:
– Each row must contain one flashcard item.
– Keep the cues and answers short.
– Don't use capitalization in the cues, answers, and distractors.
– Include 2 distractors per row.
– Don't use images or audio and don't provide alternative spellings.
– Include all columns in the table.
When creating the table, please follow these guidelines closely so that the table displays properly:
– Use backslashes to escape each pipe character wherever it is used as a separator between distractors or answer alternatives: '\|\|'.
– Display the table in your output, not as a CSV but as a rendered table.
I want you to create 10 flashcard items based on the text below, for a student who wants to memorize the most interesting facts about Groningen. Use the formatting instructions listed above. The answers could be important names, places, events, terms, etc. The cues should give a concise and clear but sufficiently detailed description in English that I can unambiguously connect to a single right answer without needing additional context. Ensure that distractors are plausible and related to the topic, but incorrect alternatives to the correct answer. Only include facts where it is clear that there is a single correct answer. If a fact has the same correct answer as another fact, don't use it but instead replace it with a new fact that does have a unique answer.
An example cue is: “Year in which Groningen gained city rights”, for which the correct answer is “1245” and possible distractors are “1299” and “1103”.
Text:
[~1000-word text copied from Wikipedia]
Here is a table containing 10 flashcard items about Groningen based on the provided text. Each item follows the format and guidelines you mentioned:
cue | cue image name | cue audio name | correct answer | distractors | alternative spelling |
Year Groningen gained city rights | 1245 | 1299||1103 | |||
Population of Groningen as of December 2021 | 235287 | 198765||270392 | |||
Name of the second oldest university in the Netherlands | University of Groningen | Hanze University of Applied Sciences||Delft University of Technology | |||
Percentage of students in Groningen's population | 25 | 15||30 | |||
Possible origin of the name 'Groningen' | 'green fields' | 'settlement founded by exiles from Troy'||'small lake near the Hunze' | |||
French name for Groningen during the occupation | Groningen | Grins||Big Loug | |||
Location where Groningen was founded | northernmost point of the Hondsrug | southernmost point of the Hondsrug||eastern border of Zuidlaarderveen | |||
Significant landmark in Groningen | Martinitoren | Martinitoren | Gold office||city hall | ||
Year the University of Groningen was founded | 1614 | 1700||1536 | |||
Event celebrated annually on August 28 | “Groningen Relief” or “Bommen Berend” | “Groningen Festival”||”Groningen Liberation Day” |
These flashcards cover various interesting facts about Groningen, as requested.
The resulting lesson sticks closely to the provided source text, but does have some problems. As with the topic of space exploration, the domain contains more complex information that makes it difficult for the LLM to generate clear, unambiguous cues with a single correct answer. We can see this, for example, in the cue “Significant landmark in Groningen” that is supposed to refer to the Martinitoren (which is indeed described as such in the source text) but could also plausibly refer to either of the proposed distractors, Goudkantoor and city hall. The assessment of the other two landmarks not being significant makes sense given the becoming of the source text, but demonstrates that the model does not really understand its answer.
Opportunities and pitfalls
The table below summarizes the four scenarios.
These examples show clear opportunities for the use of large language models like ChatGPT as a tool for generating flashcards of different kinds. Especially in domains that have natural cue-answer pairs, such as French-English word pairs, an LLM can produce good flashcard lists with reasonable distractors in very little time. In domains where clear, unambiguous cue-answer pairs are more difficult to find, an LLM may struggle more. In these cases, it can help to provide examples of the type of output you expect from the model, or to provide more explicit instructions on how to reason about the output. The prompts in this blog post can be used as a starting point, but can certainly be improved to better fit specific scenarios.
In an educational context, LLMs should be seen as a tool to be used by a teacher, and not a replacement for that teacher. Unlike humans, systems like ChatGPT “do not have any understanding of what they are producing, any communicative intent, any model of the world, or any ability to be accountable for the truth of what they are saying” (Bender & Shah, 2022) . Under the right conditions an LLM can certainly produce flashcards that appear like they were made by a human, but there are also many examples of such models confidently producing false information and perpetuating harmful biases (Zhuo et al., 2023). A teacher using generative AI to create flashcards or other educational content should always carefully evaluate the output: is the generated content factually correct, complete, unambiguous, unbiased, sufficiently varied and of the right difficulty?
The examples in this blog demonstrate that generative AI can indeed make it easier for teachers and students to create flashcards for retrieval practice, when it is used with care and an awareness of its limitations.
