In collaboration with the University of Groningen (UG) and Noordhoff, MemoryLab is using AI to facilitate maths education. At the MathPsych/ICCM/EMPG 2023 conference prof. dr. Niels Taatgen (UG) presented the Graaf Tel Project (Taatgen, Blankestijn & van Rijn, 2023). This initiative explores the potential of a hybrid approach to student models to improve student learning outcomes in maths. This blog post presents the insights gained from the project and the promising endeavours that lie ahead.
Embracing a Hybrid Methodology for Better Learning
Graaf Tel challenges traditional student models used in cognitive tutors. Student models track a learner’s progress, give feedback and select exercises based on skill level. It is difficult to confirm whether a model actually aligns with the needed skills to understand the material. Instead of a modeller choosing the knowledge states, Graaf Tel adopts a hybrid methodology using bottom-up machine learning techniques to construct knowledge states. A knowledge graph is constructed from a vast set of student maths entry test data, where each node corresponds to a student’s knowledge state, reflecting their proficiency in underlying maths skills. This data-driven approach helps expose knowledge states based on individual differences in performance on different maths exercises, optimising the learning progression.
The Graaf Tel dashboard. The left panel gives an overview of test performance in a topic based manner. In the middle, the knowledge graph, each circle represents a node (i.e., a knowledge state with associated underlying skills). The bars below the nodes indicate the accuracy on entry test exam exercises that belong to the knowledge state. The right panel gives an overview of skills that combine into knowledge states.
A Skill-Based Approach to Guiding Student Learning
A knowledge graph was created for the entry-level maths course for MBO students in the Netherlands, using data from 413 students. The project found that students’ mastery of maths subjects is better explained by general underlying skills (e.g., general arithmetic, reading, multi-step reasoning) rather than specific topics (i.e. topic-to-topic progression in a paragraph-by-paragraph manner as seen in maths education books). Namely, students differed on mastery of general underlying skills instead of specific topics. With a shortage of maths teachers in secondary education (Times, 2023), the project highlights the significance of supporting educators by facilitating helping and understanding hardships of multiple students in a shorter time frame. The emphasis on foundational skills and a skill-by-skill approach can address students’ maths difficulties effectively.
A Collaborative Endeavor between Research and Practice
The interesting findings of the initial Graaf Tel stages can be attributed to the effective collaboration between the UG, Noordhoff, and MemoryLab. Dr. Niels Taatgen designed the algorithm, Noordhoff provided maths exercises and test data, and MemoryLab conducted research in classrooms. MemoryLab UX researcher and developer, Jori Blankestijn, designed and developed the tool in collaboration with mid-level vocational education (MBO) level 4 teachers.
Valuable Insights in Student Learning Difficulties
To allow students and teachers to use this theoretical insight in practice, the classroom, an interactive dashboard representing the knowledge graph was created. The dashboard was used in two schools, and received positive feedback from teachers and students alike. Teachers appreciated the time-saving aspect of the tool, as it helped them identify specific student difficulties more efficiently. Students found the tool easy to use, and its gamified element served as a motivational factor.
A Bright Future Ahead
The current pilot serves as a strong foundation for future developments of the education system. Considering its promising outcomes, the UG and Noordhoff wanted the project to continue and display quantitative learning benefits for students and teachers. Therefore, Corné Hoekstra (PhD student at the UG) will be taking the research lead on the project while MemoryLab continues the development of the tool.
The collaborating parties are happy to announce that the second pilot is set to launch in September! Therein, student progress will be displayed when working through assignments. Teachers will receive breakdowns of sub skills as well as other breakdowns to investigate underlying student difficulties. The goal is to provide insights into students’ learning patterns and personalised learning experiences to show increased student learning outcomes. Building on the outcomes of the initial project we are looking forward to further developments of the upcoming pilot!
Stay updated on future developments and milestones by following the MemoryLab social media channels.
Taatgen, N., Blankestijn, J., & van Rijn, H. (2023, July). Identifying cognitive skills in student data with an application in education. Abstract published at MathPsych/ICCM/EMPG 2023. Via mathpsych.org/presentation/1025.
Times, N. (2023, January 30). Netherlands struggles to find qualified teachers; Shortage hits Amsterdam hard. NL Times. https://nltimes.nl/2023/01/30/netherlands-struggles-find-qualified-teachers-shortage-hits-amsterdam-hard
