Engaging in usability testing with younger users demands a careful and considerate approach, especially when working with children. The primary goal of usability testing is to have users interact with your product, allowing close observation of their navigation through a set of tasks. In this way you can test the functionality of your product. In our previous blog post, we shared our firsthand experience and results after testing the usability of our educational app Tafel Trainer with children in a primary school setting. Today, I will delve deeper into our approach of conducting interviews with children, shedding light on the methods we employed to create a secure environment for them to express their thoughts openly. In doing so, we aimed to gather valuable insights for further development and improvement of our educational app.
In the current context, our aim was to assess the usability of our educational app Tafel Trainer, specifically designed for children aged 6 to 12 to practice multiplication tables. The objectives of our study were to uncover features that may be less intuitive for children, measure the engagement level of design elements, evaluate the appropriateness of lesson durations, and identify potential sources of frustration within the app. We highly value the significance of effective product development, leading us to conduct thorough testing for Tafel Trainer across a total of 12 schools. Our testing encompasses two complementary approaches. Firstly, we administered a large-scale survey involving over 600 children who shared their feedback through the UXKQ (User eXperience Kids Questionnaire – Wöbbekind, et al., 2021). While this method provided us with substantial data, it lacked detailed descriptions and in-depth observations. Accordingly, we also carried out smaller-scale interviews with 27 children. This approach allowed us to delve deeper into the children's interactions with Tafel Trainer, providing us with more detailed insights and opinions about our product. Our visits included sessions with children from grades 2 and 3 in two primary schools. Recognizing the young age of our participants, we understood the importance of approaching them with extra care in our communication. Consequently, we undertook thorough preparations to not only ensure the acquisition of necessary information but also to have a comfortable interview in a child-friendly manner.
The significance of establishing connection
We anticipated that establishing a report with the child preceding the interview would significantly contribute to the success of usability testing. We thought this was important for several reasons. Firstly, the initial interaction would help us understand the child's personality, providing a foundation upon which we could build throughout the rest of the conversation. Children vary in confidence levels; some may exhibit assertiveness and outgoing traits, while others may be more timid and less talkative. Consequently, a more outgoing child might thrive with an interviewer who is talkative and maintains an upbeat tempo, while a shy child may prefer a calmer approach, allowing them more time to express themselves. In addition, establishing a connection during the initial interaction allows the child to become more familiar with us. This familiarity fosters a sense of comfort, potentially encouraging them to share their opinions in greater detail and motivating them to approach the interview more seriously and with increased cooperation. Findings from Lewis and Graham's study in 2007 suggest that, due to the unfamiliarity of the interview process, respondents seek guidance from interviewers to establish a sense of 'comfort.' Therefore, creating a comfortable environment is crucial for maintaining testing quality, as it facilitates honest and open responses from interviewees. Additionally, being at ease aids interviewees in better comprehending information and accurately recalling details, as highlighted by Ghosh et al. in 2013. This aspects recall not only contributes to testing accuracy but also encourages a more engaged and active interview experience.
We handled this initial interaction by initiating a conversation about the child's day and recent subjects that they were doing. We discussed their preferences by asking about the subjects they enjoy, or whether they had a favorite (magical) animal, creating a more personalized connection. In return, we shared aspects about ourselves, including the purpose of our visit and some personal insights. We chose to sit beside them rather than facing directly, intending to induce a sense of ease and to avoid any feeling of us evaluating them. It is important to note that these introductory conversations differ with each child, recognizing the uniqueness of their personalities and interests. The initial goal was to transition our relationship from that of strangers to a more familiar connection.
Creating effective questions
While observing the child's engagement with our app is essential, the nature of questions asked to the child could arguably be the most critical element in the entire usability testing process. We encouraged the children to think aloud and articulate their thought processes during specific tasks. We would for instance ask the child to select a multiplication table of their preference and start practicing for a fixed amount of time. We clarified that we were not there to judge or critique their decisions; there were no right or wrong answers. Moreover, we would pose questions such as 'Could you describe your emotions while performing this particular task?'. Every question was intentionally open-ended to obtain more comprehensive responses, and to avoid one-word answers. If a question can be answered with a simple yes or no, a child is very likely to provide an unelaborated response. However, when responding to open-ended prompts, children tend to offer more information (Sternberg et al., 2001). Furthermore, we made a deliberate effort to formulate questions without leading tendencies and worked towards reducing cognitive biases. These cognitive biases, characterized by the unconscious adjustment of choices influenced by external comments or behavior, could pose a threat to the reliability of our study outcomes (Natesan et al., 2016). For instance a question like 'Did you enjoy using our product?' might direct a child toward giving a more positive answer, while that might not be their initial opinion. Furthermore, when assessing the duration of a task within TafelTrainer we asked 'What did you think about the duration of this task?' instead of 'Did you think this task lasted too long?'. This approach encouraged more authentic and unbiased responses, contributing to our efforts to enhance our product.
Insights and strategies
As this was my first experience with usability testing and interviewing children, it provided me with valuable insights, and new thought patterns. My key takeaways are as follows:
- Establish familiarity: Building a connection prior to the interview could lead to more open and honest conversations;
- Adapts to the child: There is no one-size-fits all approach to interacting with children; each child is a unique individual;
- Create effective questions: Obtaining comprehensive answers is crucial for gathering valuable information. Think about: open-ended questioning, avoiding biases and reducing cognitive influences;
- Prioritize significant questions early on: Children's attention spans are relatively short, therefore it is advised to gather the most valuable information early in the interview;
- Give undivided attention: Listen attentively to the child's expressions, their openness can be influenced when they notice a shift in attention.
This overall experience was both highly enjoyable and insightful. I am pleased to share it in this blog post, with the hope that it may inspire others to reflect on my key insights in comparable circumstances.
References
Ghosh, S., Laxmi, T. R., & Chattarji, S. (2013). Functional connectivity from the amygdala to the hippocampus grows stronger after stress. The Journal of neuroscience : the official journal of the Society for Neuroscience, 33(17), 7234–7244. https://doi.org/10.1523/JNEUROSCI.0638-13.2013
Lewis, J., & Graham, J. (2007). Research Participants' Views on Ethics in Social Research: Issues for Research Ethics Committees. Research Ethics, 3(3), 73-79. https://doi.org/10.1177/174701610700300303
Natesan, D., Walker, M., & Clark, S. (2016). Cognitive Bias in Usability Testing. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, 5(1), 86–88. https://doi.org/10.1177/2327857916051015
Sternberg, K. J., Lamb, M. E., Davies, G. M., & Westcott, H. L. (2001). The memorandum of good practice: Theory versus application. Child Abuse and Neglect, 25, 669–681. doi: 10.1016/S0145-2134(01)00232-0
