Filament Games stays connected to the scholarly side of game-based learning (GBL) through our GBL Luminaries Program. Academics in the Luminaries program work with our staff to maintain a healthy dialogue between game-based learning theory and practice through all-staff lectures and guest blog posts. Today’s GBL Luminary guest post is from Jodi Asbell-Clarke – Senior Leader, Research Scientist, and Director of the Educational Game Environments (EdGE) group at TERC, which is an independent research-based non-profit that serves education. Her work is focused on the intersection of games and neurodiversity specific to executive function.
Educational games, and the study of game-based learning, have come a long way in the past decade or more. Back in 2010 or so when I first met Dan White and Dan Norton—two young guys developing an undersea immersion virtual experience called JASON—our own freshly minted team, EdGE at TERC, was building a science mystery game called Martian Boneyards in the new HD virtual world, Blue Mars. Back then VR was booming. People were still trying to prove that games had value in education. Fast forward a dozen years and VR is back, those two guys have co-founded a really cool game studio called Filament Games, and EdGE has grown into a team of designers, educators, data scientists, and cognitive scientists who are pushing how the field thinks about learning assessment, neurodiversity, and games.
Figure 1: Martian Boneyards, a science mystery game in Blue Mars.
With generous funding from the US National Science Foundation’s Division on Research and Learning, EdGE has had a series of projects to design and study games in educational settings. Early on we saw games not only as an avenue to engage different learners but also an innovative way to measure learning, especially from people who were marginalized from traditional types of STEM learning opportunities. In our first game, Martian Boneyards, we were able to show that it was women who conducted most of the scientific investigation in the mystery game even though they made up less than a third of the participating audience. We were able to show this by analyzing the data logs generated by users’ “clicks” on the virtual tools we provided for data gathering, data analysis, and scientific theory building. You can read more about that here.
Since virtual worlds still attracted an exclusive audience back then—mostly white, tech-savvy, males—we next explored tablet games for high school science classes in order to reach a broader audience. We designed a series of games around physics and biology concepts and ran a study where dozens of teachers (and hundreds of students) played the games in class. We collected pre and post-tests related to the same science content as the games, and we also analyzed the data logs to detect what strategies players used in the game. We found that students who used strategies consistent with the science concepts showed more improvement on the tests when their teachers bridged the game-based learning to the classroom content. You can read more about that here.
Figure 2: Impulse, a game about Newton’s Laws of Motion
Measuring the strategies player used in the game was a feat in and of itself. We started by collecting videos and screen captures of a variety of players using each game. Two researchers analyzed the gameplay and labeled each move with an agreed upon set of strategies (e.g. letting a ball float without exerting a force, or redirecting the ball with a force, in the game dealing with Newton’s laws of motion). Once inter-rater reliability was established for these strategy labels, we built educational data mining (EDM) detectors to detect these strategies automatically in each player’s activities. We then continued to train these models with additional log data from the games, establishing confidence levels comparable to the standards for data mining in the medical field. You can read more about that process here.
We applied these data analytics methods in a large-scale study of students playing TERC’s popular learning game, Zoombinis, in a study of the development of Computational Thinking (CT) practices in grades 3-8. Using methods similar to the ones we used in previous games, we were able to show that we could detect practices in gameplay consistent with Problem Decomposition, Pattern Recognition, Abstraction, and Algorithm Design, all fundamental practices in CT. We also showed that the more students played Zoombinis, and the more they demonstrated these CT practices in their gameplay, the more improvement they showed on pre and post-tests of CT. You can read more about that research here.
Figure 3: Zoombinis, a Computational Thinking Game
While working with teachers using Zoombinis and other CT materials, we heard one message repeatedly. They told us it was their students who struggle in other subjects who were becoming the leaders in CT. Those students would figure something in the game and then be the ones to show the rest of the class. They were becoming the ‘go to’ students for CT in the class, and it was really changing how they see themselves as learners, and how the rest of the class treated them. We knew this was something we wanted to explore more.
We are now funded by the US Department of Education, along with partners from multiple universities and non-profits, to build and implement a set of materials for grades 3-8 to build foundational and applied CT practices, with adaptive scaffolds for neurodiverse learners. In particular, we are building supports for working memory and attention. Our goal is to support the executive function needs some neurodiverse students struggle with, while engaging them in CT where they can thrive. You can learn more about the INFACT project here.
We continue to explore new ways to reach learners and measure what they are learning in games, including using VR and multimodal analytics to push for innovation in STEM learning and assessment experiences. EdGE’s journey has not only been supported generously by federal granting agencies but also by the depth of creativity, knowledge, and passion of the game-based learning community. So many colleagues have celebrated, commiserated, and collaborated along the way. We are grateful for friends like Filament Games that keep us inspired and humble.
edge.terc.edu
Jodi_asbell-clarke@terc.edu
@Edge_at_terc
More Luminary insights: