Innovation and improvement are always ongoing in education, and at Filament, we are passionate about helping schools meet any challenge that comes their way through educational games. Our commitment to enhancing learning experiences has led us to partner with numerous schools and educational institutions. In this blog, you’ll find two different case studies, each of which showcases the profound impact of game-based learning, whether it be for experiential learning or student assessment.
Case Study 1: Contents Under Pressure, courtesy of Cheryl Anne Bodnar, PhD, CTDP, Associate Professor of Experiential Engineering Education (ExEEd) at the Henry M. Rowan College of Engineering
Project Overview
The game was designed to provide an immersive environment where students could engage with making process safety judgments without the fear of potentially devastating consequences like they would experience in an actual chemical plant facility.
Impacts
There have been several publications now that have documented the evidence of the effectiveness of the game:
In this study, researchers utilized two methods to assess the ethical decision-making of senior chemical engineering students within a process safety context: the Engineering Process Safety Reasoning Instrument (EPSRI) and Contents Under Pressure. Despite both interventions presenting similar ethical and process safety prompts, they differed in format—the EPSRI as a traditional electronic survey and Contents Under Pressure as a digital immersive game.
Researchers compared the responses of 148 chemical engineering seniors across three institutions. When engaging with the EPSRI, a majority of students demonstrated post-conventional reasoning, suggesting an awareness of the instrument’s ethical framing and potentially influencing responses to align with perceived “right” answers. In contrast, responses to Contents Under Pressure differed significantly, with students providing more typical conventional responses. This suggests that digital immersive environments can elicit more realistic responses, showcasing their potential to enhance the authenticity of ethical decision-making assessments.
- A retrospective analysis on the impacts of an immersive digital environment on chemical engineering students’ moral reasoning
In this study, researchers evaluated the impact of Contents Under Pressure on moral reasoning through a retrospective study that compared two cohorts: one receiving traditional process safety instruction and the other incorporating Contents Under Pressure into their instruction. Utilizing the Engineering Process Safety Reasoning Instrument (EPSRI) for pre- and post-assessment, the analysis revealed a distinct and statistically significant enhancement in process safety reasoning among participants exposed to Contents Under Pressure, highlighting the effectiveness of digital immersive environments in influencing ethical decision-making in process safety contexts.
Usage Numbers
Contents Under Pressure has been employed with approximately 854 senior ChE undergraduate students across the four partner institutions (North Carolina State University, Rose-Hulman Institute of Technology, Rowan University, and University of Connecticut). Contents Under Pressure has additionally been used with more than 250 students ranging from first-year to senior level ChE students across eleven other institutions external to the partner institutions by faculty interested in the tool.
Research Engagements
There have been a number of both journal publications and peer-reviewed conference publications focused upon Contents Under Pressure. This website has a summary of all current publications.
Highlights
This video does a great job of capturing insights about the game and includes direct student feedback.
Lessons Learned
We learned that there is a lot that goes into designing an effective and engaging game. Filament was great in helping us move through the game design process step by step while ensuring that the benefits of games (art, story, sound effects, etc.) did not get lost during the design process.
Future Direction
We are currently seeking to add more branching to the existing storyline and also looking at changing around some of the character art to see the impacts this has on student approaches to process safety judgments.
Case Study 2: Beats Empire, courtesy of Nathan Holbert, Assistant Professor of Communication, Media, and Learning Technologies Design at Teachers College, Columbia University, and director of the Snow Day Learning Lab
Project Overview
The Playful Formative Assessment for Computer Science in NYC (PFACS) project iteratively developed and studied a game-like assessment system to be used by middle school students periodically during their time enrolled in one of many CS modules (that is situated in a Math or Science class) deployed throughout the NYC public school system. Research on this system examined how gameplay can engage students in playful construction tasks that elicit computational thinking about data.
A core component of this system was the game Beats Empire! Beats Empire is a formative assessment game that engages players in thinking about and with data. In the game players take on the role of a music studio manager and, using data, make decisions about what artists to sign, what songs to record, and where to release new music. As they build their music empire players must use various data representations to understand the diverse markets and how to craft new music to fit market trends.
We also designed a teacher-facing dashboard that provides live data on how students are using the game, and provides teachers with actionable suggestions for how to support students as they build a sophisticated understanding of the various data representations in the game.
Impacts
We wrote a book about this project! “Playful Testing: Designing a Formative Assessment Game for Data Science” describes our design process, the unique challenges building a formative assessment game (rather than a learning game), and shares some of the key data of the project. You can read it for free here.
Within the book, researchers provide an analysis of Beats Empire concerning authenticity, playfulness, and cultural relevance. The analysis identifies both achieved thresholds and areas for further development in each concept. The book details how Beats Empire successfully aligns with expectations of stakeholders, providing an authentic experience for students and industry experts. In terms of playfulness, the game excels, displaying characteristics often lacking in serious games. Additionally, it demonstrates alignment with cultural relevance, utilizing students’ assets and enabling them to showcase complex skills and knowledge in a performance assessment context.
Usage Numbers
To date, we’ve had over 10,000 users!
Research Engagements
Refer to the impacts section.
Highlights
I was quite proud of how this game turned out–we created a game that is genuinely a lot of fun to play! The game mechanics, and the connection to music and youth culture, were highly compelling to learners. My favorite anecdote… we were running an implementation in a middle school math classroom. Two girls were sitting next to one another playing Beats Empire on their own computer. However, right before one would release a song in their game, they would hand one of their earbuds to their neighbor. Then as they released the song, and the beautiful music that was composed for the game played, the two girls would groove their heads to the beat. Seeing that Beats Empire fit so neatly into what it means to be a kid learning about and sharing music made me very happy.
One real challenge in this project was to figure out how to design an assessment game that would also be open-ended and student-centered. We accomplished this! The game provides a rich space for students to play with and explore data in a way that they find meaningful and compelling. And through that experimentation, teachers can gain actionable insights into what students understand and what they’re still struggling with.
Lessons Learned
The assessment experts on our team really came to appreciate how we were able to create personally meaningful moments for assessment. The assessment wasn’t some one-off question to be answered because a teacher demanded it. Rather, kids were playing a game they enjoyed, and playing the game in a way that fit their interests and goals. That’s huge!
We also came to appreciate having experts from all aspects of the project on the core team. In our work model the learning and game designers, the assessment experts, the developers, etc were all working together throughout the project. This meant that everyone had input in the game’s design, but also ensured we each took responsibility for the whole game–how all the various systems would work together–rather than just our own area of expertise. This made for some heated meetings, but in the end, the game is much better for it!
Future Direction
We have a few things in the works, but nothing I can report here…
With every project we’ve taken on at Filament, we’ve witnessed how games bring a new dimension to learning, fostering a player’s engagement, collaboration, and understanding of complex topics. We remain dedicated to our mission: to empower schools and educational institutions with innovative, interactive, and impactful game-based solutions.
If you want to bring the magic of educational games into your school, don’t hesitate to contact us. Together with our school partners, we look ahead to an exciting future where game-based learning continues to make a difference in the lives of educators and students alike!
Learn more about working with us: