Teachers, educators, and trainers all ask this pressing question – does the gamification of learning work? And so did we at LMU Munich. The gamification of learning has received increased attention and was hyped over the last decade for its hypothesized benefits on motivation and learning. However, as popularity increased, so did critical voices referring to the concept as bullshit or Pavlovication. From our perspective, an evidence-based approach that considers research efforts undertaken within the last years must be applied to provide reliable answers to the question of gamifications’ effectiveness.
Even though considerable efforts within reviews have been made, conclusive evidence synthesizing the state of current research on the effectiveness of gamification on learning outcomes has yet to be provided. Therefore, we approached the initial question about the effectiveness of gamification by conducting a meta-analysis that not only aims at providing evidence on the effectiveness of gamification but also on what types of gamification work best.
Gamification of Learning works!
To answer the initial question does gamification of learning work right away: Our results suggest that gamification might in fact be effective for learning. Our analysis included three different types of outcomes, namely cognitive, motivational and behavioral learning outcomes. Cognitive learning outcomes refer to different types of knowledge. Motivational learning outcomes encompass intrinsic motivation, engagement, and self-efficacy. Behavioral learning outcomes consist of skills or competencies. Based on studies that were eligible for our meta-analysis, results show that gamification can have a positive effect on all three outcomes.
Stable or biased effects?
Within our meta-analysis we run into a common problem that meta-analyses oftentimes have to face: the lack of methodological rigor of primary studies. Such problems can arise when learners are not randomly assigned to experimental conditions or when studies did not conduct pretests, which are particularly relevant for studies without random assignment of learners to different conditions. In general, gamification research to date, unfortunately, oftentimes lacks methodological rigor and thus we were aware of possible methodological problems that might jeopardize conclusions drawn from the data.
For this reason, we conducted additional analyses that only included experimental studies and quasi-experimental studies with pretests to check for stability of the effects. These further analyses were based on quite a small number of studies and showed that the effects on different types of knowledge (cognitive outcomes) were stable, while effects on motivational and behavioral outcomes were shown to be less stable. This calls for more studies that use rigorous methods that gamification research is lacking to date.
Heterogeniety of gamification
Our positive results of gamification on learning should be seen in the light of the heterogeneity of gamification. Research has often ignored that gamification can be applied using a lot of different techniques. Within our meta-analysis we thus included different moderating factors of the effects of gamification on learning, meaning that we tried to find out if certain characteristics of gamified systems worked particularly well. Results for behavioral learning outcomes show that providing narrative characteristics or introduce a game world, both of which include elements of fantasy, are especially promising forms of gamification. Further, while competition can be problematic for some learners, combining competition with forms of collaboration might yield positive results for behavioral learning outcomes. Results indicate that this might also be true for motivational learning outcomes. Exemplary forms of gamification that are effective are the introduction of a narrative combined with an avatar system, which allows for developing an avatar over time. Moreover, the creation of gamified systems that allow learners to work together in teams, while competing with other teams or the implementation of a badge system that allows peers to challenge themselves as well as awarding badges for helping others are promising.
Our meta-analysis was not able to go into detail with certain moderating factors, even though we wanted to do so. The number of studies for different types of gamification was too small to make detailed comparisons. Again, our results call for more research to allow for fully capturing the phenomena of gamification and make evidence-based claims which instantiations work best for certain outcomes.
This is only the beginning
Our meta-analysis offers a general framework for gamification research and should be understood as a stopgap of research performed to date. It is a first resume of research from the last decade and a starting point for – hopefully high quality in the sense of methodologically sound – gamification research.
This framework and resume can be used to as a basis for more theory-guided empirical research that is still needed to work towards a comprehensive understanding of which mechanisms work best for specific learning processes and outcomes.
Sailer, M. & Homner, L. (2019). The gamification of learning: a meta-analysis. Educational Psychology Review. https://doi.org/10.1007/s10648-019-09498-w
The article is available under an open-access license, meaning there is free access for everybody under the following link: https://doi.org/10.1007/s10648-019-09498-w
Michael Sailer, Education and Educational Psychology, LMU Munich, Germany, official website
Lisa Homner, Education and Educational Psychology, LMU Munich, Germany