Over at Gamification.co, Gabe Zichermann kindly responded to my review of his book Gamification by Design. At the risk of tiring even the most steadfast of readers, I would like to respond to the points he made there.
First, some clarifications
“Sebastian is a vocal critic of the Gamification industry.”
No, I am not. I have stated time and again – during my previous talks, in the review, and in my response to O’Reilly’s reaction – that I see value in gamification done well, that I highly respect certain people and companies in ‘the industry’, and that my critique is not aimed at gamification as such, but at certain ways it gets presented and implemented. I understand it would be an easier story if this were just “pro vs. against gamification”, rather than “pro gamification, but against the specific rendition in Gamification by Design“, but that’s how it is.
“Unfortunately, he turned down our expenses-paid offer to come and share his unedited, critical opinion directly at GSummit, choosing that moment to level his criticism of my book and our widely proven methodology online instead.”
Indeed, Zichermann kindly offered me to join the Gamification Summit on the Summit’s expenses. As he will also remember, I replied to him that I would likely not be able to make it since I had at the time of his email already submitted a proposal for a gamification roundtable at the bi-annual DiGRA conference on game research, scheduled in parallel to the Gamification Summit. The proposal got accepted, hence I couldn’t make it to New York, which I did let him know.
“Despite our differences, and many suggestions to the contrary, I believe Gamification’s critics share a common goal with its advocates: to make the world a better (and more fun) place, … Perhaps my optimism is as naive as some of our critics’ personal advancement strategies, but I remain firmly committed to these goals and excited about the great future we’ll create together.”
This statement implies that the true motivation behind my review was “personal advancement”, not the actual concerns I stated here. But since, as I also stated, my review likewise unintentionally slipped into the personal once (similarily trying to guess an underlying motivation), I’m happy to call it even with that.
So moving on, the reply states that my critique falls in three categories: errata, plagiarism, and differences of opinion on theory and practice. Let’s look at those point-by-point.
It is interesting to put this as the first category of critique, as it implies that this is the most important part of the issues raised. In my review, I counted all in all two things which can be seen as errata: The misreading of the social engagement loop diagram, as mentioned by Zichermann, and a typo in the name James L. Halperin.
The remainder – and the overwhelming majority – of the review points out either underlying conceptual flaws, or systematic misrepresentations, and what I see as problematic with them – such as positively endorsing to obscure the actual value users get out of an interaction with a brand; using a misunderstood, obsolete typology of players to understand the motivation on non-game application users; or calling things “game mechanics” that have nothing “gamy” to them.
So I’m happy to submit the “James L. Halperin” typo to the official errata page, as Zichermann suggests (the engagement loop is already there), to free him up to respond to my substantive critique of said flaws if he wants to.
I did not write the review with the intent of accusing Zichermann of plagiarism. The title of the review points chiefly at the writing and structure of the book, one that – to my eyes – displays a hasty collection of things mentioned in the discourse surrounding “gamification” without a coherent flow of argument underneath, putting incompatible bits side-by-side, like Csikszentmihalyi’s flow theory and reinforcement schemes. The main issue of my review are those conceptual flaws.
Furthermore, I did not accuse, but pointed out case-by-case, open to any reader to see for herself, where I saw no due credit given, which is a different thing than plagiarism, like for instance the unmarked verbatim copy of a piece of a blog post by James Halperin I also pointed out (see the post here and the book on p. 83). I concluded from the amount of these instances that the book is “lacking due credit to an extent that borders on plagiarism” (highlighting added).
As for the slides by Amy Jo Kim, which I gave as one example for such lacking credit, Zichermann states that both his slides
“(and Amy Jo’s) are based on substantial prior research. For example, the player types research comes from Richard Bartle and the stages of mastery work from Oppenheimer and Oppenheimer. One can also find inspiration for the viral loops from Penenberg (and myriad others) and game design action verbs from Lopes, Kuhnen et al – without taking anything away from Amy Jo.”
Indeed, Lopes and Kuhnen have written a nice text on “Game Design Cognition“, Dreyfus and Dreyfus have layed out a model of five stages of skill acquisition (I assume Zichermann means Dreyfus and Dreyfus , not Oppenheimer and Oppenheimer – and I happily grant that this is an honest typo for sure ). And to Zichermann’s credit, he has already used Bartle’s player types in his 2009 book Game-Based Marketing, and his player journey exercise replaces Kim’s three stages of online community with Dreyfus & Dreyfus’ five stages, and also features some additional design exercises (like ‘find out what your Bartle type is’).
But I would ask him to kindly point me to the place where any of these people before Amy Jo Kim suggested to create a gamified design in a series of exercises entailing to first write the player story, then rank their top 5 actions from a list of verbs (matching almost all words on his list, but with no match with the game verbs Lopes and Kuhnen mention), then place those actions on the Bartle diagram, then place them on progression of levels of expertise – in that precise order?
And yes, one can find inspiration for talking about loops and virality from other sources. (Personally, I find the work of Kontagent on social game metrics informative, even beyond social games.) But again I would ask Zichermann to point to the place where Penenberg’s Viral Loop with its three categories of viral expansion loops and features of viral businesses (stackability, point of nondisplacement etc.), or any other person in the viral marketing space, has modeled a “social engagement loop” with that exact name and exactly four phases exactly named “visible progress/reward”, “motivating emotion”, “social call to action” and “player re-engagement” exactly in that order. The only way the model in Gamification by Design differs from Amy Jo Kim’s is that it removes the brackets around “social” in “(social) call to action” (see Kim’s presentation slide 140 and the book on p. 68).
Finally, Zichermann notes that “Perhaps the choice of omnibus citations at the outset of the book and bibliography creates some ambiguity, but the current standard is not to burden the text with dissertation-style notes inline.”
That is a curious statement. I happen to own quite a couple of similar design/strategy books from O’Reilly – excellent reference as well as introductory material written for similar audiences, like Web 2.0: A Strategy Guide, The Art of Community, Designing Social Interfaces, Designing Web Interfaces, or Building Web Reputation Systems, as well similar books by other typical industry publishers, like New Riders, Rosenfeld Media, or Morgan Kaufman.
A standard that unites all these books is that they reference used sources and recommend further reading – with a hyperlink where possible –, on the page and/or at the end of a chapter. Even the most hardcore plain-English-actionable-introductory-business-focussed books like Web Design for ROI do so. In contrast, Gamification by Design does not, has no bibliography, as Zichermann claims (or I didn’t find it), and the acknowledgement section names three books all in all. Instead, Gamification by Design funnels readers repeatedly into Zichermann’s own site GamificationU.com.
As I stated in my review, the annoying thing here is not lack of credit, but self-linking, together with the additional work created for the reader to find the sources the book merely mentions in passing, which incidentally also makes it harder to cross-check what the sources say with what the books says (like cross-checking with Lopes and Kuhnen, Dreyfus and Dreyfus, or Penenberg above, which Zichermann did not link to in his post).
3. Differences of Opinion on Theory and Practice
The third point of contention Zichermann sees is both a ‘naive overreliance’ of my views on research, and mere differences of opinion on what research says. Quote:
“It is spectacularly naive to suggest that research – by mere virtue of its publication – is somehow “the one truth”. Almost every piece of work in social science and psychology has significant methodological problems, and opinions about what works (and why) go in and out of fashion as quickly in academia as they do on the runways.”
I do not and did never claim that published research is “the one truth”, and I would not know of any researcher who does. As a pragmatist myself, nothing could be further from where I stand. As I said in another exchange we had on this, “The whole point of scientific constructs is to point to stable regularities in intersubjectively observable and reproducible data, and to define and explain them in a network of other constructs and relations to make them as robust, inflexible, noncontingent as possible.”
There is a difference between a contentious field in science, with good argument and data supporting both sides, and a field were all data and argument unilaterally point in one direction, and coincide with what practicioners have learned from experience. The importance of mastery, achievement, competence as a core component of what makes games “fun” is such a case. Again, I invite Zichermann to point to but one peer-reviewed empirical study that states that incentives and status, not mastery is at the core of what makes games fun.
There is a difference between “academics believe” and a scientific community – after a long process of debate, experiments and replications, counter-arguments, further experiments refuting those counter-arguments, meta-reviews statistically teasing out the effects found by all experiments in the field – finally settling on a certain body of knowledge as ‘well-established’. The research on the “hidden costs of extrinsic rewards” is such a case. Is that body of knowledge always incomplete and open to amendment? By the very definition of what science is. Is it also the most reliable, acid-tested piece of knowledge on the topic out there? It is. If push comes to shove, would I want to base my business decision on that knowledge, or solely on someone’s “personal experience” with his clients that the product he sells ‘works’? You decide.
There is a difference between “going in and out of fashion … as they do on the runways” and scientific progress. The cognitive revolution, replacing behaviorism with cognitive psychology as the dominant paradigm in psychology, was a slow, 25-year process from the mid-1950s to the early 1980s, working its way from argument to argument, experiment to experiment. And today’s neurosciences or evolutionary psychology build on the work of cognitive psychology, they do not just randomly flip back or bounce to some previous opinion.
Finally, there is a difference between having different “opinions” and misrepresenting what the other side says and the evidence it brings to bear – i.e. Csikszentmihalyi, Lazzaro, or the research by Amabile, Dweck, Ryan and Deci summarized by Pink, all named in Gamification by Design. For example, in Zichermann’s presentation of said summarizing book by Dan Pink, Drive, he states that “While we agree that cash is not always motivational and in some cases is actually demotivational, other extrinsic rewards can be astoundingly motivational for players. For example, long-term social status rewards can be particularly effective at nurturing creativity and play.” (p. 27)
This misrepresents both that Pink and the research he references talk about extrinsic, tangible if-then rewards in general, not just money (also certificates, trophies, food, badges, etc.), and that the researchers Pink cites explicitly studied the effect of caring about “long-term social status rewards” for creativity, and found that it was detrimental both for the commercial success and the personal well-being of the artists they studied. Zichermann may hold a different opinion on why those studies found those results (for instance, “flawed methodology”), or why they don’t matter in gamification contexts, but he should at least present them correctly, so that the reader make make up her own opinion.
Theory versus practice?
Zichermann’s reply attempts to cast a stark theory-vs-practice gap, with me, the “PhD student”, who apparently demands industry-atypical “dissertation-style notes”, “lack(s) practical business experience from which to conjecture”, and merely “believes” that “the techniques in the book work” or not, in contrast to “the doers” like Zichermann, who has written “a practical book for practical purposes”, “based on my experience with dozens of clients” and “interviews with hundreds of practitioners”.
This is a false contrast for at least three reasons:
1. Research holds enormous values for practical business decisions, and in my past presentations on the topic, I have always tried to demonstrate that connection to direct business cases. For instance, as I also noted in the review, research reliably shows that overall, men prefer competitive situations much more than women do. The startup Fanlib.com employed an early version of “gamification” by trying to motivate fan fiction authors to submit their work through competitions and sweepstakes, which stirred a huge backlash within the fan fiction community, and the startup quickly folded (see references here, here, and here). Now I don’t claim that there weren’t also many other things problematic with Fanlib.com, but had they known the research on competition, and the research on fan fiction showing that the overwhelming majority of writers are female, they might have done a smarter job. Why would anyone not have her business decisions informed both by practical experience and existing research? I agree from my own experience that business decision makers don’t have the time for a thick book and are in need of good, actionable introductory summaries. But what use is a good summary of flawed thinking?
2. As Kathy Sierra noted, my critique is not ‘one lone student’s personal opinion’. Numerous practicioners highly respected in the gamification field have publically supported my critique of the arguments presented in the book, or fashioned their own critique, such as Buster Benson or Tadhg Kelly. Jon Radoff, who spoke at the Gamification Summit, likewise publically spoke up multiple times against reducing what works in games and gamification as behaviorism.
3. As for myself, I have worked several years in the industry as a program manager and user experience designer, and continue working on projects increasingly involving the use of game design beyond games. Among them, I have helped the world’s second-largest online retailer optimise their check-out process, and worked with browser games on improving the conversion of their onboarding and virtual item sale processes – in user experience design, it doesn’t really get more ‘down to business’ than that. I happily invite everyone to ask my past employers and clients whether my work lacked practicality or business acumen.
To conclude, the argument my review makes is not a simple me “believing” that gamification ‘does not work’ versus Zichermann’s “experience with dozens of clients” that ‘it does’. As I stated both in the review and elsewhere, all things he mentions have some relevance: learning by association is important, incentives work, and social status motivates. My argument is:
1. The book Gamification by Design is seriously underplaying and misrepresenting the existing research and business cases that incentives are of limited effectiveness and often come with unwanted side effects, for no apparent reason – businesses only benefit from becoming smart about these things.
2. The book is seriously overplaying the importance of social status and neglecting the complications and prerequisites for something becoming a meaningful conveyor of social status.
3. By presenting reinforcement, incentives and social status as “the core psychology of what makes games compelling”, the book is contradicting both all peer-reviewed empirical research on the core motivational appeal of video games and play, and all game designers I know, have read and spoken with, including the researchers and designers the book points to as important sources.
4. By doing so, the book is seriously underplaying the existing research and business cases on the elements of games and play beyond incentives and status that make them highly engaging, and have been shown to be crucial for engagement in workplaces and education alike – components like achievement and competence, autonomy, trust and safety, or meaning and purpose. Thus, it needlessly leaves on the table for its readership the huge motivational potential of games, play, and extending them beyond games.
To all of this, Zichermann may counter that his book is “focused not on games at all, but Gamification as a unique, emerging and hybridized discipline.”
Now I have nothing against studying or deploying customer loyalty and employee rewards programs, long practiced and promoted in marketing and human resources by bodies like the Enterprise Engagement Alliance – I just hope that the people doing it take notice of what psychology has learned in the past decades about human motivation. I have also nothing against Zichermann branding his own variant of this, and I welcome his co-founding of his own Engagement Alliance, as this indeed helps to clarify the space he is coming from and aspires to.
My issue is: Why, when Zichermann is indeed “focused not on games at all”, then claim that his book brings to businesses “the core psychology of what makes games compelling”, and call what he proposes “game”-anything, namely “gamification”, and claim that it is “the process of game-thinking and game mechanics”? Why such forceful repackaging against the majority of both researchers and design practicioners? Why not inform readers about the complications and caveats involved? Why not be more careful in presenting what knowledge and experience is already out there? Why leave so much value for readers on the table?
I am happy to continue the discussion on these matters, as I agree it will be a teachable moment for all.