Digital Games, Design and Learning: A Meta-Analysis

Clark, D. B, Tanner-Smith, E.E, and Killingsworth, S.S. (2016) Digital Games, Design and Learning: A Systematic Review and Meta-Analysis. Review of Educational Research 86(1):  79-122.

Within this article, Clark, Tanner-Smith and Killingsworth (2016) offer a refined and expanded evaluation of research on digital games and learning.  To ground their study, the authors summarize three prior meta-analyses of digital games. It is from these three studies and their findings that the authors develop a set of two core hypotheses about how digital games impact learning  that were tested in their meta-analysis. These two core hypotheses were further examined for that the authors term as moderator conditions and from this the authors developed sub-theories for each core theory to also test in their meta-analysis. Utilizing databases spanning “Engineering, Computer Science, Medicine, Natural Sciences, and Social Sciences” the authors sought research published between 2000 and 2012 to identify studies which examined digital games in K-16 settings, which addressed “cognitive, intrapersonal and interpersonal learning outcomes”(p. 82) and had studies which either had comparisons of digital games versus non-game conditions or utilized a value-added approach (something the prior meta-analyses ignored) to compare standard and enhanced versions of the same game. In addition they required a set of criteria for these studies to meet which included specifics on game design, participant parameters, and pre and post testing data which could be used to assess change in outcomes. Overall, they identified 69 studies which met the parameters outlined in their research procedures. From this population they discerned the following signficant patterns:

1. In studies of game versus non-game conditions in media comparisons, students in digital game conditions demonstrated signficantly better outcomes overall relative to students in the non-game comparisons conditions (p. 94). This was significant for both cognitive and interpersonal outcomes (p.95). The number of studies with interpersonal outcomes was too small for statistical significance.
2.  In studies of standard game and enhanced game versions through value-added comparisons, students in enhanced games showed “significant positive outcomes” relative to standard versions (p. 98). While overall there were too few studies with specific features for cross comparisons, the one feature of enhanced scaffolding (personalized, adaptive play)was present in enough studies and showed a significant overall effect (p. 99).
3. Overall in examining game conditions, games which allowed the learner multiple play sessions performed better than those of single game play when compared against non-game conditions. Game duration (time played) seemed to have no impact on overall impact. (p. 99) These results did not vary even when considerations of the visual aspects of the game were measured.
4. Despite what was seen in previous meta-analyses, there was no difference in outcomes for games paired with additional non-game instruction versus those without the additional non-game instruction. (p. 99)
5. There was significant differences with player configurations within games. Overall, single player games had the most signficant impact on learning outcomes relative to group game structure and these outcomes were higher in single player games with no formal collaboration or competition. (p. 100). However games with collaborative team competition had signficantly larger effects on learning outcomes when compare to single competitive player games.
6. Games with greater engagement of the player with actions within the game had greater impact than those with only a small variety of actions of the screen which did not change much over the course of play.
7. Overall the visual and narrative perspective qualities of the games both simple and more complex game designs showed effectiveness in learning outcomes but overall schematic (schematic, symbolic or text-based) games were more effective than cartoon or realistic games

In reflecting on their findings, the authors recognized some limitations present based upon both their search parameters and their methodological breakdowns for analysis and encourage further examination of studies which fell outside of their range (for example simulation games) and greater examination of the subtleties of the individual studies included within their analysis before any larger generalizations can be made as to the specifics of best practices for game design.

Perhaps the most interesting aspect of this study is not the outcomes it presents for future study (even though these are great food for thought about intentional game design for educational purposes) but the proposition it makes that educational technology researchers should “shift emphasis from proof-of-concept studies (“can games support learning?”) and media comparison analyzes (“are games better or worse than other media for learning?”) to cognitive-consequences and value-added studies exploring how theoretically driven design decisions can influence situated learning outcomes for the board diversity of learners within and beyond our classrooms” (p. 116).

 

 

Intentional Design for On Screen Reading

Walsh, G. (2016) Screen and Paper Reading Research – A Literature Review. Australian Academic & Research Libraries, Vol.47(3), p.160-173

As more students move into on-line courses and as more faculty consider incorporating open educational resources (O.E.R) into their courses, the impact of screen reading and learning material design on reading comprehension and overall learning is of essential consideration.  Walsh (2016), desiring to help academic librarians gain knowledge on issues of online reading, examines the current research (last 6 years) with regards to reading comprehension and the screen versus paper debate. Overall Walsh found no consistency in research design among the studies she examined, making cross-comparisons difficult. However, she concludes that “most studies find little differences between the print and screen reading for comprehension” (p 169). But, she notes, most were not focused on scholarly readings and those that did “concluded that participants gain better understanding of the content when reading from paper” (p 169).

Overall, this article offers a synthesis of recent scholarly literature (2010-2016) located in information management databases. While the scope of the study does not specify the exact search parameters used nor if search parameters were used to eliminate any studies from consideration, it does offer a brief overall glance at some of the literature that exists on this subject from an information management perspective. If the author had opened up this research to examine databases within learning, education and educational technology, additional research may have been found. However despite this limited search parameter, the information within this article, when synthesized together, highlights several aspects to screen reading which should be considered within educational technology.

In her article, Walsh (2016) notes that when considering reading and comprehension, neuroscience research suggests that deep reading is necessary for “furthering comprehension, deductive reasoning, critical thought and insight” (p 162) but that there is variation in the areas of the brain which are stimulated by print reading and versus those stimulated by screen reading. This variation may indicate that there may be some impingement upon the screen reader’s “ability to reflect, absorb and recall information as effectively as in formation in the paper form” (p 162) and may encourage more shallow or skim reading. While not specifically addressed by Walsh but when considered further, this information suggests that educators which rely on-screen based reading to help students gain material knowledge for their course may need to develop activities which work to promote deeper reading in students. This is not something students learn early on due to the predominance of paper assigned materials in early education. At the same time, this may not be a skill that can be developed with something as simple as giving them a set of questions to answer after having read. Kuiper et. al (2005) offered that, when examining how students searched the Internet, how the teacher structured the task impacted how the student approached the content. In the case of screen reading, well-structured tasks (to borrow from Kuiper et. al) may support only a seek-and-find strategy and not necessarily support the ability of the student to creatively and critically come to comprehend and synthesize the materials.

Walsh’s review also offers information which shows that the content’s format, intention and its length can impact how much the student may learn from screen reading. Walsh (2016) notes that even though students read off of screens for entertainment, when it comes to academic documents, students prefer to print off a document rather than reading it on the screen. This preference is related to not only the “high level of concentration and text comprehension” necessary but that academic reading also required the reader to interact with the document through annotating, highlighting and bookmarking passages for reference (p 163).  Walsh’s research suggests that students do not perceive themselves as being able to accomplish as much with screen reading of academic documents as print reading.  This perception is critical since even though many students within the studies indicated interest in screen reading, they doubted their own ability to be competent with it. This perception of competence could potentially undermine student interest in engaging with the reading fully. Thus, while Walsh does not specify this within the article, it does recommend that an educator who utilizes screen based academic reading as part of their course may need to offer more guidance to the readers with regards to both how they may engage with the reading (through digital annotation, tagging and bookmarking) and more encouragement for students to build self-confidence in their abilities.  In addition, Walsh (2016) highlights research showing there is very little difference in outcomes of performance between screen readers and print readers for shorter content but that for longer, more complex materials, learning and information retrieval can be impacted when reading from a screen. Furthermore text which were less data and fact based, which were less visual, and required more cognitive reasoning were easier to read in paper format than on-screen. These two points would suggest that a simple transformation of printed text to a digital format for screen reading – a common practice among educators and journals alike, may not be sufficient for materials to be comprehended as easily as the text version. Rather that utilizing technology to optimize the reading experience through visuals, textual divisions, and structured hypertext may benefit the comprehension of more complex longer materials.

Finally Walsh presents research which outlines how the platform characteristics with regards to design, user interaction and navigation can impact comprehension. The research Walsh presents suggest that platform structures not only create technical frustrations but may limit the level of engagement the student can have with the reading or increase the level of distractions they can experience. Not all readings are equally optimized for learning for all students in all platforms. Therefore this could recommend to the educator that careful consideration of platform tools (navigate, annotate, explore), overall student familiarity with a platform and its usability, and the ability of the educator and student to turn off and on hypertext/pop-ups should be considered when selecting for digital materials.

These points, taken together, suggest that educators need to have a more thoughtful, approach to the incorporation of digital reading materials in their courses and that students may be better served by educators approaching onscreen reading with more intentional design than is currently in use.

Additional References

Kuiper, E., Volman, M., & Terwel, J. (2005). The Web as an information resource in K–12 education: Strategies for supporting students in searching and processing information. Review of Educational Research, 75, 285–328