A 10-Year Systematic Review on the Incorporation of Digital Games for
Multimodal Literacy Education
YANAN SHEN*, HABIBAH AB JALIL, RAHIMAH JAMALUDDIN
Faculty of Educational Studies,
Universiti Putra Malaysia,
Serdang, Selangor, 43400,
MALAYSIA
Abstract: - Digital games increasingly penetrate young people’s lives and have become a prominent tool for
21st-century teaching and learning. While digital games are promising resources for fostering multimodal
literacy, empirical evidence of their application in educational settings remains limited. This study conducted a
10-year systematic review of 20 relevant empirical journal articles to provide a comprehensive understanding
of utilizing digital games in multimodal literacy learning across primary, secondary, and higher education
levels. The findings revealed the main contextual features, game features, theoretical and methodological
perspectives, and the possibilities of digital games for multimodal literacy learning. In conclusion, the extant
empirical studies prove that digital games are legitimate learning resources to foster students’ multimodal
literacy, whereas limitations are also evident. Correspondingly, this study provided recommendations to
address these limitations and leverage digital games for 21st-century literacy education, thus contributing to
computer research in educational contexts.
Key-Words: - Digital Games, Digital Game-based Learning, Multimodal literacy, Literacy Education, Literature
Review, PRISMA.
Received: February 23, 2024. Revised: August 19, 2024. Accepted: September 13, 2024. Published: October 24, 2024.
1 Introduction
The rapid advancement of digital media and the
ubiquity of multimodality have fundamentally
changed how people process information,
communicate, and connect with the world, [1].
Correspondingly, the recent two decades have
witnessed the reconceptualization of print-based,
language-centered literacy into multimodal literacy,
which highlights critical, appropriate, and creative
consumption and production of context-specific
discourses made of multiple semiotic modes (e.g.,
language, images, sounds, and gestures), [2]. For
young people, multimodal literacy is crucial in
effectively navigating the technology-driven and
media-rich world, consuming value-laden
information with a discerning eye, and contributing
new knowledge as an active social participant, [3].
Multimodal literacy is mindfully nurtured rather
than naturally gained due to its sophistication.
According to [3], multimodal literacy accentuates
contextual and textual knowledge of distinct
affordances (i.e., advantages and limitations) of
different semiotic modes and their orchestration as
coherent multimodal texts appropriate for a specific
social context. Engaging in multimodal practice,
multimodally literate people can conceptualize a
metalanguage to describe how semiotic modes enact
meaning from representational (presenting ideas and
events), interpersonal (enacting social relations),
and compositional perspectives (combining meaning
into a coherent ensemble), [4]. Beyond that, they
can interrogate semiotic choices from functional
(e.g., causes and effects of using semiotic modes)
and critical dimensions (e.g., the preferred
viewpoint embedded). Furthermore, they should
have the initiative and skills to transfer their
semiotic knowledge and experience into new
multimodal products for self-expression.
Unfortunately, young people’s varied and casual
engagement in digital media practice indicates that
the young do not inherently possess proficient
multimodal literacy, [5].
Researchers and educators have incorporated
authentic multimodal texts (e.g., digital books,
films, advertisements, and webpages) to address
school learners’ multimodal literacy at different
levels of education, [6]. As [7] and [8]
conceptualized, digital games have gained
recognition as “a multimodal literacy par
excellence” among diverse resources, given their
prominent multimodality, interactivity, and
popularity among the youth. Ideally, digital
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
463
Volume 12, 2024
gameplay offers an embodied and authentic learning
environment for agentive, situated, progressive, and
social literacy learning in a multimodal semiotic
domain, developing criticality in multimodal
meaning-making. Furthermore, digital games offer
an affinity space (social groups with a shared goal)
for students’ social interactions, collaboration, and
construction of multifaceted identities, addressing
the socio-cultural nature of multimodal
communication. Meanwhile, digital gameplay
allows students to be co-designers of meaning by
applying multimodal resources and digital tools to
create new products within the games, [7], [9].
Playing games also engages students in making
game-related multimodal compositions, [10], also
called game paratexts (e.g., game walkthroughs and
reviews), [11].
Despite theoretically sound, empirical evidence
is lacking to substantiate digital games as valuable
resources for multimodal literacy learning, [12]. The
insufficient empirical knowledge fails to address the
practitioners’ concerns about game selection,
theoretical basis, pedagogical approaches, and
learning outcome evaluation, [13], making digital
games undervalued in educational practice. Given
the thriving gaming industry and advancing gaming
devices, digital games continue to penetrate the
youth’s lives as an influential cultural force [8],
simultaneously providing new resources and
opportunities for multimodal literacy practices.
Hence, it is worthwhile to fully grasp the
possibilities of digital games in multimodal literacy
education to echo students’ daily media experiences
and cater to their new demands.
This study aims to synthesize the extant
empirical evidence from the recent 10-year literature
exploring how digital games inform multimodal
literacy education at different levels. Of central
interest to this review are school learners across
elementary, secondary, and higher education and
learning activities in and out of classroom context,
thus offering a holistic view and practical
implications for researchers and educators across
sectors and contexts. The scope of incorporating
digital games encompasses playing any game on
electronic devices and transferring the digital game
experience to any form of game-related media
products. Accordingly, the paper is guided by the
following research questions (RQs):
RQ1: What are the contextual features of the
extant literature exploring students’ multimodal
literacy through digital games?
RQ2: What are the main features of the digital
games utilized in students’ multimodal literacy
practice?
RQ3: What theoretical perspectives
underpinned the exploration of digital games for
multimodal literacy practice?
RQ4: What research methods were applied to
explore digital games for multimodal literacy
practice?
RQ5: What opportunities can digital games
offer for students’ multimodal literacy learning?
2 Related Works and Gaps
Previous systematic reviews discussed the
pedagogical affordances of digital games for literacy
practices from multiple perspectives. [14] and [15]
systematically reviewed journal articles about
digital game-based language learning and
substantiated that digital games of various genres
facilitate language comprehension, interactions,
motivation, and engagement. However, these
reviews mainly focused on using digital games as
auxiliary tools to enhance linguistic-centered
literacy instead of multimodal literacy. [16],
thoroughly analyzed 30 research articles about K-16
students’ game design as literacy practices. Their
synthesized findings proved that game design
fostered diverse multimodal literacy skills.
However, this review did not include digital
gameplay and game-based multimodal
compositions, which are more prevalent, accessible,
and relevant forms of multimodal literacy learning
through digital game-based learning (DGBL). [17],
reviewed 77 articles on DGBL in K-12 literacy and
language learning contexts. This review discussed
game features, theoretical foundations,
methodologies, and empirical evidence of
cultivating multimodal literacy through digital
game-based activities such as multimodal
composition and game design. Notably, this review
did not include higher education contexts. Besides,
the specifications of the research design and
learning outcomes were limited.
Overall, the previous systematic reviews
manifest the great research enthusiasm surrounding
digital game-based literacy learning and yield
illuminating empirical evidence of its incorporation
in various educational settings. However, there is a
lack of a thorough overview of how digital games
were employed in multiple ways across educational
levels for multimodal literacy learning with different
focuses. Therefore, this study endeavors to extend
the scope of inquiry by classifying and analyzing
past studies regarding contextual features, game
features, theoretical perspectives, methodologies,
and possibilities of digital games in multimodal
literacy learning.
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
464
Volume 12, 2024
3 Methods
This study was guided by the PRISMA 2020
guideline [18] to search, identify, select, analyze,
and synthesize relevant literature in Web of Science
(WOS) and Scopus (Figure 1). The keywords were:
(“multimodal literacy” OR “multimodal” OR
“multiliteracies” OR “new literacy” OR “media
literacy” OR “digital literacy” OR “game literacy”)
AND (“digital game” OR “video game” OR
“computer game” OR “game-based learning” OR
“gameplay” OR “game paratext”). A total of 20
articles (Table 1) from 1484 results were selected
for this systematic review. The inclusion criteria are
full-length empirical studies incorporating existing
digital games in learning activities relevant to
multimodal literacy among primary, secondary, or
university students. These articles should be written
in English and published in peer-reviewed journals
between 2013 and 2023.
This study applied content analysis [19] to
classify and analyze the selected articles based on
the five research questions. For contextual features
(RQ1), the researchers categorized the participants
into primary, secondary, and higher education levels
and recorded the sample size, specifying the
reported focal cases. Also, they classified the
research contexts by country and their specific
educational setting (i.e., incorporated in school
language courses, workshops at school, or out-of-
school). Second, game features (RQ2) were coded
from various dimensions. The researchers classified
the digital game into commercial games and serious
games according to [20]. The learning principles of
video games [7] guided the framing of game
characteristics. To address RQ3 and RQ4, the
researchers synthesized each reviewed article’s
theoretical perspectives and methodology (including
data-collection methods, interventions, and data
analysis methods), highlighting the similarities,
advantages, and limitations. Concerning RQ5, the
researchers coded the learning outcomes relevant to
multimodal literacy [1], [3].
Fig. 1: PRISMA diagram for paper selection
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
465
Volume 12, 2024
Table 1. An overview of the selected articles
4 Findings and Discussion
4.1 Contextual Features
As illustrated in Table 1, the extant literature has
probed multimodal literacy practice through DGBL
among school learners across primary (n=4),
secondary (n=12), and higher education levels
(n=4). Secondary education received the most
attention, echoing the findings of [16]. This may
relate to the pedagogical emphasis on nurturing
higher-order cognitive skills, collaborative abilities,
and cultural identities among secondary-level
students who are at a pivotal stage of cognitive and
social development, [40]. By contrast, primary
education prioritizes basic literacy skills, while
higher education addresses more specialized content
areas relevant to one’s career path. Moreover, the
reported sample sizes were small, primarily
focusing on just one or two individual or group
cases, indicating the limited generalizability of these
studies.
Concerning research locations, the selected
articles were mainly situated in Western countries,
especially the US (n=10), followed by Spain (n=3)
and Australia (n=2). A plausible explanation of the
Western-centric research contexts is the advanced
technological infrastructure (e.g., computer labs and
game consoles) equipped in these developed
countries, where students have more access to
digital devices for interactive DGBL. Besides, most
studies (n=17) explored students’ game engagement
in school contexts, among which four organized
game-based workshops and 13 studies integrated
game activities into existing language courses. For
instance, [38] applied World of Warcraft in the first-
year composition course at a university, while [31]
incorporated Minecraft in the fundamental literacy
course of a high school. This proves the legitimacy
of digital games as literacy practice in different
levels of formal education. Besides, it indicates the
flexible curriculum system and supportive school
policies within the research contexts. These factors
encouraged the application of digital games in
classrooms, thus stimulating research in this field.
Moreover, studies explored students’ digital
gameplay in informal learning contexts (e.g., at
home), substantiating multimodal literacy learning
embedded in everyday gaming beyond the
classroom.
No.
Article
Level
Country
Setting
S1
[21]
Higher
education
the UK
Incorporated in school language course
S2
[22]
Primary
Spain
Workshops at school
S3
[23]
Secondary
the US
Incorporated in school language course
S4
[24]
Primary
Australia
Out of school
S5
[25]
Secondary
Iran
Out of school
S6
[26]
Secondary
the US
Incorporated in school language course
S7
[27]
Primary
the US
Workshops at school
S8
[28]
Secondary
the US
Workshops at school
S9
[29]
Higher
education
Taiwan, China
Incorporated in school language course
S10
[30]
Secondary
Australia
Incorporated in school language course
S11
[31]
Secondary
the US
Incorporated in school language course
S12
[32]
Secondary
the US
Incorporated in school language course
S13
[33]
Secondary
Spain
Workshops at school
S14
[34]
Secondary
Spain
Incorporated in school language course
S15
[35]
Primary
Norway
Incorporated in school language course
S16
[12]
Secondary
the US
Incorporated in school language course
S17
[36]
Secondary
the US
Incorporated in school language course
S18
[37]
Higher
education
Singapore
Out of school
S19
[38]
Higher
education
the US
Incorporated in school language course
S20
[39]
Secondary
the US
Incorporated in school language course
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
466
Volume 12, 2024
4.2 Game Features
The reviewed articles unanimously applied
commercial video games in their research project
(Figure 2), with sandbox construction games and
role-playing games dominating the chosen genres.
One explanation is that commercial games excel in
facilitating engaging, open-ended, and in-depth
multimodal learning experiences due to their
superior accessibility, richer game narratives,
enhanced interactivity, sophisticated game
mechanics, and higher popularity among students
when compared to educational games designed for
learning purposes, [20]. Another similarity is the
familiarity and age-appropriateness of the games for
the student participants in the research context. For
instance, many participants in past studies were
experienced in playing Minecraft before entering the
research project, [12], [28] and [30]. Some games
like Harry Potter and the Global Fire were adapted
from other media types (e.g., novels and films)
familiar to students [21], [22]. Researchers in [25],
[26] and [37] allowed students to choose their
favorite games. Ensuring students’ familiarity with
the game choice bridged their preferred daily media
practice with literacy learning, enabled them to
adapt to the game environment quickly, and
facilitated the connections between their previous
and new knowledge in multimodal meaning-making
[2]. Besides, the games were suitable for students’
age levels. For example, Super Smash Brothers and
Pokémon Go feature simple game mechanics
manageable for primary students. Similarly, making
artifacts with blocks in Minecraft is within the
regime of competency of primary and secondary
students. By contrast, games like World of Warcraft
and Second Life were more appropriate for
university students, given their sophistication for
fostering more nuanced multimodal interactions and
in-depth reflection.
Among diverse commercial games applied,
Minecraft, a globally known sandbox construction
game, dominated the game choices (n=8),
reinforcing the finding of [17]. Highlighting digital
games as interactive systems, researchers
accentuated that Minecraft offered a non-prescribed
open space and abundant semiotic resources
(building blocks of different textures) for agentive
exploration and creative multimodal designs [30].
Such features resonate with other selected games
such as LEGO World, The Sims, and Second Life. It
echoes “empowered learner” (i.e., good video
games should empower players to be co-designers),
a crucial learning principle of good video games,
further corresponding with the core concept of
“learning by design” highlighted in multimodal
literacy [7]. Another key feature is social
interactions enabled by the selected games.
Specifically, some selected games (e.g., Minecraft,
World of Warcraft, Second Life, and Harry Potter
and the Global Fire) offer an online platform for
multiplayer communication and collaboration in the
virtual world. Besides, there are large affinity
groups around the games for players to be
knowledge contributors, such as sharing expertise in
game forums and YouTube channels of Minecraft).
Fig. 2: Digital games in the reviewed studies
Emphasizing game narratives, [21], [27] and [36]
selected games with potentially biased
representation that address the critical aspect of
multimodal literacy. For instance, Grand Theft
Auto: San Andreas, L. A Noire, and Super Smash
Brothers are characterized by dominant male
characters and the underrepresentation of female
characters, offering resources for critical reflections
on gender stereotypes embedded in multimodal
texts.
4.3 Theoretical Perspectives
The reviewed articles were grounded in new literacy
and sociocultural theories, echoing [17].
Specifically, many studies (e.g., [12], [25], [31] and
[32]) introduced theories broadening the scope of
literacy in the digital era, such as multiliteracies
[41], digital media literacy [42], and video game
literacy [43]. Besides, [22], [30], [33] and [34]
referred to the theoretical perspectives highlighting
the sociocultural nature of literacy practice, such as
“participatory culture” [44] and “affinity group” [7].
Both theories emphasize meaning-making as a
constructive process in close social interactions
where students enact agency and negotiate self-
identities. The above theories justify the
researchers’ epistemological stance that recognizes
literacy learning as a multimodal, diversified,
constructive, and sociocultural practice, further
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
467
Volume 12, 2024
legitimizing incorporating digital games to cultivate
students’ multimodal literacy.
On the other hand, the selected articles revealed
limited specifications on the theoretical
underpinnings. Most studies offered concise
descriptions of relevant theories without a structured
conceptual framework as provided by [12] or [30].
Besides, although they described reconceptualized
literacy, it remains ambiguous regarding the criteria
for being multimodally literate in a certain age
group (e.g., what can be counted as effectively
utilizing digital multimedia for high school
students). Moreover, few studies elaborated on
pedagogical theories that guided the researcher’s
interventions. The lack of comprehensive
conceptual and pedagogical frameworks poses
challenges for non-expert practitioners to construct
instructional knowledge and effectively incorporate
game activities in practical educational settings, thus
limiting the scope and scale of this research field.
4.4 Methodological Perspectives
The selected articles unanimously adopted the
qualitative research paradigm, corroborating [17].
One explanation is that the constructive
philosophical stance of the qualitative paradigm
resonates with the contextualized nature of
multimodal literacy through DGBL, [45]. As shown
in Figure 3, case study (n=10) and ethnography
(n=6) were two main approaches (Figure 3-a). Four
studies did not specify the research approach.
Despite slight differences in the researcher’s
position and focus, one common goal of these
approaches was to gain a fine-grained understanding
of students’ constructive learning progress through
DGBL in a natural context, [45]. Besides, all the
research designs involved multiple data-collection
methods, such as observation, video recordings,
interviews, students’ artifacts, and field notes.
Various data sources offered multiple perspectives,
a dynamic and holistic picture of students’ learning
process, and enabled data triangulation to enhance
trustworthiness, [45].
Researchers designed different digital game-
based activities (Figure 3-b). Five studies focused
on students’ digital gameplay, including their in-
game actions, creations, and oral expressions. For
instance, [37] observed how the students critically
analyzed and applied multimodal information in the
gameplay and empathized with the game characters.
Additionally, 15 studies incorporated game paratext-
making activities to explore students’ multimodal
literacy through game paratext-making. For
instance, students engaged with game video-making
in [23], [28] and [34], social media posts in [22],
[30] and [33], digital game reviews or walkthroughs
in [27] and [35], and reflective writing regarding
gaming experiences and knowledge in [12], [21],
[31], [32], [38] and [39].
Fig. 3: Research design of the reviewed studies
One primary similarity of the research design
was the researchers’ emic view and active
participation in the research programs. For instance,
researchers in [24] and [36] directly interacted with
the students, asking probing questions during the
game activities or tasking students to “think it
aloud” in terms of the “why” and “how” of their
game actions [37]. In [21], [26] and [37],
researchers worked closely with teachers and
facilitated the whole process. Pedagogical tools
were also adopted, such as handouts containing
instructional knowledge [12] and guiding questions
[39]. The interventions set clear learning goals for
students’ navigation, enhanced students’ semiotic
awareness and reflective thinking, and allowed them
to articulate their ideas for rich findings.
Nevertheless, these studies revealed insufficient
details regarding instructional knowledge and the
pedagogical approach to building the intervention
model. It affects the transferability of the research
design in other contexts.
Concerning data analysis, the reviewed articles
reported varied approaches. It encompasses
multimodal discourse analysis [27], [35], inductive
analysis [28], [33], content analysis [25], template
analysis [12], [39], and comparison analysis [26].
Although termed differently, these methods shared
the same core of analyzing multiple data sources
inductively to interpret students’ learning. This is a
contextualized and constructive meaning-making
process, with theoretical frameworks being a broad
reference, [45]. Consequently, researchers relied on
their keen visions, profound theoretical knowledge,
and outstanding analytical ability to interpret and
synthesize a sheer volume of multimodal data. The
demanding workload and high competency required
in the analytical process constrain the sample size of
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
468
Volume 12, 2024
these studies. Besides, although such flexible
analysis methods yielded thorough findings, they
are prone to subjectivity due to vague criteria. For
instance, it may raise debate whether the student’s
surface description: “in the video game it’s you who
moves the characters, and in the film, you see them”
[22] can be claimed as demonstrating critical
thinking. The lack of criteria may lead to
inconsistent evaluations of students’ learning
outcomes and hinder the overall robustness of the
research findings, potentially undermining the
recognition of digital games as an effective way of
literacy learning.
4.5 Possibilities of Digital Games for
Multimodal Literacy
The reviewed articles substantiated that digital
games incorporated through different pedagogical
approaches offer possibilities for students’
multimodal literacy learning regarding critical
analysis of multimodal texts, creative design of
multimodal compositions, and active engagement in
social-cultural meaning-making (Table 2).
Most studies exemplified digital games can
develop students’ multimodal literacy in reading,
analyzing, and applying abundant multimodal
semiotic modes critically from multiple
perspectives. Representationally, [21], [27] and [36]
proved digital games as legitimate literature that
sparked students’ critical insights into stereotypical
depictions of gender and race in multimodal texts. It
further inspired interrogations of these issues in
their lifeworld, such as a girl reflecting that the
under-represented female characters mirrored the
minimization of herself in a boy-dominated group
[27]. Interpersonally, [12], [37], and [39]
demonstrated digital games as valuable interactive
texts, allowing students to form a critical awareness
to examine the affordances of multimodal semiotic
modes in enacting relationships with players (e.g.,
affecting players’ decision-making and problem-
solving). Compositionally, [26] and [33] evidenced
digital games as versatile resources for students to
approach from different angles (as narrative texts
and interactive experiences), thus forming a
comprehensive understanding of multi-layered
meaning-making systems in multimodal texts.
Creative design of multimodal composition took
place within and around the games. Studies applying
construction games (e.g., Minecraft and the Sims)
highlighted students’ agentive creations in the game
world. For instance, [23] and [24] emphasized that
Minecraft provided students with rich multimodal
resources and open space to assemble their artifacts
and realize their design ideas with minimum cost,
thus cultivating creativity. Such flexible game space
also enabled educators to combine traditional
literature with digital gameplay, connecting
students’ preferred media practice and school
literacy. For instance, students in [31] and [32]
recreated scenes from a classic novel in Minecraft,
strengthening their bond with the literature and
enhancing their ability for critical multimodal
analysis and creative composition.
Besides, digital games provide abundant design
resources for students to engage in interest-driven
and close-to-life transmedia multimodal
compositions [23], such as digital videos [28], game
reviews or walkthroughs [35], and social media
posts containing written comments and game videos
[30]. These activities extended students’ multimodal
literacy learning to a broader context, merging the
boundaries between literacy activities in and out of
the classroom and reshaping students’ perspectives
of what literacy means and how it relates to their
lives [31]. Also, they empowered students to
leverage multimodal semiotic resources, digital
tools (e.g., video-editing software), and social
spaces (e.g., social media platforms) to share
experiences as producers and knowledge
contributors [10].
Meanwhile, researchers underscored the social-
cultural nature of students’ critical and creative
engagement with digital games. The game-based
activities involved students in affinity space, in
which students applied multiple semiotic modes to
share game experiences with the real audience, such
as viewers of social media posts [30]. Students also
learned from more capable learners, including peers,
teachers, researchers, or other players. For instance,
expert gamers in [29] and [30] facilitated novice
ones during gameplay and co-constructed their
meaning-making system through constant
communication and collaboration. Immersed in this
sociocultural practice, students were empowered to
take multiple identities [28], such as avatars,
gamers, producers, knowledge distributors, students,
and teachers, through which they negotiated and
reconstructed their self-identity as active social
participants [7].
Overall, the reviewed studies contributed
positive learning outcomes proving that digital
games are “a multimodal literacy par excellence”
[7]. Nevertheless, one major limitation is the
insufficient specification and depth of the findings.
For instance, some studies briefly presented the
student’s self-reported enhancement in the ability to
process multimodal information in gaming but did
not offer concrete examples to support the claim
[25].
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
469
Volume 12, 2024
Table 2. Possibilities of digital games for multimodal literacy learning
Game
Pedagogical Approaches
Learning Outcomes
Evaluation Method
Minecraft
/LEGO World
• Gameplay (creation mode on
computers/consoles), creating artifacts with
assigned themes using blocks (S3, S4, S8,
S10, S11, S12)
• Design digital videos for storytelling using
game scenes (S3)
• Share game knowledge by videos
(S8)/social media posts (S10)
• Individual reflective writing of decision-
making based on multimodal information
(S16, S20)
• Creative and agentive design of
multimodal compositions in and
around the game for self-expression
(hereafter LO1)
• Critical analysis of multiple semiotic
modes for decision-making and
problem-solving (hereafter LO2)
• Active social interaction in an affinity
group and identity building (hereafter
LO3)
• Qualitative evaluation
of students’ digital
gameplay process
through field
observation or video
recordings without
specified criteria) (S2,
S4, S5, S9, S17, S18).
• Qualitative evaluation
of students’ in-game
artifacts (S3, S4, S8,
S9, S10, S11, S12) or
paratexts (S3, S7, S8,
S10, S13, S14, S15)
without specified
criteria.
• Qualitative analysis of
students’ reflective
accounts regarding
their learning
experiences and gains
without specified
criteria (all the
reviewed articles).
The Sims
• Create and manage a community in the
game on computers (S13, S14).
• Design digital videos for storytelling about
avatars (S14)
• Online discussions for game knowledge
sharing (S13)
• LO1
• LO2 (in video design)
• LO3
Pokémon Go
• Gameplay on tablets
• Create digital game reviews or
walkthroughs in groups (S15)
• LO1
• LO2 (in paratext design)
• LO3
World of
Warcraft
• Individual gameplay on computers
• In-depth game analysis and essay writing
(S19).
• Critical analysis of broader
implications of game representation in
academic writing
Harry Potter
and the
Global Fire
• Group gameplay
• Classroom discussion (compare the game
narratives with the novel (S2).
• LO3
• Critical analysis of games as
multimodal composition.
Super Smash
Brothers
• Group gameplay
• Design digital posters or drawings about
game character design (S7).
• Critical analysis of game
representation (stereotypes) in
multimodal texts (hereafter LO4)
L.A Noire
• Analyze game narratives (e.g., gender) and
write an essay (S1).
• Share game expertise with teachers
• LO3
• LO4 (e.g., gender)
Grand Theft
Auto: San
Andreas
• Observe the teacher’s gameplay.
• Classroom discussions regarding gender and
racial stereotypes with guiding questions
(S17).
• LO3
• LO4 (e.g., gender and race)
Second Life
• Group gameplay
• Recreate scenes of the traditional literature
in the virtual world (S9)
• LO1
• LO2
• LO3
Student-
selected
Games
• Think-it-aloud gameplay of The Walking
Dead (S18).
• Reflect on learning gains from previous
gaming experiences (S5)
• Game analysis and design (S6).
• LO2
• LO3
• Critical analysis of games as
multimodal composition and
multimodal practice
Additionally, the findings focused mainly on
students’ multimodal literacy embedded in game-
based activities, whereas scarce evidence showcased
measurable improvements in their multimodal
literacy levels and specific criteria of the outcome
evaluation. This deficiency raises concerns about
the significance of incorporating digital games in
fostering students’ multimodal literacies in school
settings and calls into question the justification for
the substantial investments in digital devices and
allocating curriculum time for these activities.
5 Conclusion, Implications, and
Recommendations
In conclusion, this study systematically reviewed 20
articles exploring digital games for multimodal
literacy learning across education levels, focusing
on the contextual features, game features, theoretical
perspectives, methodologies, and learning
outcomes. The findings offered empirical evidence
that digital games have been applied across primary,
secondary, and higher education, but limited to
small-scale studies and narrow-scope locations. The
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
470
Volume 12, 2024
researchers employed diverse commercial digital
games that offered students agency for free
exploration and creation, rich narratives for critical
interrogation, and affinity groups for social
interactions and identity reconstruction. The
selected articles referred to various theoretical
perspectives grounded in the core principles of
multiliteracies and socio-cultural theories. However,
they generally lack systematic organization and
specificity concerning conceptual and pedagogical
frameworks. Methodologically, the qualitative
research paradigm dominated the research design, in
which researchers took an emic view, employed
multiple data-collection methods, enacted active
interventions, and conducted inductive analysis to
yield rich research findings. The synthesis of the
research findings unveiled the possibilities of digital
games for multimodal literacy in terms of critical
analysis of multimodal information, creative design
of multimodal composition in and around the game,
and active engagement in social-cultural meaning-
making empowering knowledge construction,
collaboration, and identity reconstruction.
This review has implications for future research
to venture deeper into digital game-based
multimodal literacy studies. Given the contextual
features discussed in section 4.1, researchers can
pioneer similar research programs involving larger
groups of student participants from less researched
social contexts where digital games also play a
significant part in youth’s lives (e.g., Asian
contexts). The broadened scope can address
concerns about the Western-centric focus of prior
studies, contributing to a more globally inclusive
understanding of digital games’ impact on
multimodal literacy education and highlighting its
socio-cultural diversity [2].
Second, this review offers a variety of digital
games suitable for educators across different
educational levels to enhance students’ multimodal
literacy. This addresses educators’ uncertainty about
game selection [13]. Besides, this study identified
main game features pertinent to multimodal literacy,
which can facilitate researchers and teachers to
relate digital games to their teaching objectives and
explore more digital games within their specific
contexts, thus extending the educational value of
digital gameplay beyond limited options.
Third, this review provides an overview of
theoretical and methodological perspectives of the
extant literature. The findings can guide researchers
and educators to make informed decisions,
effectively selecting appropriate theoretical
frameworks and research methods to explore digital
game-based multimodal literacy. Furthermore, this
study illuminates a promising avenue for researchers
to improve the clarity and systematization of
existing theoretical frameworks, aligning with more
specific pedagogical theories. This alignment can
substantially facilitate educators and researchers to
structure instructional knowledge, adopt systematic
pedagogical approaches, and evaluate students’
learning outcomes. Consequently, it can bolster the
practical applicability and effectiveness of
cultivating students’ multimodal literacy through
digital games, as well as enhance the trustworthiness
of studies in this study field.
Last, students’ learning outcomes synthesized in
this study offer different perspectives for future
research to explore the possibilities of digital games
in multimodal literacy learning (e.g., critical
analysis and creative production of multimodal
texts). However, since the findings were based on
textual data from small-scale qualitative inquiries,
the effectiveness of digital games on multimodal
literacy has not been statistically examined,
unveiling a new terrain for further investigations.
Researchers can develop quantitative instruments
and adopt pre and post-tests to assess the effects of
different digital games on each dimension of
multimodal literacy learning to substantiate these
possibilities. Such approaches also empower larger-
scale studies to enhance the rigor and
generalizability of the findings.
While offering synthesized existing knowledge
and valuable insights, this study acknowledged the
potential omission of relevant studies despite our
rigorous search methodology. Some relevant articles
were not included in this review due to the
inaccessibility of full-length articles, affecting the
comprehensiveness of this review article. Besides,
as a qualitative systematic review, this study
contains a certain level of subjectivity in
interpreting the findings. Despite these limitations,
we believe this systematic review serves as a new
benchmark for future research and educational
practices in game studies and literacy studies to
keep updates and refinements, thus better fulfilling
students’ needs and social demands in the ever-
changing media landscape.
References:
[1] Jewitt, C. & Kress, G. R. Multimodal literacy.
New York: Peter Lang. 2003.
[2] Cope, B. & Kalantzis, M. Making Sense:
Reference, Agency, and Structure in a
Grammar of Multimodal Meaning. Cambridge
University Press. 2022.
https://doi.org/10.1017/9781316459645.
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
471
Volume 12, 2024
[3] van Leeuwen, T. Multimodal literacy.
Metaphor, 2017(4), 2017, pp.17-23, [Online].
https://search.informit.org/doi/10.3316/aeipt.2
19392 (Accessed Data: April 5, 2024).
[4] Kress, G., & van Leeuwen, T., Reading
images: The grammar of visual design (2nd
ed.), London: Routledge, 2006.
[5] Selwyn, N. The digital native–myth and
reality. Aslib Proceedings, 61(4), 2009, pp.
364-379.
https://doi.org/10.1108/00012530910973776.
[6] Lim, F. V., Toh, W., & Nguyen, T. T. H.
Multimodality in the English language
classroom: A systematic review of literature.
Linguistics and Education, 69(2022), 2022,
101048, pp.1-30.
https://doi.org/10.1016/j.linged.2022.101048
[7] Gee, J. P. What video games have to teach us
about learning and literacy. New York:
Palgrave Macmillan, 2007.
https://doi.org/10.1145/950566.950595.
[8] Toh, W., & Lim, F. V. Using video games for
learning: Developing a metalanguage for
digital play. Games and Culture, 16(5), 2021,
pp.583-610.
https://doi.org/10.1177/1555412020921339.
[9] Beavis, C. Multimodal Literacy, Digital
Games and Curriculum. In: Lowrie, T.,
Jorgensen (Zevenbergen), R. (eds) Digital
Games and Mathematics Learning.
Mathematics Education in the Digital Era, 4.
Springer, Dordrecht. 2015.
https://doi.org/10.1007/978-94-017-9517-3_7.
[10] Apperley, T., & Walsh, C. What digital games
and literacy have in common: a heuristic for
understanding pupils' gaming literacy.
Literacy, 46(3), 2012, pp.115-122.
https://doi.org/10.1111/j.1741-
4369.2012.00668.x.
[11] Consalvo, M. Cheating: Gaining advantage in
video games. Cambridge, MA: MIT Press,
2007.
https://doi.org/10.7551/mitpress/1802.001.000
1.
[12] Stufft, C. J., & von Gillern, S. Fostering
multimodal analyses of video games:
reflective writing in the middle school.
Journal of Adolescent & Adult Literacy, 65(3),
2021, pp.245-255.
https://doi.org/10.1002/jaal.1198.
[13] Dashtestani, R. The Winding Path towards
Implementing Digital Game-based Learning
(DGBL) in an Educational Context: the
Voices of Pre-service Teachers. Computer
Assisted Language Learning, 23(3), 2022,
pp.70-93.
[14] Hung, H. T., Yang, J. C., Hwang, G. J., Chu,
H. C., & Wang, C. C. A scoping review of
research on digital game-based language
learning. Computers & Education, 126(2018),
2018, pp.89-104.
https://doi.org/10.1016/j.compedu.2018.07.00
1.
[15] Zou, D., Huang, Y., & Xie, H. Digital game-
based vocabulary learning: where are we and
where are we going? Computer Assisted
Language Learning, 34(5-6), 2021, pp.751-
777.
https://doi.org/10.1080/09588221.2019.16407
45.
[16] Hung, H. T., Yang, J. C., & Tsai, Y. C. Student
Game Design as a Literacy Practice. Journal
of Educational Technology & Society, 23(1),
2020, pp.50-63, [Online].
https://www.jstor.org/stable/10.2307/2691540
6 (Accessed Data: March 10, 2024).
[17] Hanghøj, T., Kabel, K., & Jensen, S. H.
Digital games, literacy and language learning
in L1 and L2: A comparative review. L1-
Educational Studies in Language and
Literature, 22(2), 2022, pp.1-44.
https://doi.org/10.21248/l1esll.2022.22.2.363.
[18] Page, M. J., McKenzie, J. E., Bossuyt, P. M.,
Boutron, I., Hoffmann, T. C., Mulrow, C. D.,
& Moher, D. The PRISMA 2020 statement: an
updated guideline for reporting systematic
reviews. BMJ,372(n71), 2021, pp.1-9.
https://doi.org/10.1136/bmj.n71.
[19] Finfgeld-Connett, D. Use of content analysis
to conduct knowledge-building and theory-
generating qualitative systematic reviews.
Qualitative research, 14(3), 2014, pp.341-352.
https://doi.org/10.1177/1468794113481790.
[20] Van Eck, R. Digital game-based learning: It's
not just the digital natives who are restless.
EDUCAUSE Review, 41(2), 2006, pp.1-16.
[21] Berger, R., & McDougall, J. Reading
videogames as (authorless) literature. Literacy,
47(3), 2013, pp.142-149.
https://doi.org/10.1111/lit.12004.
[22] Checa-Romero, M. Developing skills in
digital contexts: Video games and films as
learning tools at primary school. Games and
Culture, 11(5), 2016, pp.463-488.
https://doi.org/10.1177/1555412015569248.
[23] Cipollone, M., Schifter, C. C., & Moffat, R. A.
Minecraft as a creative tool: A case study.
International Journal of Game-Based
Learning, 4(2), 2014, pp.1-14.
[24] Dezuanni, M. Minecraft and children’s digital
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
472
Volume 12, 2024
making: Implications for media literacy
education. Learning, media and technology,
43(3), 2018, pp.236-249.
https://doi.org/10.1080/17439884.2018.14726
07.
[25] Ebadi, S., & Ahmadi, R. A native video
gamer's journey toward multi-literacy
development: A narrative inquiry. Journal of
Language, Identity & Education, 2022, pp.1-
14.
https://doi.org/10.1080/15348458.2022.20294
50.
[26] Gerber, H. R., Abrams, S. S., Onwuegbuzie, A.
J., & Benge, C. L. From Mario to FIFA: what
qualitative case study research suggests about
games-based learning in a US classroom.
Educational Media International, 51(1), 2014,
pp.16-34.
https://doi.org/10.1080/09523987.2014.88940
2.
[27] LaMear, R., & von Gillern, S. Super Smash
Sisters: Critical Literacy, Gender, and Video
Games in the Elementary Classroom.
Changing English, 30(4), 2023, pp.359-372.
https://doi.org/10.1080/1358684X.2023.22379
48.
[28] Lewis Ellison, T. Digital participation, agency,
and choice: An African American youth's
digital storytelling about Minecraft. Journal of
Adolescent & Adult Literacy, 61(1), 2017,
pp.25-35. https://doi.org/10.1002/jaal.645.
[29] Liang, M. Y. Pragmatic socialization through
gameplay directives: Multimodal conversation
analysis of avatar-embodied interactions.
Journal of Pragmatics, 171(2021), 2021,
pp.36-48.
https://doi.org/10.1016/j.pragma.2020.09.028.
[30] Marcon, N., & Faulkner, J. Exploring
Minecraft as a pedagogy to motivate girls'
literacy practices in the secondary English
classroom. English in Australia, 51(1), 2016,
pp.63-69.
[31] Marlatt, R. Literary analysis using Minecraft:
An Asian American youth crafts her literacy
identity. Journal of Adolescent & Adult
Literacy, 2018, 62(1), pp.55-66.
https://doi.org/10.1002/jaal.747.
[32] Marlatt, R. Get in the game: Promoting justice
through a digitized literature study.
Multicultural Perspectives, 20(4), 2018,
pp.222-228.
https://doi.org/10.1080/15210960.2018.14677
69.
[33] Mndez, L., Garca-Perna, M., & Corts, S.
Virtual communities in a secondary school–
Discovering the internal grammar of video
games. Journal of New Approaches in
Educational Research, 3(1), 2014, pp. 2-10.
https://doi.org/10.7821/naer.3.1.2-10.
[34] Monjelat, N., Checa, M., Varela, A. B. G., Del
Castillo, H., & Herrero, D. Using the Sims 3
for narrative construction in secondary
education: A multimedia experience in
language classes. In New pedagogical
approaches in game enhanced learning:
Curriculum integration (pp. 180-213). IGI
Global, 2013. http://doi.org/10.4018/978-1-
4666-3950-8.ch010.
[35] Strømman, E. Crossover literacies: A study of
seventh graders’ multimodal representations
in texts about Pokémon Go. Computers and
composition, 59(2021), 2021, 102629, pp.1-16.
https://doi.org/10.1016/j.compcom.2021.1026
29.
[36] Tanner, S. J. Reading Grand Theft Auto:
Improvisational Urban Literacy. Urban
Education, 58(8), 2020, pp. 1827-1852.
https://doi.org/10.1177/0042085920923026.
[37] Toh, W., & Lim, F. V. Learning in digital play:
a dual case study of video gamers’
independent play. Research and Practice in
Technology Enhanced Learning, 17(1), 2022,
pp.1-17. https://doi.org/10.1186/s41039-022-
00182-2.
[38] Vance, B. Video games and multimodality in
first-year composition. CEA Critic, 79(1),
2017, pp.120-134.
https://doi.org/10.1353/cea.2017.0008
[39] von Gillern, S., & Stufft, C. Multimodality,
learning and decision-making: children's
metacognitive reflections on their engagement
with video games as interactive texts. Literacy,
57(1), 2023, pp. 3-16.
https://doi.org/10.1111/lit.12304.
[40] Steinberg, L., & Morris, A. S. Adolescent
development. Annual review of psychology,
52(1), 2001, pp.83-110.
https://doi.org/10.1146/annurev.psych.52.1.83.
[41] New London Group. A pedagogy of
multiliteracies: Designing social futures.
Harvard educational review, 66(1), 1996,
pp.60-93.
https://doi.org/10.17763/haer.66.1.17370n67v
22j160u.
[42] Dezuanni, M. “The Building Blocks of Digital
Media Literacy: Socio-material Participation
and the Production of Media Knowledge.”
Journal of Curriculum Studies, 47(3), 2015,
pp.416–419.
https://doi.org/10.1080/00220272.2014.96615
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
473
Volume 12, 2024
2.
[43] Apperley T., & Beavis, C. A model for critical
games literacy. E-learning and Digital Media,
10(1), 2013, pp.1-12.
https://doi.org/10.2304/elea.2013.10.1.1.
[44] Jenkins, H. Convergence culture: Where old
and new media collide. New York, NY: New
York University Press, 2006.
[45] Merriam, S.B., & Tisdell, E.J. Qualitative
Research: A guide to design and
implementation (4th ed.). San Francisco, CA:
Jossey-Bass. 2016.
Contribution of Individual Authors to the
Creation of a Scientific Article (Ghostwriting
Policy)
- Yanan Shen conducted the literature review,
analyzed the selected articles, and drafted the
manuscript.
- Habibah Ab Jalil formulated the research
questions, checked each procedure, and revised
the manuscript.
- Rahimah Jamaluddin contributed to the cross-
check and finalized the manuscript.
Sources of Funding for Research Presented in a
Scientific Article or Scientific Article Itself
No funding was received for conducting this study.
Conflict of Interest
The authors have no conflicts of interest to declare.
Creative Commons Attribution License 4.0
(Attribution 4.0 International, CC BY 4.0)
This article is published under the terms of the
Creative Commons Attribution License 4.0
https://creativecommons.org/licenses/by/4.0/deed.en
_US
WSEAS TRANSACTIONS on COMPUTER RESEARCH
DOI: 10.37394/232018.2024.12.46
Yanan Shen, Habibah Ab Jalil, Rahimah Jamaluddin
E-ISSN: 2415-1521
474
Volume 12, 2024