Introduction

The evolution of technology in the 21st. Century has had a massive impact on the vast education policies across many countries. The changes are rather prominent, especially on the aspect of assessment where it has been the major deciding factor on the ranking of our education system worldwide. In the present visually-oriented world, exposure of students to multimodal platform is inevitable, which includes blogs, short messaging system (SMS), and social media platforms (e.g., Facebook, Twitter, YouTube, and Skype) (Rajendra, 2015). Hence, turning a blind eye towards the advancement taking place around us is unwise. Many changes have taken place in the international education policies with the emergence of new literacy practices manifested by the New London Group on multiliteracies. In fact, many leading countries in the circle of education, such as Australia, Finland, Sweden, the UK, and Singapore, have implemented multimodal in their curriculum policy.

Although numerous countries have executed multimodal teaching, a proper guideline is absent regarding the assessment of multimodal reading. The concern of teaching multimodal reading has been highlighted only in the tertiary level, wherein it should have been emphasised in the primary and secondary level itself (Perry, 2020). Despite the substantial number of studies on multimodal reading assessment, a comprehensive and systematic literature review (SLR) of the related studies appears to be in scarcity. As such, this study had attempted to narrow down the gap identified in understanding, identifying, and characterising the criteria of multimodal reading assessment in ESL classroom.

Fundamentally, this study bridges a significant gap within the literature, as most SLRs have assessed the various types of multimodal tools applied in the learning of English. This study is vital due to lack of studies that have focused on the status of multimodal reading assessment in ESL classroom (Yusof, 2017). It is noteworthy to highlight that most studies have failed to list the details of their review procedures (e.g., databases involved, exclusion and inclusion of articles, and search terms). As a consequence, it is challenging for future scholars to replicate the study, validate the interpretation, and assess the comprehensiveness. Moreover, only limited studies concerning multimodal assessment that have called for urgent review on this matter, to provide a clear overview on the aspects of multimodal assessment that are beneficial for studies purposes (Pillai, 2010). Many educators are not clear on how to conduct a reading assessment using multimodal text. Therefore, details derived from the literature may shed light on comprehending its emphasis and significance. Besides, other countries use a range of multimodal aspects to evaluate reading assessment, thus demanding globalised specification, along with solid theoretical and empirical evidence, to ensure usage of a synchronised multimodal framework across all countries.

Towards a Systematic Review Framework of Worldwide Multimodal Framework

The SLR involves studying the research question for a stated problem by employing systematic and explicit methods. The approach determines, selects, and critically analyses the relevant studies to gather and assess data from the works used in the review. Since this method demands critical analysis, statistical approach was excluded. The SLR is important because it will become the basis to justify a researcher’s claim about the gaps in the existing studies and to prescribe suggestions for future researches.

Many studies have looked into multimodal assessment, but scarcity was noted in SLR of these studies (Periasamy, 2015). As such, this present study bridges the existing gap in understanding and applying the multimodal assessment in ESL classroom. The peer reviewed studies were selected for this study to get a close review on the aspects that should be included in reading multimodal assessment in ESL classroom, and how the approach can improve the existing assessment methods. This SLR is vital due to the absence of a benchmark review on multimodality in assessing ESL reading within the literature.

The main aspect used to facilitate this present SLR is through the research question on how multimodal reading and viewing comprehension assessment is defined in various articles in ESL classes. The main emphasis of this analysis is reading and viewing assessment rubric (Pillai, 2012).

This SLR includes the analysis of multimodal reading assessment to identify the common aspects used in the teaching and assessment of multimodal reading. The subsequent section describes the methodology and PRISMA statement. The following section lists the review and synthesis of the literature sources to select and critically review studies concerning multimodal reading assessment rubric.

Methodology

This section discusses the methods used in this study. One method refers to PRISMA, which identified the resources used to perform the SLR, as well as the inclusion and exclusion criteria that determined the eligibility of the studies. This was followed by reviewing and analysing data.

Prisma

This SLR was performed in accordance to PRISMA due to several advantages offered by this method, including: defining research questions that are compatible with the systematic research, differentiating inclusion from exclusion criteria, and enabling the analysis of a huge database retrieved from the scientific literature within a certain duration. The PRISMA offers flexibility to perform high-scale research on multimodal reading assessment rubric, apart from identifying its impact on teachers and students. This method is useful to monitor the different assessment rubrics used by other researchers across the globe.

Resources

Web of Science (WoS) was the ultimate database used for this study. The database stores massive volumes of journals and hundreds of research areas. The WoS was selected as it is a well-known robust database with vast quality journals in the area of education, namely assessment, linguistics, and second language studies. The database contains over 100 years of systematic structure of journals, which were reflected by the complete back file and updated reference list developed by Clarivate Analytics. The second database used to review articles was Scopus. This database is rich in peer-reviewed articles from various publishers globally. Similar to WoS, Scopus is composed of journals from multi-disciplinary studies.

Systematic Literature Review Process

Identification

The SLR process involved four stages. The first stage began in November 2019. In the first stage (identification stage), suitable key words were listed to be used in the database research. Reference to past studies was made to gather key words that could lead to multimodal assessment rubric. Table 1 lists the key words used in each database. This stage was useful because it covered the overall studies of journal articles related to the topic to perform analytical comparison. Duplicate studies were discarded to yield more precise outcome.

Table 1 Key Words and Article Search Summary
Database	List of key words
Scopus	Assessment, multimodal, reading, pre-school primary school, secondary school, high school, tertiary, ESL
Web of Science

Screening

In the screening process, the exclusion criteria were employed to choose only the related articles. The initial criterion was the type of literature reviewed. Publications that consisted of empirical data were selected, while the rest were excluded including review-based articles, books series, proceedings from conference, and book chapters. Second, the screening was done in-depth by excluding non-English journals. This is to prevent from the hassle and confusion that might occur due to translation. Another criterion used was year of publication. The articles were systematically traced as far as 20 years back, which was from 2000 to 2019. The reason to screen the two-decade timeline was to gather adequate amount of journals, so as to identify the evolution of multimodal reading assessment. Next, only articles indexed in the area of multimodal assessment were selected. The final criterion was the focus on reading assessment among ESL studies. Therefore, only articles that had focused on multimodal reading assessment rubric were selected. Table 2 summarises the inclusion and exclusion criteria used for this study.

Table 2 Inclusion-Exclusion Criteria Summary
Criteria	Eligibility	Exclusion
Literature Type	Journal (research articles)	Journals (SLR), book series, book, chapter in book, conference proceeding
Language	English	Non-English
Timeline	All years in the database	None
Indexes	Social sciences.	Science Citation Indexed

Data Abstraction and Analysis

In-depth data abstraction was performed by reading the abstracts in detail to identify the most suitable themes and sub-themes related to the title. A qualitative analysis was conducted based on content analysis to extract themes related to multimodal reading assessment. The authors synchronised the themes and sub-themes via thematic analysis (Table 3).

Table 3 Thematic Analysis
Author	Title	Population age	Themes	Sub-themes
1. Wyatt-Smith and Kimber [7] (PRIMARY)	1.Valuing and evaluating student-generated online multimodal what counts texts: rethinking	13 – 15	1.Production	1.e-proficiency (technical operations, and discriminating use) 2.Cohesion (unifying structure, representation, and organisation of ideas and links) 3.Content (working with existing knowledge to create new knowledge) 4.Design (creating an aesthetic and artful design)
2. Chitra and Weninger [8] (PRE-SCHOOL)	2.Intertextuality in preschoolers’ engagement with popular culture: implications for literacy development	5 – 6		1.Creation of new multimodal text
3. Vassilikopoulouet al. [9] (SECONDARY)	3.Pilot use of digital educational comics in language teaching	12-13		2.Collaborative production of a multimodal narrative with a simple multimodal
				3.Experiencing 4.Conceptualising 5.Analysing 6.Applying
4. Sarjit et al. [10] (SECONDARY)	4. Designing Learning Elements Using the Multiliteracies	16
Chitra and Weninger [8]	Intertextuality in pre-schoolers’ engagement with popular culture: implications for literacy development	7-8	2.Cultural	7.Engagement with popular cultural text
Vassilikopoulouet al. [9]	Pilot use of digital educational comics in language teaching	22-24		8.Functional use of language
5.Andersson and Sofkova-Hashemi [11] (PRIMARY)	5.Screen-based literacy practices in Swedish primary schools	12		9.Background experience 10.Learners’ socio-cultural setting
6. Ghahari and Ahmadinejad [12] (TERTIARY) 7.Pantaleo [13] (PRIMARY) Vassilikopoulouet al. [9]	6. Operationalisation of Bachman’s model via a multimodal reading comprehension test: Screening test method facets and testees’ characteristics.	18-22	3.Typography	11.Word size used assessed in all multiple choice, short answer, and essay versions.
	7.Matters of Design and Visual Literacy: One Middle Years Student's Multimodal Artifact	4		12.Images’ typography (caption size, letters, and symbols)
				13.Dialog and captions 14.Functional use of language
	Pilot use of digital educational comics in language teaching	22-24
8 Cope and Kalantzis [14] (OVERALL)	8 ‘Multiliteracies’: New Literacies, New Learning	11	4.Visual	15.Colours 16.Perspective 17.Vectors 18.Foregrounding 19.Backgrounding
9.Chan and Unsworth [15] (PRIMARY)	9. Image–language interaction in online reading environments: challenges for students’ reading comprehension.	7-11		20.Engagement with image-text relation
Vassilikopoulouet al. [9]	Pilot use of digital educational comics in language teaching	22-24		21. Effective connection between picture and text.
10.Yamada-Rice [16] (PRE-SCHOOL)	10. Beyond words: An enquiry into children’s home visual communication.	18-20		22.Photographs Texts
11. Sabbah et al. [17] Masip-Alvarez et al. [18] (PRIMARY)	11.Effects of graphic novels on reading comprehension in Malaysian year 5 students	17-19		23.Graphic visual
12.Unsworth [19] (PRIMARY & SECONDARY)	12.Multimodal reading comprehension: Curriculum expectation and large-scale literacy testing practices	13-15		24.Identification of visual representation 25.Image language 26.Understand analytical images 27.Build inferred meaning 28.Understand combination of media in text creation
13.Honeyford [20] (PRIMARY)	13.From Aquí and Allá: Symbolic Convergence in the Multimodal Literacy Practices of Adolescent Immigrant Students	16-20		29.Readier grammar of visual text
14.Cook [21] (SECONDARY)	14.Teaching Multimodal Literacy Through Reading and Writing Graphic Novels	11
15.Bowen [22] (TERTIARY)	15.Assessing visual literacy: a case study of developing a rubric for identifying and applying criteria to undergraduate student learning	10-15		30.Acknowledge meaning from image 31Sequence and narrate story from image 32.Create images read
16.McGrail and Behizadeh [23] (PRIMARY)	16.K-12 multimodal assessment and interactive audiences: An exploratory analysis of existing frameworks	7-10		33.Image
				34.Engagement with image-text relation distribution, augmentation, and divergence
Chandler [24] (PRIMARY)	To what extent are teachers well prepared to teach multimodal authoring?	7-11
17.Jimenez and Meyer [25] (PRIMARY)	17.First Impressions Matter: Navigating Graphic Novels Utilising Linguistic, Visual, and Spatial Resources.	7-11		35.Synthesis of colour 36.Accuracy of predicted stories 37.Decode story plot
18.Chandler [24]	18.To what extent are teachers well prepared to teach multimodal authoring?	17-18		38.Special visual effects 39.Visual rhetoric 40.Image 41.Explosions, fades, and animations 42.Availability of on-screen text, as well as visual and audio elements.
19.Exley and Cottrell [26] (PRIMARY)	19.Reading in the Australian Curriculum English: Describing the Effects of Structure and Organisation on Multimodal Texts	7-11	5.Gestural	43.Gestural
20.Guo and Feng [27] (PRIMARY & SECONDARY)	20.Infusing multiliteracies into English language curriculum: The visual construction of knowledge in English textbook from an ontogenetic perspective	7-15		44.Social activities 45.Verbal languages 46.Action Speech
21.Ferro et al. [28] (PRE-SCHOOL)	21.ReadLet: Reading for Understanding	4		47.Finger sliding
22. Cope and Kalantzis [14]	22.‘Multiliteracies’:New Literacies, New Learning	17-18		48.Behaviour 49.Bodily physicality 50.Gesture 51.Sensuality 52.Feeling & Effect 53.Kinesics 54.Proxemics
Vassilikopoulouet al. [9]	Pilot use of digital educational comics in language teaching.	22-24	6.Spatial	55.Structure of the plot
Exley and Cottrell [26]	Reading in the Australian Curriculum English: Describing the Effects of the Structure and Organisation on Multimodal Texts.	7-11		56.Space of the action
				57.Synthesis of spatial
Jimenez and Meyer [25]	First Impressions Matter: Navigating Graphic Novels Utilising Linguistic, Visual, and Spatial Resources	13-15		58.Time used
Chandler [24]	22. To what extent are teachers well prepared to teach multimodal authoring?	13-15		59.Setting and location
Cope and Kalantzis [14], Kress and Van Leeuwen [29]	‘Multiliteracies’: New Literacies, New Learning.			60.Ecosystem meaning 61.Geographic meaning 62.Architectonic meaning
Jimenez and Meyer [25]	First Impressions Matter: Navigating Graphic Novels Utilising Linguistic, Visual, and Spatial Resources.	13-15	7.Linguistics	63.Accuracy of predicted story 64.Grammar 65.Linguistics
23.O’Byrne and Murrell [30] (SECONDARY)	23.Evaluating multimodal literacies in student blogs	13-15		66.Semantic 67. Syntactic awareness
Exley and Cottrell [26]	Reading in the Australian Curriculum English: Describing the Effects of Structure and Organisation on Multimodal Texts	7-11		Linguistics
Cope and Kalantzis [14]	‘Multiliteracies’: New Literacies, New Learning	13-15		68.Delivery 69.Vocabulary and metaphor 70.Modality 71.Transivity 72.Normalisation of processes 73.Information structure 74.Local coherence relations 75.Global coherence relations
24. Baldwin [31] (SECONDARY)	Multimodal Assessment in Action: What We Really Value in New Media Texts	13-15		76.Proximity/ closeness 77.Coherence between reading and other semiotic resources 78.Analogies and metaphors 79.Deconstruction of figurative language
25. Magal-Royo et al. [32] (TERTIARY)	Multimodal interaction on English testing academic assessment.	19-22		80.Reading Comprehension 81.Composition 82.Grammar
				84.Rhetoric and composition
Pantaleo [13] Baldwin [31]	Matters of Design and Visual Literacy: One Middle Years Student’s Multimodal Artifact Student’s Multimodal Artifact	13-15	8.Aesthetic	85.Meaning, line, and emotion represented by colours 86.Discuss points of view of authors, artist, and illustrator 87. Purpose assessments explicitly and/or implicitly value achieving and maintaining a clear communicative purpose/focus 88.Coherence
27. Lee [33] (TERTIARY)	27.An examination of ESL Taiwanese University students’ multimodal reading responses	19-22	9.Haptic	89.Game
McGrail and Behizadeh [23]	K-12 multimodal assessment and interactive audiences: An exploratory analysis of existing frameworks			90.Movement
McGrail and Behizadeh [23]	K-12 multimodal assessment and interactive audiences: An exploratory analysis of existing frameworks	16-18	10.Animation	91.Sound 92.Image Movement 93.Collaboration with audience
Chan and Unsworth [15]	Image–language interaction in online reading environments: challenges for students’ reading comprehension			94.Augmentation 95.Distribution 96.Divergence
				97. Relocation of characters through animation 98. Narrative process or dynamically describe a change in social interaction. 99. Gesture – the on the spot animation of characters 100. Convey emotion, narrative process or interpersonal relationship 101.Animation and positioning of the head and eyes to direct gazes 102.Convey interaction or emotion between characters 103.Relocation of characters through animation 104.Gesture—“on the spot” animation of characters 105.Animation and positioning of the head and eyes to direct gaze
Chandler [24]	To what extent are teachers well prepared to teach multimodal authoring?	13-15
28. Puteh-Behak and Ismail [34] (TERTIARY)	29.Multiliteracies Project Approach: Dated or a Worthy Learning Tool	18-22	11.Teamwork
Chandler [24]	To what extent are teachers well prepared to teach multimodal authoring?	13-15	12.Atmosphere and mood	106.Colour 107.Texturing worlds 108.Lighting effects 109.Fog effects
			13.Characters	110.Selection of objects, 111.Colouring 112.Texturing of objects
			14. Availability of an in-world camera	113.Sizes of camera shot 114.Camera distance 115.Vertical camera angles 116.Horizontal camera angles 117.Point of view 118.Camera movement
		13-15	15.Audio	119.Voice performance 120.Sound effects 121.Animation of objects 122. Pitch
Cope and Kalantzis [14]	‘Multiliteracies’: New Literacies, New Learning			123.Music
Chandler [24]		13-15	16.Sequencing of scenes and shots	124.Continuity 125.Sequence compositional meaning 126.Thematic orientation and sequencing 127.Expression from the resource
Baldwin [31]	Multimodal assessment in action: What we really value in new media texts	13-15	17.Habits of mind	128. Metacognitive assessment of the writer 129.Process based
			18.Agency	130.Assessing value, as well as promoting risk-taking and creative level that can develop writers’ agency 131.Audience
			19.Cognitive psychology	132. Multimodal assessment approaches with processes/criteria associated with cognitive psychology.
			20.Conceptual processes	133.Assessments criteria and processes developed through the lens of cognitive psychology that leads to rhetorically aware texts.
			21.Mediated action	134.Assessments criteria and processes developed through the lens of Wertsch’s mediated action theory as the basis of assessment
			22.Deliberate practice	135.Assessments criteria and processes developed through the lens of Ericsson’s “deliberate practice”
			23.Standard based	136.Multimodal assessment approaches with processes/criteria aligned with standards-based writing assessments, such as the NCTE’s Framework for 21st Century Curriculum
			24.K-12 policy documents	137.Assessments explicitly value processes and criteria put forth in policy documents by influential organisations, such as NCTE
			25.Prosumer/ maker	138.Assessments explicitly value criteria and processes based in the educational movement that argues students should be makers and creators, not just consumers. Different from design that explicitly evokes this movement
			26.New media	139.Multimodal assessment approaches with processes or criteria associated with theories of new media.
			27.Graphic design	140.Assessments explicitly and/or implicitly value criteria typical of the field of graphic design, such as CRAP.
			28.Technological skill	141.Assessments explicitly and/or implicitly value writers’ technological sophistication and effective use of specific technologies, such as Photoshop, iMovie, and Dreamweaver
			29.Materiality	142.Materiality refers to the “stuffiness” of a text, or the awareness of the inseparability of form and content on the part of writer and readers - how each construct meaning. Materiality, at its core, is the understanding of the mutually transformative nature of form and content as shaped by the specific social, historical, and personal contexts in which a text operates.
			30.Literary study	143.Multimodal assessment approaches with processes/criteria associated with literary studies, such as in the work of Sorapure and Ball (“Designerly”), specifically metaphor, metonym, and close-reading
			32.Interaction between textual parts	148.Proximity/closeness, as well as coherence between writing and other semiotic resources
			33.Values	149.Explicit Discussion of several aspects, such as right/wrong, us/the others, female/male, etc.

Results

Based on the thorough review performed in this study, a total of 33 themes were generated. They included production, cultural, typography, visual, gestural, spatial, visual text, linguistics, aesthetic, haptic, animation, team work, atmosphere and mood, characters, availability of in-world camera, audio, sequencing of scenes and shots, habits of mind, agency, cognitive psychology, conceptual processes, mediated action, deliberate practice, standards based, K-12 policy documents, consumer/ maker, new media, graphic design, technological skill, materiality, literary study, general structure and setting, interaction between textual parts, as well as values and multiliteracies of multimodal.

In total, 28 studies were selected for this SLR, in which 33 themes were extracted from the analysed studies. The studies involved four pre-schools, fourteen primary schools, nine secondary schools, and six tertiary-level institutions. Some articles were a combination of two or more levels.

Multimodal Reading Assessment in ESL Classroom

The review presented in this segment is composed of multimodal reading assessment in ESL classroom across the world in accordance to each identified theme.

Production

A total of five studies identified the theme of production. Wyatt-Smith and Kimber (Wyatt-Smith, 2005) asserted that the production theme generated four sub-themes, namely e-proficiency, cohesion, content, and design. Meanwhile, Chitra & Weninger, (2010) depicted a sub-theme of creation of new multimodal text in the production theme. Vassilikopoulou, et al., (2011) revealed that multimodal reading was assessed based on collaborative production of a multimodal narrative. Sarjit, et al., (2015) introduced experiencing, conceptualising, analysing, and applying for the theme of production.

Cultural

The theme cultural was identified by Chitra & Weninger, (2010); Vassilikopoulou, et al., (2011); Andersson & Sofkova-Hashemi, (2016). This theme has four sub-themes; engagement with popular cultural text, functional use of language, background experience, and learners’ socio-cultural setting, which were considered in the assessment rubric.

Typography

The theme of typography was used in the studies of Vassilikopoulou, et al., (2011); Ghahari & Ahmadinejad, (2016); Pantaleo (2013) in their reading multimodal reading assessment. In the findings reported by Pantaleo (2013), the sub-theme focused on typography of the images such as caption size, letters and symbols. While typography in the study by Ghahari & Ahmadinejad, (2016) emphasise on the sub-criteria of communication. Students’ proficiency and word size used in assessing all the multiple choice, short answer and essay versions.

Visual

The theme visual is another criterion used to assess reading of multimodal text. Overall, 12 studies had identified visual as a theme. They are as follows: Vassilikopoulou, et al., (2011); Cope & Kalantzis, (2009); Chan & Unsworth, (2011); Yamada- Rice (2010); Sabbah, et al., (2013); Masip-Alvarez, et al., (2013); Cook (2017); Bowen (2017); McGrail & Behizadeh (2016); Chandler (2017); Jimenez & Meyer, (2016). These studies have generated 30 sub-themes: colours, perspective, vectors, foregrounding, backgrounding, engagement with image-text relation, photographic texts, graphic visual, identification of visual representation, image language, understand analytical images, build inferred meaning, understand combination of media in text creation, readier grammar of visual text, acknowledge meaning from image, sequence and narrate story from image, create images read, engagement with image-text relation distribution, augmentation, divergence, synthesis of colour, accuracy of predicted stories, decode story plot, special visual effects, visual rhetoric, explosions, audio and visual elements, fades and animations, as well as on-screen text availability.

Gestural

Exley & Cottrell, (2012) identified the main theme and sub-theme of gestural, while Ferro, et al., (2018) proposed a sub-theme of finger sliding. Meanwhile, Guo & Feng, (2015) found two sub-themes of gestural, which are verbal languages and action speech. Cope & Kalantzis, (2009) identified sub-themes of proxemics, behaviour, feeling and effect, bodily physicality, kinesics, sensuality, and gesture.

Spatial

In total, five studies have highlighted the theme of spatial. Vassilikopoulou, et al., (2011); Jimenez & Meyer, (2016) noted that assessment of multimodal reading should consider synthesis of spatial and time, while Chandler (2017) emphasised on setting and location. The spatial theme was categorised into three sub-themes by Cope & Kalantzis, (2009), namely ecosystem meaning, geographic meaning, and architectonic meaning. Meanwhile, Exley & Cottrell, (2012) used the main theme as a sub-theme.

Linguistics

The theme of linguistics was detected in studies by Cope & Kalantzis, (2009); Jimenez & Meyer, (2016); Exley & Cottrell, (2012); O’Byrne & Murrell, (2014); Baldwin (2016); Magal-Royo, et al., (2012). Its sub-themes are accuracy of predicted story, grammar, linguistics, semantic and syntactic awareness, delivery, vocabulary and metaphor, modality, transivity, process normalisation, structure of information, local and global coherence, correlations of coherence with proximity and congruence between writing and other semiotic resources, analogies, rhetoric, figurative language, reading comprehension, and composition.

Aesthetic

The importance of aesthetic element in multimodal reading assessment was highlighted by Pantaleo (2013); Baldwin (2016). They added that learners should understand, appreciate, interpret, and compose both texts and meaning; have emotion represented by colours; identify the points of view presented by author, artist, and illustrator; determine the explicit and implicit values of assessment; as well as attain and maintain clear communicative coherence and purpose.

Haptic

McGrail & Behizadeh, (2016); Lee (2013) had outlined haptic as a main theme in multimodal reading assessment, along with two sub-themes; movement and game.

Animation

The theme of animation in multimodal assessment, as depicted by Chan & Unsworth (2011); McGrail & Behizadeh, (2016); Chandler (2017) has a number of sub-themes, including sound and image movement, collaboration with audience, augmentation, distribution, divergence, character relocation in animation, description of change in social exchange, gestures and emotions of/between animation characters, positions of the head and eyes to direct gazes, and interpersonal relationship.

Teamwork

This theme was proposed by Puteh-Behak & Ismail, (2018) and its sub-themes are critical thinking, technological competence, and teamwork.

Setting and location

Chandler (2017) described the symbolic meaning in multimodal, such as locality, different times of the day, some seasons, and era.

Atmosphere and mood

Chandler (2017) proposed the theme of atmosphere and mood, along with the sub-themes of colour/texturing worlds, lighting effects, and fog effects.

Characters

According to Chandler (2017), this theme in multimodal reading assessment requires symbolic meanings, hence generating the following sub-themes: colour, texture, and selection of object.

Availability of in-world camera

As identified by Chandler (2017), the assessment of multimodal reading should consider the social distance between in-world characters, as well as the distance between the character and the viewer. Its sub-themes comprise of sizes of camera shot, camera distance, horizontal and vertical camera angles, as well as camera point of view and movement. The sub-theme also includes the extent of involvement and the power relations between character and viewer.

• Audio: According to Chandler (2017) and the New London Group (1996), the sub-themes of audio are voice performance, sound and music effects, as well as animation of objects.

• Sequencing of scenes and shots: This theme involves the sub-themes of continuity sequence, compositional meaning, thematic orientation, sequencing, and expression from the resource Chandler (2017).

Habits of Mind

Baldwin (2016) indicated that habits of mind have sub-themes of metacognitive assessment of the writer and process based. These sub-themes reflect the attitude of mind towards the objects created and seen.

• Agency: Baldwin (2016) had generated the theme of agency with the following sub-theme to assess value: promote risk-taking and creative level that can develop writers’ agency. The sub-theme illustrates the process of narrative or dynamically explains the change that happens during social exchange.

• Cognitive psychology: Baldwin (2016) linked the cognitive psychology theme to criteria/processes within the cognitive psychology aspect, as noted in those depicted.

• Conceptual processes: This theme was identified by Baldwin (2016) based on cognitive psychology, which yields rhetorical text awareness.

• Deliberate practice: According to Baldwin (2016), multimodal assessment criteria/processes are established through the perspective of Ericsson’s deliberate practice.

Standards-based

This theme discovered by Baldwin (2016) shows that multimodal assessment methodologies, which are integrated with processes/ criteria, are in alignment with standardised national curriculum.

K-12 Policy Documents

According to Baldwin (2016), assessment of multimodal reading values criteria/processes that are stipulated by influential organisations in policy documents.

Prosumer/ Maker

Criteria/Processes in assessments are explicitly valued based on educational movement, which claims that instead of being consumer, students should be the creator and maker Baldwin (2016). This view varies from designing as it explicitly evokes movement.

• New media: According to Baldwin (2016), multimodal assessment methods that come with criteria/processes are related to the theories of new media.

• Graphic design: Assessments can both implicitly and explicitly place value on the criteria associated with the field of graphic design.

• Technological skill: According to Baldwin (2016), assessments can explicitly or implicitly place value on writers’ technology based on sophistication and efficient usage of certain technology. Some instances are Photoshop, iMovie, and Dreamweaver.

• Materiality: The theme revealed by Baldwin (2016) showed that materiality is related to the stiffness of the text or the awareness of the inseparability between content and forms in light of reader and writer. This type of awareness includes the construction of meaning. This theme explains that multimodal reading assessment acknowledges the mutually transformative aspect of both content and form, which can be outlined based on personal, social, and historical contexts that the texts operate in.

• Literary studies: According to Baldwin (2016), the multimodal reading assessment methods of criteria/processes are related to literary work.

• General structure and setting: The theme of general structure and setting was proposed by Danielsson & Selander, (2016). It consists of thematic orientation and sequencing. The arrangement of the content structure is given emphasis in this theme.

• Interaction between textual parts: Theme explains the proximity and the connection of reading with interaction and textual parts. The theme also integrates connection between textual parts and semiotic resources.

• Values: As depicted by Danielsson & Selander, (2016), values are given emphasis by explicitly discussing elements between right and wrong.

• Multimodality/Multiliteracy: Multimodal theme was proposed by New London Group (1996). Cope & Kalantzis, (2009) developed five main modes, namely Visual, Linguistics, Spatial, Gestural, and Audio.

Discussion

A systematic analysis was performed in this study using the available literature on multimodal reading assessment in ESL classroom. Reading assessment is a vital aspect in ESL classes, wherein correct assessment rubric must be adapted to capture the students’ performance in the most accurate way. A complete review based on the sources retrieved from two established databases yielded 28 articles that had discussed on themes, which can be included in multimodal reading assessment rubrics. The findings showed that reading assessment has shifted to multimodality, in which once it used to be just unimodal. Based on the scope of this SLR, 33 themes and 149 subthemes were identified. The main assessment rubric criteria were production, cultural, typography, visual, gestural spatial, linguistics, haptic, aesthetic, animation ,teamwork, atmosphere/mood, characters, availability of an in-world camera, audio, sequencing scenes and shots, habits of mind, agency, cognitive psychology, conceptual processes, mediated action, deliberated action, standard based, K-12 policy documents, prosumer/maker, new media, graphic design, technological skill, materiality, literary study, general structure/setting, interaction between textual parts, values, and multiliteracies/multimodality.

Multimodal reading assessment showed that students need not necessarily read a text as in the conventional reading assessment, wherein the students are assessed in various comprehensive elements in the former method. Students can be assessed based on their understanding of the visual presented or their ability to tell a story using images. Exploration of comprehensive elements can enhance their reading ability. Such assessment may increase the accuracy of reading assessment, and therefore, the importance of designing a rubric as a guideline to capture such performance.

Furthermore, through SLR, many themes related to multimodal reading assessment were discovered in this study, whereby 33 main themes and 149 subthemes were generated.

Future Direction

There are remaining gaps in the way Malaysian students are assessed based on multimodal reading, such as the type of multimodal material and the assessment criteria that demand further attention. As noted from the SLR, almost no article was within the context of Malaysia, but instead more concentrated on the abroad. As such, more studies should focus on the classroom setting in Malaysia, which differs from that in other countries where the studies had been based.

More emphasis on materiality, such as artefact (Pahl & Rowsell, 2011), movement (Leander, 2009); venue (Ruitenberg, 2005), enriches text comprehension in as an in-depth manner. This is helpful, especially in assisting students with reading difficulty, as multimodal reading comprehension can be delivered in various modes of communication. Reading may appear more appealing as it involves graphics, movement, audio, and many other multimodal features (James, 2014). The research outcome is significant for students in terms of increasing their attention span during the learning process. (Bester & Brand, 2013). Since multimodal assessment is composed of several modes, such as visual and audio instead of text alone, students become more stimulated to pay attention to the reading task. (Romero et al., 2018). In multimodal reading, students read materials that are supported by images and sounds. Hence, the students read while listening, read while viewing or read while listening and viewing. (Antona & Stephanidis, 2013). The multiple inputs in a single reading activity can attract the students’ attention towards the reading task. (Gibbons, 2012).

Methodology

The SLR was executed by adopting the conceptual approach.

Resources: All databases referred in search of the articles were described and justified.

Inclusion and exclusion criteria: A set of selection criteria that is employed to detect relevant articles during the review process.

a. Identification – Identify the list of key words used to search the articles.

b. Screening – Describes the exclusion of articles that fail to fulfil the inclusion criteria.

c. Eligibility – The selected articles are subjected to complete review, while articles that fail to satisfy the inclusion criteria are excluded.

Analysis and Presentation of Results

Description of analysis methods

Although the digital searching method is efficient in selecting articles, there are other complementary techniques, such as citation tracking. The advantage of this tracking method is that it reveals results of forward and backward times. Besides, this method enriches search results since additional publication can be retrieved. Only using the digital method is subjected to vocabulary restriction. (Wright et al., 2014). Another method that can be used is reference searching, which screens the list of references in the articles. Examining the reference list can bridge information gap.

Conclusion

The SLR revealed the significance of implementing multimodal reading assessment for Malaysian ESL students. In the context of multimodal reading, several themes (e.g., image language, visual, production, typography, animation, and audio) were used by scholars to incorporate variation in the reading assessment. The review, hence, prescribes some improvements for future endeavour, mainly because incorporation of multiple research techniques yields more search outcomes that can enrich the systematic process.

Acknowledgements

I would like to take this opportunity to thank my supervisors and fellow lecturers at University Kebangsaan Malaysia for the continuous guidance in completing the research. It had been a great journey working under your guidance and support. Your willingness to share knowledge by reading and providing adequate feedback had been a great motivation to successfully complete the article. With the cooperation from each one of you, the work achieved the desired objectives. Heartfelt gratitude is extended for the project funding which allows us to explore the design of using multimodal reading assessment rubric for ESL students. With the financial support showered on us, we were able to analyse in-depth dimensions of multimodal reading assessment rubric which will be very suitable to be used for assessing ESL students in Malaysia.

References

Rajendra, T.R. (2015). Multimodality in Malaysian schools: The case for the graphic novel, MOJES: Malaysian Online Journal of Educational Sciences, 3(2), 11-20.

Google scholar

Perry, M.S. (2020). Multimodal engagement through a transmedia storytelling project for undergraduate students, Journal of Language Studies, 20(3), 19-41.

Crossref , Google scholar

Yusof, S.M., Lazim, Z.M., & Salehuddin, K. (2017). Teacher trainees’ perspectives of teaching graphic novels to ESL primary schoolers. The Southeast Asian Journal of English Language Studies, 23(3), 81-96.

Crossref , Google scholar

Pillai, S. (2010). Popular pedagogy: Multimodal environments for the teaching and learning of literature in the Malaysian tertiary world. Asiatic: IIUM Journal of English Language and Literature, 4(2), 81-91.

Google scholar

Periasamy, M.S., Gruba, P., & Subramaniam, G. (2015). A multimodal literary analysis of a television commercial, 3L: Language, Linguistics, Literature, 21(3), 151-164.

Google scholar

Pillai, S., Wei, A.W.W., & Gidah, M.E. (2012). Towards the global postgraduate: Dynamic teaching-learning environments for optimum transfer of knowledge, Procedia-Social and Behavioural Sciences, 59, 248-253.

Crossref , Google scholar

Wyatt-Smith, C., & Kimber, K. (2005). Valuing and evaluating student-generated online multimodal texts: Rethinking what counts. Research Journal of the National Association for the Teaching of English, 39(2), 22–43.

Crossref , Google scholar

Chitra, S., & Weninger, C. (2010). Intertextuality in preschoolers’ engagement with popular culture: Implications for literacy development. Language and Education, 24(5), 431–447.

Crossref , Google scholar

Vassilikopoulou, M., Retails, S., Nezi, M., & Boloudakis, M. (2011). Pilot use of digital educational comics in language teaching. Educational Media International, 48(2), 115–126.

Crossref , Google scholar

Sarjit, K., Malini, G., & Gurnam, K.S. (2015). Designing learning elements using the multiliteracies approach in an ESL writing classroom. The Southeast Asian Journal of English Language Studies, 18(3), 119-134.

Google scholar

Andersson, P., & Sofkova-Hashemi, S. (2016). Screen-based literacy practices in Swedish primary schools. Nordic Journal of Digital Literacy, 11(2), 86-103.

Crossref , Google scholar

Ghahari, S., & Ahmadinejad, S. (2016). Operationalization of Bachman’s Model via a multimodal reading comprehension test: Screening test method facets and testees’ characteristics. Studies in Educational Evaluation, 51, 67-76.

Crossref , Google scholar

Pantaleo, S. (2013). Matters of design and visual literacy: One middle year’s student’s multimodal artifact. Journal of Research in Childhood Education, 27, 351-376.

Crossref , Google scholar

Cope, B., & Kalantzis, M. (2009). "Multiliteracies": New literacies, new learning. Pedagogies: An International Journal, 4(3), 164-195.

Crossref

Chan, E.L. (2011). Unsworth. Image–language interaction in online reading environments: Challenges for students’ reading comprehension. The Australian Educational Researcher, 38(2), 181-204.

Crossref , Google scholar

Yamada-Rice, C. (2010). Beyond words: An inquiry into children’s home visual communication practices. Journal of Early Childhood Literacy, 10(3), 341–363.

Crossref , Google scholar

Sabbah, M., Masood, M., & Iranmanesh, M. (2013). Effects of graphic novels on reading comprehension in Malaysian year 5 students. Journal of Graphic Novels and Comics, 4(1), 146-160.

Crossref , Google scholar

Masip-Alvarez, A., Hervada-Sala, C., Pamies-Gomez, T., Arias-Pujol, A., Jaen-Fernandez, C., …& Nejjari-Akhi-Elarab, F. (2013). Self-video recording for the integration and assessment of generic competencies, IEEE Global Engineering Education Conference, Technische Universität Berlin.

Crossref , Google scholar

Unsworth, L. (2014). Multimodal reading comprehension: Curriculum expectation and large-scale literacy testing practices, Pedagogies: An International Journal, 9(1), 26–44.

Crossref , Google scholar

Honeyford, M.A. (2014). From aquí and allá: Symbolic convergence in the multimodal literacy practices of adolescent immigrant students, Journal of Literacy Research, 46(2), 194-233.

Crossref , Google scholar

Cook, M.P., & Kirchoff, J.S. (2017). Teaching multimodal literacy through reading and writing graphic novels, Language and Literacy, 19(4), 76-95.

Crossref , Google scholar

Bowen, T. (2017). Assessing visual literacy: A case study of developing a rubric for identifying and applying criteria to undergraduate student learning, Teaching in Higher Education, 22(6), 705- 719.

Crossref , Google scholar

McGrail, E., & Behizadeh, N. (2016). K-12 multimodal assessment and interactive audiences: An exploratory analysis of existing frameworks, Assessing Writing, 31, 24-38.

Crossref , Google scholar

Chandler, P.D. (2017). To what extent are teachers well prepared to teach multimodal authoring? Cogent Education, 4(1), 1–19.

Google scholar

Jimenez, L.M., & Meyer, C.K. (2016). First impressions matter: Navigating graphic novels utilizing linguistic, visual, and spatial resources. Journal of Literacy Research, 2(17), 1-25.

Crossref , Google scholar

Exley, B., & Cottrell, A. (2012). Reading in the Australian curriculum English: Describing the effects of structure and organization on multimodal texts. English in Australia, 47(2), 91–98.

Google scholar

Guo, N.S., & Feng, D. (2015). Infusing multiliteracies into English language curriculum: The visual construction of knowledge in English textbooks from an ontogenetic perspective. Linguistics and Education, 31, 115-129.

Google scholar , Indexed at

Ferro, M., Cappa, C., Giulivi, S., Marzi, C., Nahli, O., Cardillo, F.A., & Pirrelli, V. (2018). ReadLet: Reading for Understanding, In 2018 IEEE 5th International Congress on Information Science and Technology (CiSt), Marrakesh, Morocco, 1-6.

Crossref , Google scholar

Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication, Arnold Publishers, London.

O’Byrne, A., & Murrell, S. (2014). Evaluating multimodal literacies in student blogs, British Journal of Educational Technology, 45(5), 926-940.

Crossref , Google scholar

Baldwin, K.M. (2016). Multimodal assessment in action: What we really value in new media texts, doctoral dissertation, Department of English, University of Massachusetts.

Crossref , Google scholar

Magal-Royo, T., Gimenez-López, J.L., & Laborda, J.G. (2012). Multimodal interaction on English testing academic assessment. Procedia-Social and Behavioral Sciences, 46, 5824-5827.

Crossref , Google scholar

Lee, H.C. (2013). An examination of ESL Taiwanese university students' multimodal reading responses, Literacy Research and Instruction, 52(3), 192-203.

Crossref , Google scholar

Puteh-Behak, F., & Ismail, I.R. (2018). Multiliteracies project approach: Dated or a worthy learning tool? Journal of Language Studies, 18(2), 313-333.

Crossref , Google scholar

The New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard educational review, 66(1), 60-93.

Crossref , Google scholar

Danielsson, S. (2016). Reading multimodal texts for learning: A model for cultivating literacy. Designs for Learning, 8(1), 25-36.

Crossref , Google scholar

Pahl, R. (2011). Artifactual critical literacy: A new perspective for literacy education, Berkeley Review of Education, 2(2), 129-151.

Crossref , Google scholar

Leander. Multimodality and mobile culture, hlm. Handbook, Routledge, New York, 2009.

Ruitenberg. (2005). Deconstructing the experience of the local: Toward a radical pedagogy if place, Philosophy of Education, 30(12), 212-220.

James. (2014). The internet and the Google age: Prospects and perils, Research publication, Ireland.

Bester, Brand. The Effect of Metacognitive Intervention on Learner Metacognition and Achievement in Mathematics, PhD Thesis, School of Mathematics, University of the Free State Bloemfontein, 2013.

Crossref , Google scholar

Romero. (2018). Teaching Literature and Language Through Multimodal Texts, IGI Global, New York.

Crossref , Google scholar

Antona. (2013). Stephanidis. Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments, Springer, New York.

Gibbons. (2012). An Architectural Approach to Instructional Design, Routledge, New York.

Crossref , Google scholar

Wright (2014). The wider context of performance analysis and its application in the football coaching process, International Journal of Performance Analysis, 14(3), 709-733.

Crossref , Google scholar

Received: 09-Dec-2021, Manuscript No. JMIDS-21-7745; Editor assigned: 11-Dec-2021, PreQC No. JMIDS-21-7745(PQ); Reviewed: 22-Dec-2021, QC No. JMIDS-21-7745; Revised: 02-Jan-2022, Manuscript No. JMIDS-21-7745(R); Published: 09-Jan-2022