Keywords

1 Introduction

Studies of the out-of-school literacy practices in K-12 classrooms learners report that digital technologies can elicit changes in how emergent bilingual learners interpret and create modes, authorship, genres, and time and space in texts (Gee and Hayes 2011; Lam and Warriner 2012; Stewart 2014; Yi 2007); and research has also shown that these kind of digital textual practices support L2 language development, allowing for use of expanded semiotic resources as well as a wide variety of rhetorical goals and audiences (Gee and Hayes 2011). Compared to the abundant literature on language learners’ out-of-school literacy practices, little is known about their uses of social media in school. This lack of focus is closely associated with the traditional privileging of school-based literacies like writing over others that involve image, sound, color, and video for communication and higher-level mental development (Shanahan 2013; Smagorinsky 1995). However, the recent and ever-growing dominance of digital technologies as communication and representation media in our everyday lives along with the expansion of multimodal communication have spurred teachers to incorporate digital technologies into their curriculum and instruction. More teachers have started to expand their views of literacy by supporting students in developing their ability to make meanings with multimodal resources available in digital technologies.

Studies have shown that teachers, even from very early elementary grades, incorporate digital literacy practices in their curriculum while fostering their school-based academic language and literacy development (Atkinson and Swaggerty 2011; Gebhard et al. 2011; Shanahan 2013; Shin 2014; Toohey et al. 2015). For example, Gebhard et al. (2011) show how a second grade teacher in a U.S. urban elementary school incorporated blogs into her writing curriculum and instruction of English language arts (ELA) to support young children’s understanding of how to negotiate diverse social and political goals in learning school-based academic genres. Their study demonstrates that the varied purposes and audiences available in a new medium provided an expanded semiotic potential for young bilingual learners’ writing. Similarly, Shanahan’s study (2013) explores how a fifth grade teacher’s multimodal composing instruction supported or not her students’ conceptual understanding of acid rain. Shanahan found that the teacher’s lack of knowledge and experience with multimodal writing prevented the students from strategically appropriating affordances of multimedia-based writing for composing multimodal texts. Considering these varied findings, studies are needed that investigate digital literacy practices in school settings, to better inform teachers of possibilities and challenges of multimodal writing that use digital technologies.

To contribute to the literature on in-school use of social media, this current study investigates how and if a sixth grade teacher’s use of the online multimedia platforms Edmodo and Glogster supported students in learning to construct multimodal argumentative essays. Specifically, it examines how new multimodal writing supported or not multilingual learners’ academic literacy development and critical language awareness for various language use.

2 Writing as Design

The current research is based on a social semiotics perspective to multimodal writing that considers communication as meaning making with two or more semiotic systems (e.g., linguistic, audio, visual, spatial) (Bezemer and Kress 2008; Jewitt and Kress 2003; Kress 2003, 2005, 2010; Kress and van Leeuwen 2001; van Leeuwen 2003, 2005), and meaning-making resources are known as semiotic modes and communication channels as mediums. Mode can be defined as a meaning-making resource that includes processes such as writing, sound, images, layout, and videos while medium is any technology that carries modal resources for communication. From this perspective, writing is conceptualized as a designing resource to make meaning with different semiotic modes for one’s communicative goal. In the current technological era, students communicate through a variety of media; from emailing and texting to social networking tools (e.g., blog, Instagram, Facebook, Twitter, wiki, YouTube). Each medium has different semiotic resources and the modes are used distinctively in fulfilling social cultural practices. For example, font size, paragraph indentation, and alignment are crucial modes in the medium of email, while sound and action in video recordings are crucial in the medium of YouTube. These media are embedded in particular socially defined contexts of communication, reflecting the norms of the social groups in which they are used.

Considering the social norms that are attached to use of media, the affordances of multimodal composition represent not only an author’s representational intention but also an author’s perception of configured audiences, in the contexts of a cultural practice of writing. In addition, all employed modes interact with one another to create a culturally specific meaning that is not available from these discrete resources in isolation. A newly co-created meaning is grounded in the synesthetic semiosis of multimodal authoring processes (Kress 2003; Kress and Van Leeuwen 2006). That is, semiotic modes do not create meanings as separate autonomous meaning-making resources nor are they employed disjointedly. For academic writing, writers need to master “not only the role played by the mode of representation as a design element but also the effects of both the absence and the existence of design elements on readers’ responses to the multimodal text” (Shin and Cimasko 2008, p. 378). Such an understanding involves an awareness of how intermodal relationships construct meaning as a multimodal ensemble, rather than just linking two different modes (see Liu and O’Halloran 2009, p. 369 for meanings across modes).

3 Metafunctions: Linguistic, Visual, and Aural

Semiotic resources are different yet interconnected sign systems for making meaning. Developing meta semiotic awareness of sign use involves building an understanding of the complex interrelationships of sign systems, meanings, and context. Inter-semiotic meanings and the functions across semiotic modes will change according to contexts of culture and situation.

Informed by SFL theory of register variables and context (Halliday 1994; see Harman, this volume; Martin 1984), Kress and van Leeuwen conceptualized similar semiotic metafunctions for analysis of the visual mode and its communicational grammar: representational, interactive, and compositional (1996). The representational function deals with how visual resources construct ideas in communication. The interactive function relates to how interpersonal and evaluative meanings are constructed and how the visual resources give, demand or exchange information or services (see Eggins 2004, pp. 183–184 for interactional roles in language use). The visual resources, which may resemble images from the natural/real world, use high degrees of modality related to credibility. Compositional function explains how a text is organized in its synthesis of different modes, and its meaning-making involves the layout, placement, and relative salience of the pictures and text. For learners in multimodal composing, they need to become aware that configuration of resources in multimodal designing (e.g. images, color and text in a commercial) are constrained by the cultural expectations of configured audiences and genres. In other words, agentive use of modes needs to be conciliated with awareness of contextual purposes.

In defining how relationships among modes (e.g. image and text) create ideational meaning, Unsworth (2006) introduces three types of relations across modes —concurrence, complementarity, and connection. Concurrence across modes explains how one mode specifies or describes the meaning of the other without adding any new information. It takes the forms of explication, exposition, equivalence, or homospatiality (see Daly and Unsworth 2011, pp. 61–63 for examples of concurrence relations between images and language). Complementarity is a term used to explain how a new element is added by either text or image in relation to augmentation, distribution, and divergence. Finally, connection explains how quoting or reporting speech and thoughts is inscribed within the intermodal links, and how conjunctive relations of time, place, and cause are conveyed.

To support understanding of the functions across and within semiotic modes, learners need to become aware of how their configuration functions to realize field, tenor and mode. In terms of the interpersonal meaning, realistic images may demand a response from the viewer through the gaze of a represented participant or may implicitly demand a response because of the lack of eye contact of the represented participant (Kress and van Leeuwen 1996). The images evoke feelings, co-articulate attitude with verbiage (Martin 2002), and convey the modality of truth or credibility through use of real images representing the natural world. For compositional meanings, visual images tend to construe the ideal and real structure by arranging abstract, general information at the top and concrete, specific information at the bottom. In case of the electronic media, the size and position of a text block on the screen construct a spatial relationship between image and text to emphasize different aspects of the image on the screen (Jewitt 2002). As such, as the visual mode in multimodal composition, images construe their modal meanings and intermodal meanings in interconnected relations with other modes.

Although research has often focused on multimodal visual and verbal texts, sound as a semiotic resource in a multimodal text can be an important component. It can be understood as a schema with four-part elements—vocal delivery, music, special effects, and silence (see McKee 2006, p. 337 for constituents of sound schema as semiotic mode). Speech as a semiotic resource in multimodal composition adds different interpersonal meanings to the content and style of a speech depending on its vocal delivery. The elements for vocal delivery concern nonverbal resources such as tension, roughness, breathiness, loudness, pitch, tone, and vibrato. Similar to the subtle elements of the appraisal system in SFL (e.g. graduation, force and focus in Martin and Rose 2003), the vibrations, tones and density of the sounds are called upon to function as important semiotic resources for forming culturally oriented meanings of voices (van Leeuwen 1999). Another key element in the analytical framework of sound as semiotic mode that is relevant to the current study is music. The interpretation of music is made within several planes: sensuous (e.g., voice, tone, loudness), expressive (e.g., evoked feelings), and musical (e.g., rhythm, melody) (McKee 2006). These categories overlap, simultaneously operating in making meanings of the music.

In sum, multimodal writing is a co-articulation of different modes for communicative goals. This paper explores a focal child’s multimodal designing process and developing metalanguage awareness; on his understandings of semiotic and intersemiotic functions in designing multiple modes.

4 Methodology

4.1 Context

This study was conducted in a sixth grade classroom of the Liberty Elementary School,Footnote 1 located in a rural area in the northeast of the United States. The school serves students from the third to the sixth grade. A significant portion of the students were from economically challenging backgrounds, as indicated by the fact that 27% of students of the Liberty School qualified for free and reduced price lunch during the school year when the current study was conducted. The school made their annual yearly progress goals in English Language Arts, Mathematics and Science for all of their students for three years prior to the study.

The Liberty School’s English Language Arts (ELA) instruction followed the state-mandated curriculum framework that directly aligns with the Common Core State Standards (National Governors Association Center for Best Practices, Council of Chief State School Officers 2010). Its curriculum maintained writing workshops drawing on Lucy Calkins’ (2010) writing workshop model for intermediate grades and the 6 + 1 Writing Traits.Footnote 2 The school also provided a literacy club with reading specialists before and after regular instruction to promote students’ literacy skills as one of its extracurricular activities. Many teachers held extra support sessions for striving students to improve academic literacies in content areas at least two or three times a week. In terms of computer technologies, the school had a computer lab with 35 computers that all of the classrooms could sign up and use for their instructional activities throughout the academic year.

This study took place in a sixth grade inclusive classroom (i.e. integration of ESOL, special needs and mainstream students) that had eighteen students ranging from eleven to twelve years old. The class was composed of nine girls and nine boys. Four of the students were classified as students with disabilities and had Individual Education Plan (IEPs) and one student was a bilingual learner. The daily routine of the sixth grade classroom started with individual morning work including literacy and mathematics activities, followed by regular daily classes for a range of content areas such as mathematics, ELA, science and social studies. The ELA block in which I conducted this study lasted for two hours and fifteen minutes, and consisted of vocabulary, spelling, read aloud, guided reading, independent reading, and writing lessons. The classroom had a large class library that had ample books organized by subject. The class had six computers and six IPads that they used in many learning activities across content areas throughout the day. This classroom was equipped with a SmartBoard where the classroom teacher delivered the majority of lessons and an Elmo, a document camera that the teacher used to display papers and student work.

4.2 Participants

The classroom teacher, Julia Hunt, was a second year teacher working on her Master’s degree at a college nearby. She was interested in instructional technologies and, particularly, using Web 2.0 technologies in literacy activities for her students. Ms. Hunt took courses with me, a teacher educator in her Master’s program. In one of these courses, she conducted a project that incorporated Web 2.0 technologies into the ELA curriculum, and invited me to her classroom so that I could provide support for curriculum research she was conducting.

The focal child for this study was Sonny, an eleven-year old Laotian bilingual boy whose family immigrated into the states when he was a toddler. He spoke Laotian with family members at home, and learned and used English as a school language. Sonny was a social and active student who enjoyed playing football and computers games. He was academically successful and achieved advanced proficiencies in mathematics. He loved engaging in digital literacy practices, having access to up-to-date computer devices and Internet in and out of school. For instance, he often typed his grandfather’s stories for him and played digital games with his family and friends. Sonny enjoyed any school project where he could work on the computer and use Internet resources. In addition to being a bilingual English learner who needed improvement in ELA (i.e., writing) compared to his advanced achievement in mathematics, his interest in digital literacies led me to select Sonny as a focal student for the study.

4.3 Curriculum Unit

The study was based on a curricular unit of argumentative writing for English language arts curriculum, as mandated by the Common Core State Standards (National Governors Association Center for Best Practices, Council of Chief State School Officers 2010). According to the CCSS writing standards, sixth graders are expected to “[w]rite arguments to support claims with clear reasons and relevant evidence” (p. nd). Argument writers initiate an argumentative text with a thesis followed by background information concerning the debated issue, and evidence to support or disprove the thesis. Writers conclude the text with restatement of the thesis. In legal, academic and formal social contexts, the register of argumentation tends to use a higher density of lexico-grammatical resources that express values (e.g., good, bad), judgments (e.g., hasty, slow), comparisons (e.g., similar, comparable), and contrasts (e.g., different, disparate); the passive voice with logical conjunctions and nominalizations may also be used to seem more objective (Martin and Rose 2008; Schleppegrell 2004). To teach the genre, Ms. Hunt designed a curricular unit on writing an argumentative letter to the president in which students selected their own topics about changing America into a better country; studied related information, and wrote their argument with supporting claims and evidence.

Ms. Hunt turned the argumentative curricular unit into a multimodal writing project through use of multimedia authoring tools and online resources. Through use of these media, she aimed to validate students’ out-of-school literacy practices and to support their development of semiotic competence of various meaning-making resources. The class had been using various technologies including Web 2.0 technologies (e.g., Edmodo, Glogster) in reading and writing activities across content areas. For instance, right before the current study project, students created brochures about Greece in social studies classes using Microsoft Publisher, and PowerPoint slides on Planets in science classes. In Ms. Hunt’s multimodal argumentative writing unit, the teacher used Edmodo, a web-based platform that supports students in connecting, sharing ideas, and collaborative learning; and Glogster, an online platform that provides multimedia resources for digital composition and interactive learning to enable students to both write and publish texts. The teacher provided students with mini-lessons on how to sign up to key aspects of Edmodo’s interface including Notifications, Reply, and Turn-in functions. To help students’ use of Glogster, she provided mini-lessons and a handout that students could use later, which allowed the students to add contents (e.g., text, image, song) to their pages without problems.

Adopting an SFL-informed genre pedagogy (Feez 1998; Rothery 1996), Ms. Hunt created mini lessons on the language features needed in argumentative writing; and taught students how to write a multimodal argumentative letter through a teaching-learning cycle that she developed, drawing from both the school writing workshop approach (Calkins 2010) and the work by SFL scholars that she had read in her graduate courses (Gebhard and Harman 2011; Harman 2013; Unsworth 2006; Schleppegrell and Go 2007). The cycle involved the following stages:

Orientation and Modeling

Ms. Hunt oriented the students to the purpose and function of the argument by discussing with them the features of mentor argumentative texts from books and the Internet. In addition, the class did a close reading activity (e.g., Presidents are just like us, President Obama’s Back to School Speech). For instance, students read an informational text about Rosa Parks’ bus boycott and posted their answers to an argumentative prompt, drawing on evidence from the text in Edmodo. After completing answers, the students took a poll about the how their textual evidence provided good support for their answer to the questions. Overall, the classroom teacher focused on building a shared context for learning while simultaneously familiarizing students with new technologies for multimodal text production.

Deconstruction and Joint Construction

Ms. Hunt further supported multimodal argument writing through a class activity where students created a multimodal argument for or against year-round schooling in Glogster. Her joint construction of the projects supported students in seeing how the semiotic resources available in Glogster, including linguistic and non-linguistic resources, could be used to realize the ideational, interpersonal, and textual meanings of their arguments. To enhance students’ understanding of the force of intermodal meanings across various modes, she showed how image and word interacted together in creating ideational and interpersonal meanings through co-elaborating, complementing, and connecting relationships. To align with students’ previous learning with the writing workshop and the 6 + 1 traits, she also used such meta linguistic terms as “ideas”, “voice”, and “organization” respectively for ideational, interpersonal, and textual meanings to foster students’ metalanguage awareness.

Independent Writing

Once they had spent time thinking and jointly constructing an ensemble of multimodal and linguistic resources to build arguments, Ms. Hunt requested that students begin writing their argumentative letter to the President on their selected topics for changing America. After brainstorming and composing a first draft in their writers’ notebooks, they completed their text using Microsoft Word on the computers or Pages on an IPad. While writing their drafts, the students exchanged ideas about selected topics with peers in Edmodo to develop a deeper, more critical understanding of their topics. When the students finished their drafts, they exchanged feedback on each other’s texts in Edmodo about how various modes were orchestrated into each multimodal ensemble. The classroom teacher also conducted group conferences with the students to check on their progress in writing and provided feedback on their letters throughout the writing processes via face-to-face and Edmodo.

Publishing

The students published their multimodal Glogster texts in Edmodo and made comments on each other’s work, in addition to sending the letters to the President. In doing so, they were encouraged to critically reflect on their own textual practices. The students’ letters and Glogster texts were later posted on the school district website.

4.4 Data Collection and Analysis

I collected multiple domains of data over the course of a semester to make a thick description of participants’ multimodal writing processes (Denzin and Lincoln 2003; Dyson 2003). The data collected included student’s written texts, Glogster postings, field notes about classroom interactions, interview data, and instructional materials. Student’s written texts and Glogster postings were the primary sources for examining student writing process, while field notes, interview data, and instructional materials furnished supplementary data for contextual information about student’s designing processes. Drawing on a case study model (Merriam 2009), I conducted a textual analysis of a focal student’s texts and Glogster postings with a backdrop of the face-to-face and Edmodo classroom interactions. As stated before, the analytical framework that I developed was grounded in a social semiotic perspective of writing as design and multimodality (Jewitt and Kress 2003; Unsworth 2006). Employing a constant comparative analysis (Strauss and Corbin 1998), I analyzed the collected data and coded it for modes, intermodal relationships, rhetorical choices, and evaluative stance. The unit of analysis was the context of production and creation of the different drafts of Sonny’s work: that included exploration of the instruction, discussions and drafts of his multimodal text. The exploration allowed me to understand Sonny’s orchestration of multimodal resources and intermodal awareness across time.

5 Findings

5.1 Appropriation of Semiotic Affordances in Digital Technologies

Among the modes available in Glogster, Sonny selected image, text, and sound in designing his multimodal argumentative letter with use of images and texts as primary semiotic resources. He first brainstormed various possible topics (e.g., shorter school days, longer specials, no homework, no reading, getting your car, longer recess). While Sonny was finalizing his topic (i.e., shorter school days) for his letter, Ms. Hunt co-constructed an argumentative letter about having year-round schooling with the students to orient them to argument writing.

During the writing unit, there was a funeral service for two firefighters who lost their lives in a shooting tragedy in a neighboring town. The shooting occurred ten days after the Sandy Hook elementary school tragedy, and Ms. Hunt had a class discussion on public safety regarding gun violence and protection at school. The discussion led Sonny to change his topic and write a letter arguing for a day for commemorating the victims of the Sandy Hook elementary school tragedy and for more funding for protection at school. Drawing on the class discussion, Sonny composed his first draft mainly using linguistic mode, as seen in the Fig. 1.

Fig. 1
figure 1

Initial handwritten letter

Full size image

To make ideational meaning, Sonny drew from class discussion on the Sandy Hook tragedy. His intertextual appropriation of the class discussion led the argument letter to have two theses: creating a Memorial Day for the children in the Sandy Hook tragedy and providing better security and protection for children. In mediating his letter into a multimodal text through Glogster, Sonny narrowed down the theme of his letter to arguing for a Memorial Day. As seen in Fig. 2, he outlined his argument by mainly employing visual and linguistic modes such as vivid color and verbal descriptors.

Fig. 2
figure 2

First draft of Glogster letter

Full size image

Sonny first added a background color and a design with a logo of the school that he found on the Sandy Hook school homepage. The black background color enacted a somber mood while the school logo contributed to the main ideational meaning. He outlined key components of the argumentative letter including a title, thesis, and rationale for their claims along with the author’s name. After searching for information about the Sandy Hook Elementary shooting and victims online, he selected key ideas from his research for his letter. To make visual interconnections in the text, he colored the thesis statement in green to be consistent with the school’s logo color and the title of the text in red as a core part of the main thesis. In other words, he deliberately connected verbal and visual modes to highlight the overall cohesion of his multimodal text. I could see in my discussions with him that he was making deliberate choices, indicating an emergent semiotic awareness.

In his design of the first draft (Fig. 2), Sonny replaced an image of a hot dog that he had initially chosen as a way of amusing readers with an image of a white horse, to honor the victims. Considering the seriousness of the message to the President, he explained that the images should evoke a reverential sentiment, and that the white horse construed the appropriate interpersonal meaning that he intended to represent. In terms of textual meaning, the spread logos on the screen and the stark title in red on the top left created intense compositional meanings. Regarding visual grammar (Kress and van Leeuwen 2006), the modal ensemble of the black color, the Sandy Hook logo, and an image of a white horse clearly calls on readers to commemorate the children solemnly while engaging with the verbal thesis statement.

After setting up the layout of the text, Sonny added the reasons for having a Memorial Day by drawing on what he had written in the initial hand-written letter. Sonny’s redesign of the first Glogster text (see Fig. 1) also involved adding a new aural mode (i.e., song) and omitting some of the verbal elements (e.g., his name, details of Sandy Hook shooting). Overall, he synthesized semiotic resources into a multimodal ensemble. Figure 3 below shows his redesigned second Glogster text:

Fig. 3
figure 3

Second draft of Glogster letter

Full size image

In this version, Sonny used appraisal resources to explicitly point to the tragic nature of the event. Aligned with the expansion of modes and more evaluative language, Sonny placed a link to a song from the Sandy Hook school site that plays in the background; he also replaced the white horse image with a picture of the school. These visual and audio elements which directly resemble and represent a real image of Sandy Hook school reinforce the credibility and coherence of the text (Unsworth 2006). To obtain readers’/viewers’ attention and to engage their curiosity, Sonny added a text “Sandy Hook has their very own song!” on the top of the rainbow icon for playing music. The added image and song convey the information in a direct and accessible way, in comparison to his earlier use of the white horse image that could lead to multiple connotations and interpretations. The employed modes were laid out on the screen with text on the left and non-linguistic modes on the right following the traditional writing arrangement; he constructed coherent compositional meanings with semiotic resources that blended together through similar semiotic saliences.

5.2 Design and Rhetorical Decisions

The most salient rhetorical decisions that Sonny made relate to his developing semiotic awareness of how modes and language function in the context of designing multimodal argumentative texts. That is, his clear awareness of experiential and interpersonal functions and meanings of the letter led Sonny to make appropriate semiotic choices and rhetorical decision.

The dialogues between Sonny and his peer at the computer lab while composing the first draft of his letter show his rhetorical decision process:

  • Sonny was searching images for his text and found an image of a hot dog on the web. Michael looked at Sonny’s search while working on his own letter next to Sonny.

  • Michael: Cool. Put it.

  • Sonny: Yeah. It’s fun. (Smiling)

  • Michael: That’s really fun.

  • Sonny inserted the hot dog image, and a few minutes later he started to search for images again. He found an image of a white horse, and replaced the hot dog image with it. (Field Notes on February 22, 2013)

The dialogue and actions above show Sonny’s initial decision to construct an amusing and casual relationship with readers, as both Sonny and his peer expressed the intention of entertaining audiences. Later, Sonny changed the image and reconstructed the interpersonal meaning of his letter. In an informal interview with me about this image resource change, he expressed that if he had kept the hot dog image, “They will know I’m goofing around.” This explanation shows that he wanted to present the argument in a serious way and to address the audiences in a formal manner.

The knowledge that Sonny was developing of the function of semiotic resources continued to shape his semiotic and rhetorical choices throughout the designing processes. He received feedback for his drafting and revision from his peers and the teacher in Edmodo (e.g., “When do you think this memorial should be?”; “Why would this be important for all of America?”). In this way the two multimedia programs, Edmodo and Glogster, provided Sonny and the other students with an expansion of semiotic choices to enhance their ways of creating text. In his second draft, for example, Sonny responded to some of the critique from his peers and teacher in Edmodo by expanding on his verbal text; he also drew on other modal resources from Glogster to enhance the multi semiotic nature of the letter: he inserted the Sandy Hook song to convey a stronger collaborative tone to his artifact, employing the school song. In addition, he replaced the white horse image with an image of the school building. In an interview with me about this change, he explained that the readers would interpret the white horse image differently from his intended meaning, saying “I like the white horse, but some people won’t like it”. This explanation demonstrates his understanding of the force of interpersonal semiotic choices; his intention to address audiences in a solemn way guided his semiotic choices and rhetorical decision.

5.2.1 Intermodal Meanings and Relations

The modal resources that Sonny employed in his letter included text, image, color, and sound. These resources interact with each other to construe ideational, interpersonal, and textual meanings of the text. The modal resources that generate intersemiotic meanings could be categorized into the typology of text-image, text-sound, and image-sound relations. The following section presents how textual, visual, and aural resources construed intermodal meanings.

Text-Image Relation

The linguistic and visual texts that Sonny employed co-construct meanings in that an image of Sandy Hook and its logo enhance the meaning of the linguistic modes. In addition, the modes relate to each other by construing ideational meaning, as the school image and logo show the place where the tragedy occurred. The written text starts with the point of departure “The event that happened at the school” in the first sentence without providing further specific information about the location of the school. The verbal mode is spatially and ideationally connected to the image. The dark black background enhances the solemnness of the text, showing a complementary relationship of text and image for ideational meaning construction. Regarding interpersonal meaning, the employment of the dark background functions as an attitudinal intensifier to the emotion conveyed in the linguistic text. In terms of textual meaning, the text follows multimodal compositional grammar of text image that provides given information on the left and new information is placed on the right (Kress and van Leeuwen 1996). As such, the text and the image construe the ideational, interpersonal, and textual meanings in various relationships such as augmentation and connection (Unsworth 2006).

Text-Sound Relation

Among the four-element schema of sound as a semiotic resource in a multimodal text, music is the primary sound element in Sonny’s letter. Upon opening Sonny’s letter on Glogster, the reader/listener/viewer becomes engaged with children singing the Sandy Hook song in a chorus to a guitar melody. The music of the song engages the listener/viewer of Sonny’s text on the sensuous plane through the sound quality, and intensity of the sound; on the expressive plane that elicits feelings through the sound; and on the musical plane through the rhythm, tempo, and pitch (McKee 2006). On the expressive plane, the song in isolation is cheerful and hopeful with children singing about their hard work and their fun learning experiences in Sandy Hook as seen in the lyrics below:

  • Three cheers for the green and the white,

  • And Sandy Hook School forever.

  • Think you can, work hard,

  • Get smart, and be kind.

  • Sandy Hook Elementary

  • A very special place to be.

  • We’ll have lots of fun and we’ll know,

  • We’ll do our best, our very best,

  • To learn and grow.

The force of Sonny’s overall composition emerges from the juxtaposition of the cheerful lyrics and rhythm of the song to the somber verbal argument in a complementary relationship of divergence (Unsworth 2006, p. 62). With the inclusion of the song, the artifact forces the viewer to consider the lives the children could have lived without the tragedy when deciding about having a Memorial Day or not. As such, the intermodal relationship between the linguistic text and music adds another layer of ideational and interpersonal meanings to the letter.

Image-Sound Relation

The images of the Sandy Hook logo and the school building show an ideational concurrence with the school song. In a concurrence relationship, these semiotic resources represent a form of redundancy across modes; however, they are not a simple inter-modal repetition of meaning. Each of the semiotic resources provides different information about the school—name of the school, the school building, and the school song sung by its students. From the intersemiosis of the visual and aural resources, the total meaning of these parts is “more than adding up the meaning made by each independent modality” (Fei 2004, p. 225). This semiotic expansion is comparable to the homospatiality (see an example of a “visual image of the smoke emitted by the campfire” on Fei 2004, p. 240) that describes reinforced meanings construed through disparate elements but within the same spatial entity (Unsworth 2006; O’Halloran 2004). The intermodal relationship in Sonny’s text highlights how two different modes multiply meanings, even though co-occurring in a spatially bonded homogenous entity (Fei 2004). The Sandy Hook logo, song, and building picture in the multimodal text collectively compel readers/listeners to commemorate the victims with an intensified solemnness, which increases the legitimacy of his argument for a memorial.

6 Conclusion and Implications

The study investigated the designing process of an argumentative multimodal letter of a sixth grade bilingual student in an English language arts class. It focused on the semiotic modal choices, intermodal relationships, and semiotic and rhetorical decisions that the focal student made in a multimodal compositional curricular unit. The findings show that the student Sonny was able to produce a multimodal ensemble that employed linguistic, visual, and aural modes and semiotic choices that were appropriate to the purpose and audience; he distributed ideational meanings of the letter across linguistic and visual modes with growing understanding of the inter-semiotic relationship. In sum, because Sonny was given instruction and permission to draw from an expanded repertoire of media and modes for multimodal designing, he developed an embodied understanding of how to employ various semiotic modes and intermodal relations.

Although Sonny showed sophisticated knowledge of modes and intermodal relations in his multimodal writing, my analysis of his process and products reveal that Sonny could develop his semiotic competence in expanded ways. This section discusses the key features of Sonny’s semiotic competence development as well as the additional multimodal instruction that could deepen his meta-semiotic awareness.

Sonny’s initial choice of semiotic modes for his multimodal composition mainly focused on linguistic and visual modes among available modes in the new medium Glogster. Although he was oriented to all the available modes in the medium by Ms. Hunt, the linguistic and visual modes provided more affordances for Sonny than other modes (e.g., movie, size, spatial relation, shape). This confirms that our past habitual use of media and modes can determine how we avail of the new, showing that “new media has dimensions of old media within” (Leander 2009, p. 163). Similarly, his previous school writing, mostly written with paper and pencil, privileged linguistic resources as the primary carriers of information and visual modes as an interactive hook. Sonny’s design process demonstrated the “old wine in a new bottle” issue that uses traditional compositional norms with new media (Shanahan 2013, p. 223). For instance, when he inserted images of a hot dog and a white horse image, his intention was to use the images as resources for entertaining readers and grabbing their attention or for illustrating the emotion that the linguistic text intended to construe, rather than as an ideational meaning function. Sonny could have benefitted from explicit instruction to bridge the gap between new and old ways of using media in multimodal composing (see Potts, this volume).

Another prominent feature of Sonny’s designing process relates to his constructing of intermodal meanings across linguistic and visual modes. Drawing on intermodal relations such as concurrence and complementarity, he distributed meanings across modes. Sonny showed a developing awareness of how intermodal meanings function across modes, as he started to utilize images to create ideational meanings in the text. Instructional scaffolding that focuses on the intersemiosis of various sign systems and intermodal relations would support Sonny in continuing to develop nuanced understandings of how to represent and communicate multiple meanings in multimodal composition.

In conclusion, explicit instruction on multimodal writing should provide opportunities for students not only to engage with a variety of media of communication, but also to appropriate modal and intermodal affordances of semiotic resources for various purposes of multimodal writing. That kind of instruction would foster the development of multilingual students’ metalanguage for multimodal meaning-making processes and composition. In addition, it would support students in representing and communicating ideas in a strategic way with various semiotic systems. Such semiotic competence allows students to develop critical awareness of semiosis in the increasingly multilingual and multimodal communications of the current era.