Self-Directed Language Learning: A Semiotic Analysis of a Language Learning App

Abstract

Virtual learning sites give learners unprecedented control and autonomy over their own learning. The flexible learning pathway is mainly contributed by the design of the apps, which offers a variety of linguistic and semiotic resources for language learning. So far, there is only limited research on the multimodal design of language learning apps. This chapter examines the semiotic arrangement of the Memrise app, which is a multilingual virtual learning site with a focus on vocabulary learning. In particular, this chapter aims to (1) map out the kind of resources that are provided for learners, (2) the way these resources are realised in the design of the app and (3) the way in which these resources are orchestrated and contribute to language learning through conducting multimodal social semiotic analysis. This chapter concludes with a critical view of the use of technology in education.

1 Introduction

The widespread availability of mobile technologies, helped by the positive attitude of viewing multilingualism as an asset, have together contributed to a change in the way people learn a language, leading to the rise in popularity of language learning apps. So far, research on technology-assisted language learning, as well as virtual learning, has been focusing on usability (Brick, 2011; Stevenson & Liu, 2010; Zourou, 2012), material design and evaluation (Chapelle, 1998, 2009; Reinders & Lewis, 2005), as well as learner autonomy and motivation (Alm, 2006; Healey, 2007; Lai, 2013; Lamy & Goodfellow, 1999; Terhune, 2015; Ushida, 2013). Moreover, a considerable number of these studies focused on the use of digital technology in institutionalised settings, e.g. to be used in language classes, or to be assigned as out-of-class supplementary learning activities (see e.g. Hampel & Stickler, 2012). There is oftentimes an implicit, if not explicit goal for using these digital technologies to enhance social interaction, both between learners, as well as between learners and teachers. While the above areas are important in understanding the different aspects of virtual learning sites, a more fundamental analysis of the semiotic arrangement of these platforms is also required to shed light on how the platforms are designed to facilitate language learning.

In this chapter I am concerned with examining a popular language learning app which is called Memrise. It has been downloaded over 1,000,000 times on Google Play as at mid-2018, and is awarded as the ‘Best App’ in Google Play Awards in 2017 (Google Play, 2018). This chapter is partly informed by a larger study by Ho (2018b) on self-directed Chinese learning using Memrise (see also Ho, 2018a; Li & Ho, 2018). In that study, Ho observed how eleven transnational learners mobilised their multilingual and multimodal repertoires, as well as the resources provided by Memrise to learn Chinese as a foreign language. Using the Memrise app as an example, this chapter sets out with the following research question: How does the multimodal design of the Memrise app contribute to language learning?

This chapter is structured as follows: I begin by reviewing some of the existing studies in the area regarding the use of mobile technologies in language learning, and how multimodality is helpful in understanding the design of this kind of learning environment. I then proceed to unpack the kind of resources that are available to language learners. Afterwards, I demonstrate the pedagogic work performed by individual modes and discuss how each of them facilitate the learning of Chinese. Lastly, I examine how the complex interplay of the multimodal ensemble can offer us some insights into material design in virtual learning sites.

2 Mobility and Language Learning

Language learning, once an activity that was conducted within the four walls of a classroom, usually supervised by a teacher, has now gone mobile. It can now be done in the comfort of one’s home, on the train, in a café, and so on. The possibilities are limitless, thanks to the rise in availability of smartphones and tablets that are small enough to be carried around. From the discussion of Pachler, Bachmair, and Cook (2010), mobile devices enable learning to penetrate into people’s everyday lives, blurring the boundary between learning and everyday lives.

The use of mobile technology has been found to have changed students’ learning strategies. In a study that investigated learning strategies used by learners of Chinese when they learned to write Chinese characters, Kan, Owen, and Bax (2018) found that as new ways of learning characters are used, such as using mobile apps, new learning strategies emerged as a result, such as the use of pinyin (the official Romanisation system of Chinese characters) input to choose/identify the correct characters on the screen, which was not possible before technology became common. They also identified some strategies that had been used previously, but technology has made them more important than before. For instance, while previously Shen (2005) had identified self-testing as one of the strategies for character learning, its importance has grown due to the use of mobile learning apps, possibly because digital technology offers affordances that enable easier ways of self-testing without using pen and paper.

Online language learning platforms such as Memrise featured in this study have received attention from researchers in recent years. For instance, Stevenson and Liu’s (2010) study explored how learners make use of the social networking features of three selected platforms to fulfil their goal of learning a new language. Through using online surveys and usability testing methods, they found that despite students’ interests in the social elements offered by these platforms, they still valued traditional means of language learning, such as structured curriculum, more highly than the social aspects. Students participating in the study also expressed concern over the quality of user-created content. Harrison and Thomas’s (2009) study examined how students in a Japanese university used Livemocha, an online language learning platform, to learn a foreign language of their choice. The authors observed students in the classroom while they were using Livemocha through classroom monitoring and ‘think-aloud’ sessions in five bi-weekly classes. They found that the use of Livemocha helped language learners explore new social networks, and that the mediation between learners and the site, as well as between learners, has to be constructed by learners themselves in a trial-and-error process.

In addition to understanding the user experience, a fundamental analysis of the semiotic arrangement on which these platforms based their design on is also important. Chik (2015) compared two such platforms, Duolingo and Busuu, by using positioning theory in order to understand how platform designers position themselves through making various claims about language learning. She identified textual and semiotic devices that the platforms used to promote various positionings and conceptualisations of language learning. Her study revealed the importance of understanding the design of the platforms based on how they are designed textually and semiotically so as to understand the language learning beliefs that these platforms intend to promote, and how learners are put in a position to accept these beliefs.

There is currently no consensus in the literature as to what virtual learning sites for language learning featured in the review above should be called. Depending on the features that researchers would like to highlight, they have been called differently in the literature, some of these names include ‘Social Networking Sites’ (Brick, 2011, 2012; Harrison & Thomas, 2009), ‘Social Network Sites for Language Learning’ (Liu et al., 2013, 2015), ‘language learning social network sites’ (Chik, 2015), ‘(Structured) web 2.0 language learning communities’ (Zourou, 2012; Zourou & Loiseau, 2013), and the list goes on. All these names emphasise the social and interactive nature of these platforms, focusing on their affordances regarding language learning and social networking. For me, I have preferred to foreground the fact that they are online spaces for language learning, not necessarily ‘social’ or ‘networking’ spaces (although they could well be used by learners in such ways), so I have chosen to call them ‘Online Language Learning Platforms’, which hopefully could focus our attention to the ‘online’ and ‘language learning’ perspectives, without over-emphasising the ‘social’ and ‘networking’ features.

From the discussion above, it can be seen that language learning is gradually moving from classroom-based instruction to virtual sites beyond the classroom which affords higher flexibility and mobility. As Levy & Stockwell (2006, p. 107) pointed out, successful virtual learning sites need to have “clear pedagogical objectives in mind, knowledge of the technological options and an awareness of the needs, goals and skills of the learners”. While a considerable number of studies have the tendency to focus on using technology to enhance social interaction, to explore ways to enhance user experience, as well as to develop ways to evaluate these self-directed learning materials, all in all, a better understanding of the semiotic resources provided by these platforms is needed to unpack the pedagogic design of the platform, and how the affordances of the resources facilitate language learning.

3 Multimodality and Language Learning Apps

All communications are multimodal, and language learning is no exception. In Jewitt’s words, “[m]ultimodality describes approaches that understand communication and representation to be more than about language, and which attend to the full range of communicational forms people use…and the relationships between them” (Jewitt, 2009, p. 14). In the context of education, in the earlier decades before digital technology was widely used in the classroom, there had been an ideological preoccupation with language. Language had been the dominant mode used in teaching and learning contexts, with other modes being used in the periphery. In the contemporary education landscape, different modes are now more easily accessible thanks to the development of technology which enables teachers and learners to have a wider access of modes in an easily accessible way. For example, years ago, training to be a language teacher would include learning to draw simple pictures on the board. Twenty years later, ready-made images are widely and easily available, making it a valuable resource in the language classroom. This change challenges the centrality of language in the education context and points to a need in an approach that takes into account modes beyond language.

In a multimodal approach, language is considered to be one of the modes to make meaning, amongst other modes such as image, sound, colour, etc. Kress (2010) defines mode as a “socially shaped and culturally given semiotic resource for making meaning” (p. 79). Each mode has different potentials for making meaning which are shaped through their cultural, historical and social uses. Individual modes contributes to meaning in their own ways, for instance, what can be done through language differs from what can be done through image, and therefore, all modes, including language, are seen as part of a multimodal ensemble which have to be understood in its entirety (ibid.). To sum up, modes do not work individually; they orchestrate to make meanings. This will be discussed in Sect. 9.

4 A Social Semiotic Approach to Learning

This chapter employs a social semiotic approach to learning to make sense of the increasingly multimodal nature of learning and learning environments. By exploring learning through the lens of social semiotics, it can be seen that the design of the learning environment is just as important as other aspects of learning, as it affects the possibilities of learning by determining what resources are available to learners, and how learners can utilise and transform those resources. The importance of the design of learning environment is summarised by the following quote from Kress: “Learning happens in specific environments that offer specific semiotic/conceptual resources in particular configurations” (Kress, 2009, p. 20; original emphasis). Multimodal design, as explained by Kress (2010), is the use of different modes “to present, to realize, at times to (re-)contextualize social positions and relations, as well as knowledge in specific arrangements for a specific audience” (p. 139). From the above, it can therefore be seen that learning, from a social semiotic frame, involves constant interaction and meaning-making between the learner and the surrounding learning environment. Learning is about transformative engagement that sign-makers do not simply internalise the signs created by others; instead, they actively engage with the world and create new signs to demonstrate their learning. Learners are encouraged to build collaborative dialogue, to negotiate meanings, as well as to use language to mediate their conceptualisations and thinking, all of which achieved through social interactions (see Swain, 2006; Swain & Lapkin, 2013).

Furthermore, as Li (2018, p. 22) argues, language is a “multilingual, multisemiotic, multisensory, and multimodal resource for sense- and meaning-making”. Understanding language learning would require us to see it as a multimodal and embodied activity. Rather than seeing language learning as a passive activity, Li (2018) invites us to see it as “a process of embodied participation and resemiotization” (Li, 2018, p. 9; see also Gallagher & Zahavi, 2008). Adopting a multimodal lens in this study would enable us to unpack how the learning environment of the featured language learning app is designed, what kind of resources are offered, and how the multimodal design of the app contributes to language learning. With the affordances of digital technology that allows for digital mashup or remixing, it is now much easier for learners to add personal touches to existing artifacts as well as to demonstrate their signs of learning through resemiotization (Godwin-Jones, 2012; Hafner, 2015; Iedema, 2003; see also Li & Ho, 2018). As Motteram (2011) observed, the adoption of virtual learning environments not only puts the power of content-creation into the hands of teachers, but also into the hands of students as well, which echoes what Jones and Hafner (2012) has described as the move from the “read-only web” to the “read-write web” which allows internet users with a greater power to participate and contribute. In addition, it is also an indisputable fact that virtual learning platforms offer a wide range of semiotic resources for teachers and students to be content-creators, leading to wide variation in the content, style and aesthetics of sign-making practices (Adami, 2018b).

Pedagogy, from a social semiotic perspective, is broadly conceptualised as the social relations that exist in the classroom. It is concerned with how relations between school, teachers and students are constructed using different resources, such as the spatial arrangement of the classroom, the design of the curriculum and so on. Similar to a physical school, Memrise has its own pedagogic assumptions of how languages should be taught and learned, and they are realised in the way the platform is designed, more specifically, the selection of modal resources. Here Memrise takes on the role as designer of the learning environment, just like teachers in ‘conventional’ learning settings.

5 An Introduction to Memrise

Memrise is one of the many online platforms available on the Internet that offers free language courses. It was founded in 2005 and is based in London. It is available as desktop site and mobile app, and the latter is the focus of this chapter (see Ho (2018b) for an analysis of the Memrise desktop site). Thanks to its crowdsourcing nature, Memrise is able to offer courses teaching more than 200 different languages, from languages that are widely spoken in the world (e.g. English, Spanish, Chinese), to lesser-known languages (e.g. Inuktitut, Creek), to constructed languages (e.g. Esperanto, Toki Pona), to sign languages (e.g. British sign language, American sign language). It has over a million users from all over the world, and from my observation, its intended audience is people who want to learn a new language from building up functional vocabulary that they can use immediately. Memrise adopts a freemium model whereby users can access basic functions for free, but will have to pay if they want to unlock more functions. As for the ways of interaction between users, Memrise offers the possibility for users to create pedagogic materials, be it an entire lesson based on a particular topic, or a meme (note that ‘memes’ are called ‘mems’ in Memrise which refers to short multimodal texts which are created based on the principle of mnemonics), to be uploaded to the platform for others to use (see Li & Ho, 2018).

5.1 A Virtual Walk-Through of Memrise

The Memrise app and website have undergone a lot of updates during the research period, which is a common challenge associated with online research (see Fletcher (2007) for a discussion of the ephemerality of web-based research). The analysis presented here is conducted in June 2018, based on the Memrise app used on a tablet using the Android operating system. Despite the regular updates of the graphic elements and functions in Memrise, the method, analysis, as well as the results presented in this chapter are still relevant. The dashboard page features a space theme (as seen in the app version offered in the East Asia region). Now, learners take on the role as an avatar which looks like an astronaut. As learners become more advanced, they rise higher and higher up in space until they reach the next level.

Within each level there are numerous sub-levels. For instance, in the Swedish 1 course that I have just completed at the time of writing this chapter, the different sub-levels are shown in Fig. 11.1. Note that a reproduction of the page is given here instead of the actual screen, as I could not obtain permission from Memrise to include the original screenshots in this chapter due to copyright issues. This is the same with all other figures used in this chapter.

Fig. 11.1

A reproduction of the screenshot of Swedish 1 course

This page is arranged in a linear format. Learners are invited to start the learning journey from the bottom of the screen, and move up to the top as they progress from one planet to another. A close reading of Kress and van Leeuwen (2006) would tell us that the bottom-up reading path which starts from the bottom is not the usual arrangement for Western readers, who are more accustomed to reading from the top to the bottom. This can also be understood from the role of astronauts that learners are playing. The more planets they have explored, the further up they are in the outer space, thus learners are expected to progress from the bottom to the top. This reading path is also indicated linguistically by the use of writing and numbers.

In general, the dominant modes of this page are image and colour. The images shown on this page are mostly cartoon-like drawings that are colourful. The colours are unmodulated and contrastive, drawn with simple lines, which evokes the feeling of looking at a child’s drawing, but at the same time it looks professionally designed. Images shown on this page is on the theme of space, reinforced by the colour scheme of dark blue with tiny white dots, signifying the dark outer space and stars. Different sub-levels are shown as a planet. Before a sub-level is completed, the planet has no colour, and there is only a faint outline showing the silhouette of the planet. As the sub-level is being completed, a solid, thick yellow line starts to appear around the planet, indicating the proportion of the sub-level that is completed. After a sub-level is completed, the planet is fully-coloured. Colour is used to indicate several important information, such as which sub-level the learner is currently in, how many words needs reviewing (the number of words is indicated in a blue circle), and how many words are difficult for the learners (the number of words is indicated in an orange circle). The way these small circles are arranged around the planet resembles small planets oscillating around a big planet. Language is only used to name the sub-levels, and it is shown in relatively small font under each planet, in white in the original screenshot. Compared to the other elements of the page, it can be assumed that it is not intended to draw learners’ attention. Overall, salience is given to the planets (i.e. the sub-levels) by the use of bright colours against the dark blue background. The top of the page shows a grey bar showing the course being taken, the number of words learned over the total number of words that needs to be learned. The yellow line indicates the completion progress. All of these are considered ‘given’ information, according to Kress and van Leeuwen (2006).

It should also be noted that Memrise allows users to download a course (by tapping on the downward arrow on the top right corner) to learners’ devices so that they can take lessons even without Internet access. Towards the bottom of the page is a grey rectangular tap with round edges which shows the suggested learning activity that learners could do next. This is by no means prescriptive. Learners can override the suggestion and do something else if they tap on the button with four coloured dots. There they are given more options of the different learning activities they can do. The bottom part of the page is the menu bar where learners can gain access to different sections of the app (e.g. the profile page and the ‘Pro’ page which allows learners to access the paid section of the app).

Having established an initial idea of the appearance and the basic functions provided by the platform, the following sections discuss the pedagogic work performed by the selected modes: writing, speech, image, moving image, page layout. These modes are selected for analysis because they are all relevant to language learning.

6 Pedagogic Work of Modes

As Littlejohn (2011) suggested, there is a need now more than ever to analyse teaching materials closely in order to examine the implications these materials have to teaching and learning. While he made this suggestion from the context of developing English language teaching materials, the same is needed for other kinds of teaching materials as well. To do this, he called for an analytical framework which treats materials as “pedagogic device”. His framework urges us to pay attention to the tangible aspects of materials (“publication”), as well as the “thinking underlying the material” (“design”). The framework suggested by Littlejohn (2011) is indeed a comprehensive framework for material analysis in a relatively ‘macro’ manner, nevertheless, in this chapter, I present a considerably ‘micro’ semiotic analysis of materials. In view of this, a multimodal social semiotic approach is used to unpack the learning environment provided by Memrise to offer a more ‘micro’ level of analysis of Memrise as a provider of Chinese teaching materials.

The method of analysis is based on social semiotic multimodality. In particular, it is informed by the method outlined in Kress and van Leeuwen (2006) and Kress (2010). My data analysis is also inspired by the framework used by Adami (2014a, 2015) in her comparative study of the aesthetics of food blogs, and Bezemer and Kress’ (2016a) study of textbooks. I borrowed the concept of ‘semiotic work’ (Kress, 2015) from the work of social semiotics. It is a term to describe the sign-maker’s agentive, purposeful action. It brings about changes to the tools, to the worker, and to what is worked on. All these are meaningful, and lead to new meanings for the sign-maker and her/his resources (Kress; personal communication). Following from this understanding, in my analysis of Memrise I adopt the term ‘pedagogic work’ to highlight the fact that modes are purposefully used to achieve pedagogical purposes for language learning, which has the potential to bring about new meanings to learners when the mode is being used, in this case, in a pedagogical way.

In social semiotic approach to multimodality, there are two types of affordances: (1) modal affordances, and (2) affordances of the medium. Modal affordances focus on the “potentialities and constraints of different modes” (Jewitt, 2013, p. 254). This type of affordance focuses on what one mode, within a multimodal ensemble, can or cannot do. On the other hand, affordances of the medium consider the multimodal ensemble in its entirety. Instead of looking at what one mode can or cannot do, the affordances of the medium gives a functional analysis of what a range of modes can or cannot do (see chapter by Elisabetta Adami and Gunther Kress on affordances of smartphones in Multimodality: A Social Semiotic Approach to Contemporary Communication). In this chapter, I use modal affordances as my starting point. This view offers more in-depth insights about what functions individual modes play in this learning environment, and also the ‘pedagogic work’ they contribute to facilitate Chinese learning. While acknowledging the notion of multimodal ensemble and the fact that modes orchestrate to make meaning, I am separating modes in the following analysis due to methodological reasons. It has to be noted that modes always work in multimodal ensembles in the real world.

Now I turn my attention to examine the modal affordances of Memrise with reference to an actual learning page, this time featuring Chinese. The analysis below is part of a larger data set from Ho (2018b). The reproduction of the original screengrab shown in Fig. 11.2 shows a typical learning page. Different from learning from a textbook, the affordances of Memrise allow the creation of user-generated pedagogic materials which aims to promote collaborative knowledge creation and sharing (see Fig. 11.5). These user-generated pedagogic materials play an important role in language teaching and learning in Memrise. It can be seen that even though learners used Memrise to learn Chinese in isolation, once they are learning from materials offered by the site, they encounter materials created by others, and can possibly interact with these materials by ways afforded by the site. This could be seen as an example of ‘multimodal sign-making from below’ (Adami, 2018a; Pennycook & Otsuji, 2014).

Fig. 11.2

A reproduction of the learning page

The following analysis aims to compare and contrast the pedagogic work performed by different modes. This is summarised in Table 11.1, which is a modified version of the categories used in Adami’s (2014a) comparison of the aesthetics of food blogs.

Table 11.1 A summary of the pedagogic work performed by individual mode (adapted from Adami, 2014a and Ho, 2018b)

6.1 Writing

Figure 11.2 shows the learning page of a lesson in Memrise. In Memrise, there is a default language used, depending on the users’ choice. In the case of this chapter I have chosen English to be the default language, meaning that I intend to learn Chinese in English. The reason for featuring Chinese in my analysis is that Chinese has a different script system from other European languages, and its logographic nature can better demonstrate how the meme section of the app offers what Li (2011) called a ‘translanguaging space’ for learners to experiment with their creativity and criticality in their use of multimodal resources, which includes the use of multiple semiotic and linguistic resources that learners possess in their repertoire (see also Li & Ho, 2018). Different script systems are used in Memrise, such as Chinese characters, Chinese pinyin (short for Hanyu Pinyin, which refers to a standardised Chinese phonetic transcription system), and English. They are normally used separately from each other, except in the case of user-generated memes where different languages and script systems are sometimes used alongside each other in one sentence. An example of one such meme could be found in Li and Ho (2018). In Memrise, instead of using the ‘standardised’ form of pinyin symbols (i.e. hǎo for 好), it uses a different kind of pinyin symbols (i.e. hao3 for 好). This is probably due to the difficulty in typing the tone symbols, and therefore numbers are used to replace the tone symbols.

The font used in Memrise is the standard font generated from the computer. The same font is used consistently throughout the app. The writing that is presented in most Chinese courses is standard simplified Chinese. In terms of the register, it is usually formal, but in some cases, such as in memes, the register may sometimes be informal, as memes allow a space for the creative use of language.

Writing is used to show the target content matter that learners are supposed to learn. For instance, writing is used in the target Chinese character, as well as the English translation underneath it. The fact that the Chinese character is in a slightly bigger font, as well as in solid, black colour as compared to the smaller font and grey colour used in the English translation indicates that salience is given to the Chinese character. Writing is also used to indicate what ‘hao3’ means, as beginning Chinese learners may not know the pinyin system. The word ‘PRONUNCIATION’ in pink is given salience by means of colour and the use of capital letters. Looking at the meme which occupies the bottom half of the screen, the Chinese character is shown in the centre, in a large font. It indicates the importance of knowing the written form of the character. It is seen as the most important thing that learners are supposed to learn from the content designer’s perspective.

6.2 Speech

Speech is another important mode for language learning. Speech in the case of the Memrise platform refers to the audio recording uploaded by Memrise, or other users of Memrise, that allows learners to listen to the pronunciation of the target vocabulary. In Memrise, audio is represented by the image of a blue speaker that is located towards the top of the page (Fig. 11.4). The recording is triggered upon tapping on the blue speaker. The quality of the recording varies from lesson to lesson, depending on who created the recording. Some are of studio-quality and no background noise can be heard, which seem to be professionally-made, possibly made in-house by Memrise, while in others some background noise can be heard, which seem to be more amateurish. The recordings also feature different accents of Chinese, as they are user-generated. Speech offers a way for learners to control the pace of the learning. Learners are free to listen to the recording repeatedly as they wish. It can be seen that the function of speech in Memrise is mainly to demonstrate the pronunciation of the target vocabulary at learners’ own pace.

6.3 Image

On this learning page, there are two types of images: professionally-designed images such as the flower (Fig. 11.3) and the blue speaker (Fig. 11.4), as well as hand-drawn images such as the woman and the child images (Fig. 11.5). The hand-drawn images are presented in such a way that the wooden colour pencil marks can roughly be seen in the original shown on a screen. When displayed on the same screen together with the professionally-designed images, it is accentuated that these hand-drawn images are not professionally-designed, thereby not provided by the platform, but more likely by individuals. In this example, perhaps coincidentally, the meme is contributed by Ben Whately, a co-founder and Chief Operating Officer of Memrise.

Fig. 11.3

A reproduction of the image of a flower and lightning to indicate a difficult word

Fig. 11.4

A reproduction of the image of the blue speaker

Fig. 11.5

A reproduction of the image of a user-generated meme to illustrate the relationship of the components of a character

Images are used both for navigation and to illustrate learning materials. For example, the image of the blue speaker means audio. When learners tap on it, they can hear the pronunciation of the word. The flower shown on the top right-hand corner of the screen shows how familiar the word is to a learner. For words that learners encounter for the first time, only a leaf can be seen. As learners encounter it for a few more times, it gradually turns into a flower, as shown in Fig. 11.3. The lightning image in orange indicates difficult words that learners need to pay extra attention to (Fig. 11.3). This is determined by the number of wrong answers given by learners in previous exercises. In other words, this is a way for the platform to make learning more individualised by making use of some kind of algorithms. In memes, images are used to provide visual resources to help learners to remember a word, often by association. In Fig. 11.5, the character 好is made up of two radicals: 女 and子, meaning ‘woman’ and ‘son’ respectively. The corresponding images are shown in the meme so that learners can remember this combination more easily. The constant use of cartoon images in Memrise may be what Chik (2015) described as a kind of “discursive practices of infantilising learners as a display of power relations” (p. 127). Nonetheless, from the perspective of language learning, when images form a part of teaching materials, they are likely to make it easier for learners as language learning is a multimodal and embodied experience.

6.4 Moving Image

In Memrise, moving image, or animation, is used primarily in memes. Learners can swipe back and forth between different memes and decide which one they want to choose. This use of moving image allows learners to control the pace of learning, and also to decide on the learning content that they would like to pay more attention to. When used in a meme, moving image is used to illustrate the relationships between different radicals, for instance, how they combine to form a new word. This is particularly helpful for Chinese learners to understand how a character is constructed visually. Nonetheless, moving image is not a dominant mode used throughout the app, probably because creating moving images requires more technical skills and so it signifies certain degree of professionalism and technical know-how.

6.5 Page Layout

The page contains only the essential information, with minimal use of language and predominant use of images. The page looks clean, minimalist, but informative. This design ensures that learners are not overloaded with information, and they can focus on the important aspects of the target word without distractions, bearing in mind that a lot of learners may be using the app on the go. The page is divided into three parts, marked by the colours blue, white and grey. The blue part shows the progress of a lesson, as indicated by the words learned over the total number of words. The white part shows the content of the lesson, showing the form of the word/phrase, English translation, and pronunciation in pinyin. The grey box shows the memes created by other learners (Fig. 11.5 shows one such meme created by Ben Whately, one of the co-founders of Memrise). From a social semiotic perspective, the page is arranged in a linear way, and learners are encouraged to read from top to bottom, from left to right, a typical Western reading path (Kress & van Leeuwen, 2006). Salience is given to the grey area where the meme is located, and the images of a woman and the son is coloured, which aims to draw attention from learners when compared to the white background. Table 11.1 summarises the pedagogic work performed by each of the above modes.

7 Implications to Chinese Learning

The five modes that were analysed—writing, speech, image, moving image, page layout—all contribute to Chinese learning in their unique way. Not only are their individual contributions important, their collective contribution as multimodal ensemble is just as profound. Since language learning is a multimodal and multisensory experience (Li, 2018), it is crucial to understand the design of the learning environment through a multimodal lens which takes into account modes beyond language, and explore how the multimodal resources afforded by technology contributes to language learning.

The Chinese writing system is logographic, which is different from alphabetical writing (e.g. English, French, German, etc.), as Chinese writing is derived from graphs whereas alphabetical writing is derived from syllables (Xing, 2006). Chinese characters are made of strokes. However, a stroke, which is the smallest orthographic unit of a character, on its own does not take on a great deal of meaning. Strokes combine to form radicals—the smallest meaningful units which make up a Chinese character. They are largely similar to morphemes in the English language. Mastery of a Chinese character very often depends on the ability of learners to deconstruct the character into its constituting radicals which either give clues in terms of meaning from a semantic radical, or pronunciation from a phonetic radical (Shen & Ke, 2007). Learners then have to extract the meaning and in some cases, the pronunciation, based on what radicals are present, and how they are arranged. As a result, writing plays an important role in Memrise because it not only shows the completed form of the character, it also provides clues to which learners can try to deconstruct a given character, and subsequently make educated guesses when they see a new character that is unknown to them (Xing, 2006). To a certain extent, it encourages learners to make self-discovery and to manage their own learning.

Chinese is a tonal language. Subtle differences of tone would result in a completely different character, with a different meaning. Furthermore, as Chinese is a logographic script which lacks sound-script correspondence (Shen, 2005), very often pronunciation has to be learnt separately, as it may not be transparent to learners. It would be very helpful for learners to listen to the pronunciation and imitate the sounds. In the study by Ho (2018b), it was found that learners of Chinese often find remembering the pronunciation and practising them repeatedly a useful strategy to learn Chinese pronunciation, as language learning involves embodied participation. The affordances of including audio input in Memrise allow ‘proficient’ Chinese speakers to upload a recording of how they would say the character to Memrise so that it could be shared with other Chinese learners. It also exposes learners to a variety of accents of the language, which is something a standard textbook may not be able to provide.

As identified by Shen (2005), one of the major challenges that learners of Chinese faces is the complex graphic configurations of Chinese characters. Characters are made up of radicals, and radicals are formed by individual strokes. It could be difficult for beginners to deconstruct a character in order to extract the meaning and sound out of it. To make this important process easier, images are often used by educators to make salient the logographic property of the Chinese character more transparent by showing what the radicals represent, and how they are combined to give meaning to a character. Sometimes moving images (animations) are used to make the relationship more explicit, and for learners to visualise how a character is formed by its radicals.

The page layout of the screen serves a more general indicative function in which it shows learners the arrangement of the different learning content. The clean, modern, and minimalist design helps learners to focus on the learning content and not be distracted. It suggests a reading path for learners that is easy to follow, from top to bottom. Pedagogically, the clear, linear reading path reduces learners’ semiotic work in navigating the site, and directs them to the learning content. Nevertheless, a space (e.g. meme) is also offered to learners to remix and resemiotise content from other sources, and recontextualise it to create new meanings.

8 Orchestration of Modes

After examining the pedagogic work of individual modes, and their contributions to Chinese learning, it has to be reminded that as mentioned in previous sections, each mode has its own unique set of affordances and constraints. Jewitt (2009) pointed out that the meanings of different modes in a text are always interwoven and that they “co-present” and “co-operate” with each other in the communicative event (p. 15). A simple metaphor of the relationship between modes and multimodal ensemble could be illustrated by an orchestra. Different instruments play their own tunes, and together it becomes a piece of music that is enjoyed as a whole, not as music produced by separate instruments. Each mode has its own situated meaning and it varies from context to context. While the above analysis seems to have examined each mode in isolation, and how they contribute to meaning in different ways, one important understanding of multimodality is that modes combine and interact with each other to create new meanings.

As Kress (2010, p. 162) pointed out, “orchestration describes the processes of selecting/assembling/designing the semiotic ‘materials’ which seem essential to meet the rhetor’s interests…through the processes of design”. Platform designers are tasked with the work of understanding the affordances of modes, and bring them together in a way such that “semiotic harmony” of the multimodal ensemble can be achieved—orchestration (Kress, 2010). As each mode takes on different meaning, they are selected, arranged, sequenced, and orchestrated by the sign-makers, the platform designers.

As Adami (2014b; 2015) reminded us, interactivity is an affordance of digital texts. It enables users to act on texts using ‘interactive sites/signs’ such as links and buttons so as to create a limitless number of ways to navigate the site. This sets virtual learning platforms apart from other ways of learning, e.g. textbooks. The affordances of digital texts allow language learners to control their pace of learning, and to generate their own learning path based on their interests. In such kind of environment, learners are invited to ‘redesign’ the page and shape their own learning experience. This has profound impact for learning. In particular, the affordance of user-generated memes provide the space for learners to use resources creatively, which leads to a transformation of their semiotic/conceptual resources (Kress, 2010).

Examining Figs. 11.1 and 11.2 again, which show the dashboard page as well as the learning page respectively, it can be seen that when modes are brought together as an ensemble, a new meaning, a different aesthetic emerged. The space theme and avatar in the dashboard page has added the element of ‘gamification of language learning’ by means of inviting language learners to take on a different role and to conquer different levels of challenges; the cartoon-like, colourful images and animations has injected elements of fun and relaxation to the platform. On the other hand, the standardised writing and pronunciation in the learning page has brought us back to the reality of language learning, just like being in a language classroom. Although these elements may not look cohesive in a ‘traditional’ sense, however, once they are combined, there is a sense of “semiotic harmony” which cannot be accounted for without using the tools of multimodality.

The complex interplay of the five modes under investigation creates a fun and relaxing learning environment, which at the same time one which displays professionalism and expertise. This is done through the platform designers’ strategic and creative deployment of a range of modes, with an understanding of the functionalities, affordances, and aesthetics of each of them. The resulting virtual learning environment resembles the popular notion of ‘edutainment’, characterised by the heavy reliance on “visual material, on narrative or game-like formats, and on more informal, less didactic styles of address” (Buckingham & Scanlon, 2005; see also Ho, 2018b for an analysis of the alternation between formal and informal register used in the About Us page of Memrise).

9 Is Technology for Language Learning a Panacea?

The use of technology in the context of education is a fast developing field and it warrants more attention from researchers to understand the potentials and pitfalls of it, especially in self-directed learning contexts whereby learners have to make all learning decisions on their own. This chapter does not intend to provide a one-sided account of the advantages of technologies and virtual learning environments. It also calls for a critical view of the use of technology in language education. In language classrooms all over the world, it is not uncommon for teachers to equate the use of technology with interactivity. Numerous research done in the UK concluded that convincing evidence showing improvements in learning brought about by technology is elusive (see, e.g. Livingstone, 2012; Macaro et al., 2012). Selwyn (2015) argued that there is “a lack of a sustained critical perspective” in the present academic study of technology and education and asserted that the academic studies of technology and education tend to focus on the future benefits of using technology, but not enough attention is paid to the realities of the present.

No one can say for certain what the rapid growth of virtual learning sites would mean for language educators and learners. While there is some encouraging evidence that these sites have great potentials for language learning (see e.g. Hampel & Pleines, 2013; Hampel & Stickler, 2012; Lamy & Goodfellow, 1999), some educators would see mobile, virtual learning as a threat due to the fact that learners are learning ‘in the wild’, beyond the reach of teachers (see, e.g. Kukulska-Hulme, 2009). There are also studies showing that learners may lose motivation and become frustrated due to isolation, as well as the routine nature of tasks (Isbell, Rawal, Oh, & Loewen, 2017). Furthermore, more research is needed to understand, for instance, the impact of user-generated materials to the quality of learning. In particular, as researchers, language practitioners, or as content providers, how can these stakeholders strike the balance of adopting user-generated materials that are authentic on the one hand, and at the same time ensure the quality and appropriateness of the content on the other? Take for example the platform featured in this chapter whereby members of the community are able participate in the process of material creation. In an earlier study conducted by Ho (2018b), as well as in Stevenson and Liu (2010), quality of user-generated content is a concern for learners, and it affected how they choose to engage in the platform. For instance, in the case of Memrise, some of the learning content are created ‘in house’, which have presumably undergone some quality assurance processes. In the larger study in Ho (2018b), these courses are more popular than those that are created by other members of the platform. As researchers and content creators, there is an urgent need to account for these issues, as technology is allowing more and more participation from users to involve in content creation.

10 Conclusions

Language learning apps have become a popular way to learn new languages. The surge of the use of these apps is partly a result from the increase in mobility of the globalised world, and partly due to the increasingly positive attitude of multilingualism. People are no longer confined to learning languages from attending classroom-based lessons, and are more inclined to learn languages through apps so that learning can be done continuously, with minimal interruptions even with high transnational mobility. However, very little is known about how these apps are designed, and how these design decisions impact on language learning. This chapter demonstrated how a multimodal semiotic analysis can be done on an online language learning environment to achieve better understanding of the relationship between pedagogic material design and language learning. Nevertheless, it has to be recognised that, as Littlejohn (2011) reminded us, “analysing materials…is quite a different matter from analysing ‘materials-in-action’” (p. 181). Whilst the analysis of “materials-in-action”—such as the creation of meme and the creation of lesson—are examined in Li and Ho (2018) and Ho (2018b) respectively, this chapter aims to take the analysis one step back and consider the fundamentals of the design features of this virtual learning environment by offering a mode-by-mode analysis of the platform. Furthermore, this chapter have also elucidated how language learning is closely related to multimodality by examining the ‘pedagogic work’ performed by five selected modes and their relation to language learning, in the context of learning Chinese as a foreign language. Among the Mobile-Assisted Language Learning (MALL) implementation studies from 1994 to 2012 that Burston (2014) studied in his review, over 60% of MALL studies still focused on the learning of English; only less than 10% of the studies were on learning Chinese. The learning of Chinese through mobile apps is certainly an expanding area in the literature in need of more research.

The increasingly ‘isolated’ and mobile nature of language learning is largely a result of the advent of mobile technologies for language learning. Such kind of emergent learning practice has profound implications to the pedagogic design of language learning materials, as well as shaping our understanding of learning in the age of mobility which has become more fluid and flexible. From the analysis, it can be seen that not only can learners interact with the site through accessing content, Memrise also offers a space for teachers and learners to remix and resemiotize existing content, and transform them into new knowledge and share it with a wide audience, as in the case of the meme shown in the analysis. Language learning is an embodied activity which is multimodal. The analysis shown in this chapter shows the contribution of multiple modes to language learning. Contrary to the belief that writing is the dominant mode of language learning, I argued that the role of other modes, such as speech, image, moving image (animation), as well as page layout are equally important resources for language learning. Learning, as seen from a social semiotic perspective, is transformative engagement with the world (Bezemer & Kress, 2016b). It invites us to conceptualise learning using a different paradigm that can reflect the heterogeneity and mobility of the society. Language learning using virtual sites does not mean that learners are cut off from the outside world. On the contrary, learners can be exposed to a wider repertoire of ways of representation, thanks to the affordances of virtual learning sites such as Memrise. Not only can they interact with the site itself, learners can also interact with other learners through interacting with the signs created by each other. To sum up, this chapter has illustrated a how multimodal semiotic analysis can be useful for the analysis of learning environment, and to unpack the implications of learning. While each mode contribute to language learning in different ways, the orchestration and interplay of modes have created a kind of virtual learning environment that is both fun to use but appear professional at the same time. While interaction is not the focus of this chapter, it has profound effect on learning. Therefore, the way learners interact in such kind of learning environment is certainly an area that warrants more of our attention in the future.