Keywords

1 Introduction

English language learners (ELLs) can find it difficult to communicate with others (Demmans Epp et al. 2015b; Gambino et al. 2014) as can many speakers of English as a first language. These first language speakers of English struggle with communicating on their own because of medical conditions rather than a lack of language proficiency. To overcome these barriers, they employ a collection of strategies and tools that are referred to as assistive and augmentative communication (AAC) (Todman et al. 2008). Existing AAC was designed to meet the specific needs of these English as a first language users (Bruce 2009), and commercial interests have recently made smartphone-based AAC available. This move to enable access to AAC through commodity devices means these supports are now available to ELLs who could employ them to scaffold their communication (Demmans Epp 2013). However, ELLs may have different needs that are linked to their limited understanding of English, and tools have yet to be developed that support their communication.

As a result of these differences and the potential for AAC technologies to support ELL needs, an exploratory study (N = 12) was conducted to see how ELLs might use an AAC tool. Following this study, a new tool was developed to better meet ELL needs. The most important of which was the addition of on-demand content generation to support emergent communication and learning needs. This feature was validated through two studies (N = 16 and N = 202) before it was integrated into a new mobile-assisted language learning (MALL) tool. This hybrid MALL-AAC tool was then evaluated with a group of advanced ELLs (N = 8) to see how they integrated it into their activities. This evaluation showed the potential for these types of tools to support ELL communication and language-learning activities.

2 Mobile-Assisted Language Learning

MALL gets its name from the mobility of the learner or the tools learners use (Palalas 2011). MALL tools have become increasingly available (Beatty 2013) even if their use is still uncommon (Burston 2014a). Existing applications typically support a transmission-based model of learning (Beatty 2013; Burston 2014b; Stockwell 2012) that exposes learners to language by providing support resembling that of a dictionary (Demmans Epp et al. 2013; Procter-Legg et al. 2012) or by supporting vocabulary development (Duman et al. 2015; Veras et al. 2014; von Ahn 2013) through highly constrained memorization (Elmes and Fraser 2012; von Ahn 2013) or testing tasks (Garcia 2013; von Ahn 2013). These transmission-based systems are being expanded to include GPS-based features that situate learning (Dearman and Truong 2012; de Jong et al. 2010; Demouy and Kukulska-Hulme 2010) by delivering location-relevant content.

The lack of adoption of applications that are dedicated to supporting language learning may result from the focus on transmission-based models and the mismatch between these models and the pedagogical approaches that are currently favored (Burston 2014b; Kukulska-Hulme 2013; Sweeney 2013). This preference for communicative and socio-collaborative approaches makes the use of MALL tools within and outside the classroom appropriate because smartphones can help learners realize their potential by adaptively responding to changes in learners or their environment (Hung and Zhang 2011; Traxler 2013). In spite of this capability, socio-collaborative learning activities and the types of effortful free-form language production (i.e., languaging) that are known to support learning (Robinson et al. 2012; Swain 2006) have not been enabled through MALL tools unless a teacher has repurposed a tool by having learners record and submit samples of their language production (Burston 2014b) or communicate through other media, such as mobile blogs or email (Beatty 2010; Stockwell 2012). Languaging has been minimally supported through dedicated MALL tools that require the learner to verbalize or respond to a specific prompt (von Ahn 2013; Demmans Epp 2017; Liu 2009), providing learners with rehearsal opportunities. However, these have not helped ELLs to transfer their knowledge to settings where they must interact with others.

In contrast, AAC tools support the communication of proficient speakers of English who cannot communicate on their own (Todman et al. 2008). AAC has also shown its potential for scaffolding the learning of first language speakers who have limited comprehension of that language (Demmans Epp et al. 2015a). Therefore, AAC could support ELL needs by providing learning content or by scaffolding their communication. Given the potential usefulness of these tools and the lack of communication support provided by existing MALL tools, this research explores how AAC and MALL approaches can be improved to support ELL needs in English language environments. This exploration includes the development of a hybrid approach to MALL that is then evaluated.

3 Exploring ELL Use of AAC

To understand how to improve the support provided by AAC, it was first necessary to understand how ELLs would use this class of mobile tools. Given the exploratory nature of this work, a user-centered design perspective (Rogers et al. 2011) was chosen to investigate how ELLs would use a commercial AAC tool to support their needs. This meant giving the AAC tool to ELLs and collecting information about how they used that tool in real-world settings. Twelve learners (Table 1) were given training in how different AAC features worked but were not told how they should use those features to support their language learning or communication. This guidance ensured they were able to use the app while allowing their emergent practices to be identified. These learners used the initial release of a specific AAC tool, called MyVoice (2011), for a little over 3 weeks before reporting on their experiences.

Table 1 Participant demographics: AAC study
Full size table

The app they used is representative of the general approach taken by digital AAC tools. MyVoice resembles a visual dictionary: it displays vocabulary as image-word or image-phrase pairs that can be verbalized using text-to-speech (Fig. 1). These vocabulary entries are hierarchically organized within categories the user can browse but not search. The provided support materials can be modified through a separate web interface but cannot be modified through the mobile interface.

Fig. 1
figure 1

A screenshot of one of the higher-level vocabulary categories (left) and the contents of the Tim’s category (right)

Full size image

Participant reports of how they used the app indicated these learners seemed to prefer receptive approaches to learning, which included their use of the AAC tool to support their study activities. This focus on using technology to support receptive learning activities was in conflict with learners’ concern over their ability to produce language.

3.1 Language-Learning Strategies

These learners were primarily concerned with their ability to communicate. Like so many learners before them, they used a collection of strategies to facilitate their communication and overcome barriers that were often the result of lacking vocabulary knowledge. Among these strategies was using cognates, using examples to illustrate what was meant until an interlocutor guessed the correct word, or seeking clarification.

However, this primary concern was not evident in the majority of the activities learners performed. Rather learners tended to perform receptive learning activities. They sought and consumed authentic English media that included community newsletters and children’s books because these texts were at an appropriate reading level. ELLs consumed these materials to develop their reading and listening comprehension. Some of the more advanced learners would read the textbook from their courses to scaffold their aural comprehension within lectures. Others used music, movies, radio, and television to develop their listening comprehension (see Table 2). Ju’s listening practice also involved eavesdropping on others’ conversations. Like Ju, those who aimed to develop their language production used conversations to develop their listening skills. However, they went a step further by interacting with others using English. This choice forced them to produce language which was believed to benefit their learning.

Table 2 ELL technology use – AAC study
Full size table

The general lack of activities that were dedicated to improving learner speaking ability appears to have resulted from a lack of opportunity to interact through English and a perception that speaking was the hardest aspect of language learning. Learners expressed a general sense of frustration because most people were not helpful or cooperative. This lack of cooperation led learners to rely on friends and family members when they needed to communicate orally, or they gave up on communicating orally. When they gave up on oral communication, either they resorted to preplanned written communication through letters, or they found someone who could communicate on their behalf.

3.2 Technology Use

As expected, participants used a variety of technologies to support their language-learning needs with computers, television, and mobile phone use being widespread (Table 2). Smartphones were not widely adopted (only Arash owned one), which partially explains why few of the reported technologies were dedicated to enabling language learning. Participants had instead repurposed or appropriated technologies (Dourish 2003) to support their language-learning activities. One example of this appropriation is their use of subtitles to verify their understanding of program dialogue. Another is Ju’s use of Wikipedia instead of a dictionary to find definitions for words.

Learners used paper and electronic dictionaries as well as electronic translators to support their communication (Table 2). None of them had used dedicated or adaptive language-learning software, and none had used communication support tools before.

3.3 Application Use

Like others (Demouy and Kukulska-Hulme 2010; Liu 2009), these participants welcomed the use of a mobile application to support their language-learning activities. No one reported problems with using the device or application, and learners expressed a desire and need for the type of vocabulary and communication support these tools provide. Their interest in accessing new and better tools seemed to be a primary motivator for learner participation. Like many other mobile applications (such as games or dictionaries) that deliver predetermined content, MyVoice was seen as a tool that delivered limited content with only one learner trying to improve that content. This behavior is consistent with a larger tendency to view apps as content delivery tools rather than content creation tools (Demmans Epp 2017).

Participants’ belief that their limited vocabulary knowledge inhibited their language production and comprehension is evident in their application use across spaces: they spent most of their time studying and reviewing word meanings (Fig. 2). Like previous mobile learners (Demmans Epp 2010, 2017; Munteanu et al. 2013; Tsourounis and Demmans Epp 2016), these ELLs practiced their listening skills and their pronunciation in private spaces so as not to bother others or draw attention to themselves, which could be embarrassing. Participating ELLs further used the text-to-speech feature to assess their phonetic decoding skills and vocabulary knowledge by performing dictation tasks when they were alone.

Fig. 2
figure 2

ELL use of an AAC tool

Full size image

Application use in public spaces by Luis, Adora, Mei, and Arash typically involved silently reviewing vocabulary while commuting. The public use of the AAC tool occasionally involved showing the application to friends or using it to support communication. For example, learners would use the images as a visual support. They would scan through these images to find the one associated with a word they wanted to use. This allowed them to see the text, which reminded them of the English labels for objects so they could make requests. They also used the text-to-speech feature to make requests when their accent impeded communication.

Beyond the above uses, participants requested additional features that could help them. The ability to record samples of language use and see how words are used was among these features. These results were consistent with prior findings (de Jong et al. 2010; Demouy and Kukulska-Hulme 2010). However, learners need to obtain new learning materials that are appropriate to their situations if they are to take full advantage of the opportunities afforded through mobile learning.

3.4 Summary

Learner experiences indicate it is not enough for these tools to provide a fixed set of support and learning materials. These materials need to be adjustable so emergent learner needs can be met. Keeping the limitations of the deployed communication support tool in mind, a new MALL tool that aimed to better support the communication and language-learning activities of ELLs was developed. This included the development of an on-demand content generation feature to support emergent learner needs.

4 Adaptive MALL Development Process

The design and validation of a new system was informed by user-centered design principles (Rogers et al. 2011) and the layered approach (Paramythis et al. 2010), which decomposes adaptive systems into the high-level stages that support adaptation. The layered approach also identifies appropriate methods for evaluating adaptive system components.

In keeping with these two practices, the base features and visual design of the new mobile learning system were grounded in the evaluated AAC. However, several modifications were made because of how ELLs used that tool. This included changes to how support and learning materials were organized and presented as well as the ability to translate words and phrases, look up definitions of newly encountered vocabulary, share content, and import content that has been created by other learners. This also included the adaptive recommendation of learning materials and the ability to request new learning materials based on a user-identified need in a way that is similar to how one might perform a Google search when trying to learn about something new.

The development and refinement of these new features and the learner-facing interface were conducted in parallel. To familiarize readers with the visible design components and increase familiarity with this new application, its design is discussed first. The adaptive system features are then explained. This explanation is followed by discussion of the methods used to create the on-demand content generation feature.

5 Application Design

Based on learner experiences, a number of elements from the tool they had used were adjusted. Following these adjustments, a variety of methods were employed to ensure system usability. The first was paper prototyping, which was followed by higher-fidelity (Hi-Fi) approaches. The Gestalt principles of visual design (Mullet and Sano 1995; Ware 2004) were applied to ensure the user’s visual attention was drawn to the appropriate interface elements. This resulted in several small changes to the color and spacing of interface elements. Nielson’s usability heuristics (Nielson 1994) were also applied: ten evaluators identified interface elements that were inconsistent with Nielson’s heuristics and those elements were refined accordingly. These types of evaluation methods were reapplied following modifications to ensure the design changes did not introduce new problems.

In addition to the use of the above evaluation methods, café studies (Konno and Fong 2013) were employed to ensure system usability. Café studies are a type of brief case study where individuals who are in a coffee shop or other public location are approached and asked if they are willing to test a mobile application. Those who agree are asked to complete a highly constrained task, such as editing the sentence that is associated with a particular word. The designer watches how the person proceeds and notes the person’s behavior. The designer then modifies the application in an effort to improve people’s ability to complete the targeted tasks.

This iterative, continual redesign process can be seen in Figs. 3 and 4, with Fig. 3 showing the process for the web-based client and Fig. 4 showing the process for the Android-based mobile client. In both cases, the initial designs that included random content were evaluated and then iteratively refined.

Fig. 3
figure 3

The interface (re)design process for the web-based client

Full size image
Fig. 4
figure 4

The interface (re)design process for the mobile-based client

Full size image

In the case of the web-based client interface, the process of going from initial system mock-ups to a final prototype was faster with most of the changes being relatively minor adjustments to the visual design. An example of one of these adjustments was the change in the delete button’s color: it was changed from bright red to gray to reduce both its visual importance and its contrast with respect to the surrounding buttons. This change was made because the high-contrast red version of the button seemed to encourage people to click on the delete button, and while anything that was deleted could be easily recovered through a recycle-bin-like feature, deleting content was not an action that should have been encouraged. The relative size of various interface elements (e.g., the category labels and images) was also adjusted, paging was added to improve performance, and colloquial language was removed (e.g., trashed was changed to deleted).

For the mobile interface, adjustments were made to accommodate learners’ existing cultures of use and differences in their individual learning behaviors. One of the features that was added to support these differences was the search feature. Learners were also given increased control over the audio features: they could choose when to use text-to-speech, record themselves, or record someone else saying something for later playback. They could also mute the audio should they want to. Another change was the addition of content recommendation and adjustments to the visual appearance of recommended materials so their addition was less obvious or jarring to users. Smaller design refinements included increasing the consistency in the visual representation and interactions used to engage different functions across platforms. This included changing the icon that was used to initiate content editing on the mobile client: it went from being a button with text that said “Edit This Term” to a button that looked like a pencil. The placement of certain functions was also changed to ensure that the same action had the exact same meaning on each screen: in this case, the text-to-speech and recordings were changed so they could only be played from the screen where individual words were shown.

This iterative redesign process was applied to both platforms. It enabled rapid improvement to the adaptive app. The use of these lightweight but powerful evaluation methods helped ensure the system, and its individual features were usable. However, the improvement of the aspects of the app learners see does not ensure the usefulness or effectiveness of the underlying computational methods. Concern over the computational methods employed is especially important in settings where the system uses complex procedures to make inferences, fulfill requests with limited information about user desires, or adapt elements of the system to a user. These types of adaptivity and adaptability require additional evaluation since they introduce complexity and uncertainty into the system.

6 Developing Adaptive System Features

The system’s adaptive and adaptable features were developed while the user interface was being refined. These features include adaptive testing, the recommendation of new learning content, and an on-demand content generation feature. This section will discuss the adaptive testing and content recommendation features. The development of on-demand content generation will be discussed later.

The recommendation of new learning content within this app is based on specific situated learning theories: extended and fast mapping. These theories state people learn the meaning of new vocabulary items by encountering them in context (Carey 2010). When people encounter a new word in context, they begin to develop a sense of that word’s meaning, and their understanding of its meaning and usage increases with subsequent and varied encounters. This theory was used to reason over student actions and infer when they knew a word and were ready to see new related words (i.e., synonyms and near-synonyms).

This reasoning process can be seen in Fig. 5. At the beginning, it is assumed the learner knows none of the vocabulary. The system then tracks each of the learner’s interactions with a vocabulary item (VCi) and infers a word is known when the learner has interacted with that word at least four times. At this point, the system shows the synonyms and near-synonyms (Si and SSi) of the word that is now, at least partially, known to the learner. The synonyms also show their synonyms. If the learner already knows a word with which the current vocabulary item is a synonym, then the first-degree synonyms (Si) of that vocabulary item are shown regardless of how many times the learner has interacted with that particular vocabulary item.

Fig. 5
figure 5

The state diagram describing the reasoning process for recommending new content for an individual content item (i). VCi = the number of times a learner has interacted with the vocabulary item, Si = the direct or first-degree synonyms and near-synonyms of an item, Ssi = the second-degree synonyms and near-synonyms of an item

Full size image

An example of content recommendation can be seen in Fig. 6. In this example, the system believes the learner knows SUV so the system shows the synonyms and near-synonyms (Si in Fig. 5) that are directly linked to SUV: vehicle and sport utility vehicle. The system also shows the related vocabulary that is one level removed from the word it thinks the learner knows (SSi in Fig. 5). So, in this case, the vocabulary entry for vehicle will display truck, lorry, van, car, SUV, and sport utility vehicle as being related. Similarly, truck will show vehicle as a related word, but it will not show lorry because the word that the learner is believed to know is too far removed from that synonym.

Fig. 6
figure 6

An example of content recommendation where the learner knows SUV so SUV shows its directly connected near-synonyms and those items show their directly connected synonyms. The arrows point to the synonyms shown by a word

Full size image

Since the adaptive provisioning of new learning content is based on the system’s ability to infer when a learner knows something, an adaptive testing feature was added. These adaptive tests had multiple purposes. The first was to allow the learner to test his or her knowledge. The second was to use the testing data to improve the underlying recommendation algorithm by adjusting its parameters using the results of learner tests. Reconfiguring the recommendation using thresholds that were empirically obtained based on evidence of each user’s knowledge allows the recommendation of new content to be more appropriately controlled: it would prevent those who take more time to learn new words from being overwhelmed by the recommendation of too many new learning materials and those who learn quickly would be less likely to become bored because the materials were too easy.

As with all features not based on prior empirical results, this adaptive feature ran the risk of encountering what is generally referred to as the cold-start problem. The cold-start problem is similar to that of the chicken and the egg. In an adaptive system, a model of the learner is needed to adapt something to him or her, but the learner needs to have interacted with the system for the system to have a model that can drive this adaptivity. This means learners must have a history of interaction if the adaptive feature is to work appropriately, but a new learner cannot have this history.

To overcome this problem, the developed mobile application used knowledge of the frequency of use of English words to select test items for newer users. It then transitioned to using the systems’ logs of which content the learner had interacted with to select test items. In both cases, learners are tested on frequent and infrequent items so that the test has some items where the learner should experience success and some that will challenge the learner. Selecting items that should have different levels of familiarity also provides information about how quickly the student learns these new words and can allow the system to adjust the processes represented in Figs. 5 and 6. This adjustment would be to the threshold used to infer learner knowledge: that threshold could be increased or lowered based on individual learner characteristics and histories. Alternatively, this information about individual learners could be used to adjust which activities are counted when determining whether the learner has met the threshold required to infer a word is known and thus receive recommendations for new learning materials. The full details of the recommendation process the system used can be seen in (Demmans Epp 2016).

7 Developing On-Demand Content Generation

As can be seen in Fig. 6, the content was reorganized using a graph structure so that it represented the multiple semantic relationships that exist among English phrases and words. This change from the simpler treelike structure that is commonly used in communication support tools added flexibility, and it enabled the scaffolded recommendation of new learning materials through the display of near synonyms. This change in data organization was made tractable through the addition of a search function that enabled users to find words by directly searching for them or by searching for all words belonging to a single category or set of categories. The application was also changed to allow users to see these categories alongside the words with which they were associated because the visibility of this information communicates content organization and can help improve vocabulary knowledge (Graves 2013; Wagner et al. 2007).

To further support learners, the ability to obtain learning materials on an as-needed or on-demand basis was added. However, it is not possible to predict all user needs given the variety of situations and contexts in which language learners can find themselves. So, instead of trying to solve this incredibly difficult problem ahead of time by creating a glut of potentially useless content, a feature was developed to support user-identified needs. This feature could create content when the user requested it, provided the user could identify what his or her needs were. To enable both of these requirements, the search box was repurposed to allow users to translate a word from their mother tongue to English, which supported their ability to identify the appropriate context in English. They could then ask the system to find or create materials to support the need they had identified.

If a user need could not be met with existing content, then computational methods could be used to meet emergent learner needs. This on-demand content generation feature augments existing materials by allowing the learner to request additional support when it is needed. The feature has two subcomponents. The first processes text-based corpora to generate vocabulary lists that are relevant to a learner-identified need. The second retrieves images that can communicate the meaning of vocabulary.

7.1 Generating Appropriate Materials: Vocabulary Lists to Support Communication

Different approaches to automatically creating lists of vocabulary that are specific to a particular context or topic were developed and evaluated (Demmans Epp et al. 2012). These approaches apply the same class of algorithms or computational processes (i.e., information retrieval) that search engines use to find relevant pages. These algorithms were used to process web-based corpora to generate a list of words or phrases and were evaluated (N = 16) for their ability to support communication through a discourse completion study. Those that provided the shortest list of items that effectively supported communication were integrated into the system. Please see (Demmans Epp 2016; Demmans Epp et al. 2012) for details.

These computational approaches generated a minimally sufficient set of vocabulary, which was the first step in supporting emergent learner needs. The second was to provide additional scaffolding to support learner comprehension of the developed list. While the ability to retrieve the definitions of individual words in the list was added, this was not enough to support learners with lower levels of language proficiency, such as those in the above exploratory study. To better support these learners, automated methods of identifying appropriate visual scaffolds were explored.

7.2 Communicating the Meaning of Vocabulary Items

Four human-edited, open-source, web-based corpora (Table 3) were evaluated for their ability to provide images that represent a word or phrase’s meaning. The ability for images from these corpora to communicate the meaning of vocabulary entries was then evaluated in the hopes of determining which would best support learner comprehension of the automatically generated vocabulary lists.

Table 3 The evaluated image corpora
Full size table

At least half of those who helped to evaluate the corpora (N = 202) rated several sets of images, with 4,754 ratings performed over 879 vocabulary items. See (Demmans Epp 2016) for details.

These ratings revealed the corpora whose images best communicated the meaning of vocabulary items were drawing-based rather than photographic, which may be tied to people expecting a more precise representation and therefore applying higher standards to photographs than sketches. It may also be due to the simpler nature of sketches which tended to include fewer background details that could contribute to ambiguity or confusion.

It should be noted these top-performing corpora, some of which had even been designed to support learners, were merely sufficient. However, their use reduces the content creation burden that is common to learning systems and enables learners to receive support when they do not foresee their learning and communication needs. This need justifies integrating this content generation feature into the support tool so learners can request additional support from wherever they are when a need arises.

7.3 Obtaining On-Demand Support

Requests for on-demand support initiate a process where the system performs a series of operations to send support materials to learners as quickly as possible. The system first searches the existing vocabulary collections to see if there is one that is shared. When a collection exists, it is given to the learner immediately. In this situation, the learner only has to wait the length of time it takes for those materials to download before they can be used. When one does not exist, the automated approaches to generating these support materials are used, and they return the requested vocabulary list to learners. This requires a couple of minutes more than when a relevant collection of support materials already exists. However, once this has been done, these automatically created support materials are available for all learners, making it faster for others to get the same support.

While these programmatically generated support materials can help learners, they can also be imperfect. A similar challenge arises when learners create their own support materials since they may make mistakes. To address this limitation, learners can modify the content of any vocabulary item they have imported. This allows them to fix the problems they find. They can then share the fixed materials so other learners can benefit from their efforts.

8 Design Evaluation of a Hybrid AAC-MALL Tool

An evaluation of this newly developed system was conducted from a user-centered design perspective to make sure the system supported ELLs’ communication and learning activities. This final stage was needed to ensure the system met its original goals before it could be evaluated for its effects on ELLs’ ability to achieve their communication goals or learn English. This evaluation, therefore, focuses on understanding how ELLs can use this tool to support their communication and language-learning activities.

This time, the English proficiency of all of the learners was high enough for them to gain admission to a Canadian postsecondary institution: their test scores were equivalent to or greater than an IELTS 6.0. They also shared many demographic attributes (see Table 4), with all of them pursuing postsecondary programs at English language institutions.

Table 4 Participant demographics: ELL use of a hybrid MALL tool
Full size table

These ELLs reported using varied learning strategies that integrated general-purpose and dedicated language-learning technologies (Table 5). Their experiences foreground how dedicated MALL tools can be incorporated into ELLs’ existing learning strategies and expose occasions where using the developed tool extends the learning opportunities available to them.

Table 5 ELL technology use: hybrid MALL tool evaluation
Full size table

8.1 Language-Learning Strategies

Participants relied on courses to develop their knowledge, and everyone used text-based media to support their study activities: some reviewed or practiced grammar rules, while others used dictionaries to explore word meanings and improve their understanding of vocabulary (Table 5). Unlike those from the previous group, these learners pursued oral interaction to take advantage of languaging (Swain 2006), and they tried to improve their listening comprehension by watching TED talks.

8.2 Technology Use

Even though these learners used similar general-purpose technologies to those from the first study (Table 5), these ELLs exhibited greater comfort with technology. Learners increased their exposure to learning content by listening to their own English music and had integrated a diverse set of dictionary and translation tools into their communication and study habits. This included using Google search to verify word spellings (through its auto-suggest feature), Google image search to gist word meanings or Wikipedia articles to understand vocabulary.

These participants also used applications that were dedicated to supporting language learning. The applications they had used targeted learner pronunciation or grammar rather than the higher-level tasks with which many learners need assistance (Demmans Epp 2017).

8.3 Application Use

Like MyVoice, this application was predominantly used in private spaces. Learners developed their vocabulary knowledge (Fig. 7) by studying vocabulary entries; using the sentences to model word usage; checking word spellings through the search function; and training their pronunciation by listening to the text-to-speech version of words, sentences, or definitions; recording themselves; and comparing the two. Alda and Davi additionally used the application to help them complete their homework.

Fig. 7
figure 7

ELL use of the new tool to support their learning and communication goals

Full size image

Participants explained how images helped them to connect a familiar object or concept to its English label. Gil even admitted that he would only study materials that had images. Learners navigated the content by browsing and interacting with the synonyms that had been recommended to them. This behavior shows the potential usefulness of these types of subtle recommendation mechanisms. Participants also navigated the materials via the search feature. Learners would search through the available vocabulary or request new learning materials using the on-demand content features so they could find words that were of interest to them. They also used the on-demand content because they felt the default vocabulary was limited, and they expressed this feature provided meaningful collections of learning materials that occasionally included some noise. In one case, the term prenatal was included in the collection of vocabulary that was associated with a gym because it offered prenatal exercise classes. The male student could not understand why prenatal might be a word that is needed in that context, which demonstrates both a learning opportunity and an opportunity for adjusting learning materials based on the individual characteristics of learners rather than only using their behaviors and knowledge as a basis for informing adaptation.

Learners enjoyed the additional control that the new design afforded them, with Alda, Gil, Ana, and Davi choosing when to use the recordings or text-to-speech feature to listen to pronunciation models. Alda, Gil, Miao, and Davi also compared recordings they had made of themselves to the system provided pronunciation models. They felt this feature was helpful because it allowed for self-monitoring and assessment, which are essential to improving learning when people are trying to learn on their own.

While these types of audio features can support learning, they can also inhibit learner use of a mobile tool since using audio features can result in unwanted attention. Even with this potential social barrier, a subset of learners (Ya, Zhen, Miao, Ana, and Davi) used the application when they were in public spaces. These public spaces included the gym, a pub, a grocery store, their classroom, and a laboratory. Like in private settings, application use in public settings tended toward study-like activities. However, some learners felt comfortable using the application to support their communication. They would do this by using the text and images as prompts while trying to communicate. This prompting took the form of searching through images or words for the correct one and then using the information on the screen to remind them of the word that they needed or how it could be used in a sentence. For Zhen, this meant that she was able to buy the type of pumpkin she needed.

Unlike those from the first study, these ELLs were more likely to develop the provided materials by further categorizing existing vocabulary (Pio, Zhen, Miao, Ana, and Davi). They also shared learning materials with one another. These behaviors indicate that enabling the creation and editing of materials through the mobile application was of use to learners. The increased range of behaviors observed in this study indicate the merging of AAC-based approaches with ELL-specific scaffolding enabled these learners to use the tool to support a wider range of learning activities.

9 Summary

This design-based work shows the general potential for adaptive mobile learning solutions when they take the users’ needs into account rather than providing prescriptive learning activities. This is shown through the multiple ways in which learners chose to use the tool.

This early work expands on the more common study of tools that tend to support one specific learner need or the rehearsal of specific language skills. While the developed tool enables studying and the rehearsal of specific skills, it goes beyond these targets to support emergent learner needs including their ability to communicate with those in their surroundings. Providing this support was possible because a hybrid mobile tool was built using the tools that support the communication of clinical populations as inspiration.

The newly developed tool was created following user-centered design principles to ensure it would meet language-learner needs. This process included the development, evaluation, and integration of adaptable features that support ELLs’ emergent communication needs (N = 16) and their ability to understand materials that are generated through computational methods (N = 202). The separate evaluation of different components of the system allowed for the understanding of the limitations of each component so the appropriate modifications could be made to support learner needs. The final design evaluation included all of these components to ensure they were compatible outside of laboratory settings so that the language learners who must use English to survive could benefit from a complete system rather than a set of poorly integrated support features.

Both of the real-world evaluations revealed the potential usefulness of communication support tools for scaffolding language-learner activities, with the adaptive AAC-MALL hybrid tool supporting a greater range of ELL activities. Subsequent evaluations have shown the developed tool can support vocabulary learning and its use is associated with improvements in the communication of recent migrants (Demmans Epp 2016). However, the influence these tools have on ELL communication success or learning outcomes requires further study.

10 Using and Selecting Adaptive Apps to Support Learning

Based on learner experiences, it is appropriate to use this class of tools to support their cognition. In classroom settings, adaptive and other mobile tools can be used to provide local support to students at their desk, to prepare them for upcoming units through vocabulary review, or to expand their vocabulary through the recommendation of learning materials. In students’ everyday lives, these types of tools can be used to support communication by providing students with resources that enable them to get the help they need when they need it. When considered alongside the content recommendation feature, the use of such tools in students’ everyday lives holds the potential to connect what they are learning in the classroom with their lives outside of school. This connection could encourage the transfer of knowledge and help them to see the relevance of what they are learning, which could improve their motivation for learning.

Using highly structured tasks is recommended when first integrating these tools. For example, have students complete worksheets to prepare them for upcoming classroom activities where specialized or new vocabulary is required. This can be done across domains, where the pre-review of vocabulary could help students to access and understand the information presented in a physics course. The types of activities being used can then be adjusted to gradually increase learner autonomy where they use the tool to support more self-directed study activities or projects. These projects could include documenting sample collection from a local stream for science or biology courses.

When selecting tools, it is important to consider the level of alignment among the tools’ features, its underlying adaptive theory, and the pedagogical methods that are to be used in the classroom. Generally, a reasonable amount of alignment would be desired, but there may be cases when complementary approaches are wanted. For example, an app that uses a spaced-repetition approach might be valuable if considerable memorization is needed even though problem-based approaches are being used in the classroom. It is advisable to choose a tool or app that allows learners to log (e.g., type notes, record audio, or photograph) aspects of their learning experiences since this can allow them to reflect on their learning. It can also allow them to log learning opportunities for later study and exploration.

Another aspect of the adaptivity that should be considered is the amount of control learners can exercise. To determine whether an app may or may not be appropriate, it is worth asking who should decide what the learner does: the learner, the teacher, the app, or some combination of these. In a similar vein, it is recommended that one investigates how recommendation errors are handled: can the user or teacher override an adaptive feature when it gets something wrong and what are the consequences of errors in the adaptive reasoning process?

The answers to these and other questions about the appropriateness of any one mobile learning tool are rarely right or wrong. Rather they allow for someone to determine whether a tool can help meet the goals that have been set. Tools that can be used to help meet desired learning goals should be further analyzed to ensure they support both the experiences and outcomes that are desired. In short, it is important to know what evidence there is that any tool, whether adaptive or not, will support learner needs. That evidence can come from research exploring the use of the tool, the carefully documented development and evaluation of a tool, or a considered analysis of its features. Hopefully, this chapter has provided an example of the type of effort that should be invested in adaptive app development if these tools are to be used in educational settings as anything other than a supplement to support self-guided study.

11 Future Directions

So far there has been limited study of adaptive mobile tools for supporting learning. Within language learning, these tools have tended to restrict adaptivity to that dependent on spaced-repetition algorithms designed to aid memorization or simple mechanisms that are tied to the learner’s location. This chapter presents the development of a tool that goes beyond these simple adaptive approaches to supporting learning. With the advancement of mobile device capabilities, there is considerable opportunity for integrating the types of deep adaptation that have been studied within computer-assisted learning (specifically within intelligent tutoring systems) into mobile contexts. However, the more complex and inconsistent nature of the learning environment requires the development of additional mechanisms for enabling the system or learner to overcome limitations in the recommendations made by the tool.

In classroom settings, considerable work can be done with respect to learning activity design. There are several challenges that relate to ensuring similar learning outcomes and a similar quality of learning experience while using adaptive tools, especially ones learners can take with them or that adjust to the learner’s prior knowledge and context. In line with this is the need to develop tools or guidelines to aid instructors in evaluating potential apps. As a complement to these evaluation tools, additional tools could be developed to support curriculum planning so adaptive mobile tools can be incorporated while ensuring course and individual student goals are met.

12 Cross-References