1 Introduction

Blind and visually impaired people are valuable assets of our society. A substantial increase in eye-related diseases and reduction in vision are becoming a caring dimension for government and welfare organizations. The World Health Organization (WHO) reported that, globally, more than 39 million people are blind, while 285 million people are visually impaired [1, 2]. Blind people are using assistive technologies to perform several daily life activities. Among these, assistive technologies, ICT, and mobile-based assistive technologies are an emerging trend [3]. Smartphones have shown tremendous development in the domain of accessibility–inclusive–assistive technologies from the last few years [4, 5]. Smartphones and other touchscreen devices have tried to facilitate blind people in performing common activities with ease and independence. However, these devices are prone to a number of problems in user interactions, recalling activities, sequencing, organization of contents, and performing tasks in a specified sequence [2].

Blind people follow a pattern of operations in performing activities on touchscreen interfaces. They usually avoid such content, interface operations, and activities which are creating accessibility problems for them [5]. The screen reading applications and other accessibility services have considerably increased the usability of touchscreen interfaces for blind people [6]. Still, the magnitude of usability decreases when the interface is loaded with many user interface components and non-visual items on screen without any distinction or tactical clues [7, 8]. Furthermore, the multimodal interaction paradigm has several usability issues such as small keys, feedback in the text-to-speech system, and responding to voice commands in a noisy environment [9]. Besides, locating non-visual elements on the screen, organization of layouts in systematic order, placement of visual metaphor, objects, icons, and tasks in a consistent manner are some of the issues faced by the blind people contributing a frustrating user experience.

The usability of common mobile app interfaces on a smartphone is an emerging research avenue [10]. Currently, 2.8 million Android and 2.2 million iOS mobile applications [11] are available for sighted and blind people. However, most of the applications remained inaccessible for the blind people due to lack of sufficient accessibility support such as inconsistent and non-usable interface design.

A substantial number of emails are exchanged on a daily basis, and because of this, the volume and number of emails are continuously growing [12]. Email reading and replying are considered as primary email activities, while usage and calendar reminders are considered as secondary activities. Studies have shown that a significant amount of time is spent on performing primary email activities like composing, reading, and replying to emails. The mismanaged usage of email may produce frustration, which is not only a waste of time but also demotivates a user toward their work [13]. The agenda of universal design advocates in making technology accessible for all [5, 14]. Riemer-Reiss and Wacker suggested that technology should serve people with special needs according to their requirements and this can be accomplished through a collaborative process of design and assessment for adaptation/promoting a particular assistive device [15]. However, the existing touchscreen interfaces have numerous challenges, such as the non-existence of physical buttons, directional, and user interface controls. These challenges make it difficult to operate these devices and applications running on these devices [16, 17]. Many researchers have emphasized on the development of a user-adaptive paradigm for designing simple to use, accessible, and blind-friendly user interfaces based on the guidelines of human–computer interaction (HCI) [16, 18,19,20].

The organization of email-related activities such as reading, composing, arranging, etc., requires a considerable amount of time and energy [21, 22]. Similarly, managing emails into folders, labels, and email visibility [13, 23] are still challenges for blind people. Besides these issues, spamming is another usability concern for the blind people. Spam email can carry viruses, trojans, and electronic worms resulting in a security threat [24]. The available solutions for spamming such as spam-filtering software, etc. are some effective techniques. However, such techniques may not yield good results in every scenario [25]. For example, a blind person has to scroll and listen to every single email in their inbox to reach to their desired email which is time-consuming and security prone activity. In such cases, a person may confront a spam email, which may compromise his/her security and privacy.

This paper presents TetraMail, a usable email client for blind people to provide better control over user interfaces. The proposed solution in this paper may help blind people in placing the items of interest or activities in a specific order, resulting in quick memorization of shortcuts, perception clues, and placements for different email tasks. Every task and activity has been represented with states enabling navigation among the interface layouts easily and effectively. The flow of actions is expressed in state-transition diagram (STD) portraying the flow of activities and their associated actions. The TetraMail email client is evaluated by 38 blind people by performing 14 tasks. Results showed an improved user experience, task completion accuracy, and less navigational loss. In addition, the TetraMail email client is compared with the existing email clients such as Thunderbird and Gmail leveraging a significant correlation. The work presented in this paper may help researchers and practitioners for augmenting real-world problems for blind people using touchscreen user interfaces.

This paper is organized into seven sections. Section 2 presents related work on the current email systems for blind people. Section 3 describes issues and challenges pertaining to smartphone interface usability. Section 4 is related to the system design and architecture of the proposed TetraMail. Section 5 describes the evaluation and results. Section 6 reflects the discussion on results and presents a comparison with the existing email clients. Section 7 presents the conclusion and future work. References are enumerated at the end.

2 Related work

Related work is presenting an insight into accessibility, usability, user interaction, content organization, navigational flow, and effectively operating non-visual items on touchscreen interfaces. The issues and challenges pertaining to interaction with touchscreen devices using mobile applications for the blind and visually impaired people are highlighted. Universal usability encompasses both accessibility and usability. Accessibility is a primary though not sufficient requirement for achieving universal usability [26]. However, a number of considerations should be accommodated regarding interface design. The usability of touchscreen interfaces is confronted with several problems such as inadequate screen size, complex input layouts, inconsistency in UI design, and complex interactions patterns. Similarly, email clients are also dependent on several usability factors such as screen layouts, user interface components, content organization, and navigational workflow among others. The advent of multimodal interaction paradigms and user experience strategies has created a need for accessibility–inclusive user interfaces.

Azenkot and Lee [3] presented a study to explore the possibility of speech-based text-entry techniques on iPhone and Android devices. They surveyed several parameters such as frequency, the context of use, and impact of technology on people with and without visual impairment. The results depict that input through speech was approximately five times faster than text entry through touch, while 80.3% of user time was spent on text editing and correction.

The researchers have illustrated recommendations for browsing, email systems [27], management of touchscreen artifacts [28], and how effectively blind people can locate an object of interest on touchscreen interfaces [29]. The work presented in [27] has proposed an Android-based voice-operated email system for blind people. They used a text-to-speech synthesizer for converting plain text into synthesized speech, predicting intonation, and capturing stress intake during plain-text entry [27]. The mail synchronization agent retrieves and synchronizes emails from the database server, which is available on android application for visually impaired and blind people. Similarly, Shoba et al. [30], have presented a novel idea on designing an interactive email system for visually impaired people. They used fingerprint mechanism for authentication, a keyboard for input, and interactive voice response for output. Interactive voice recording (IVR) allows users to interact with an email host system via system keyboard, followed by instructions from IVR dialog. Based on IVR commands, the interface sub-components presented the representation of several interactions’ modalities.

The usability and accessibility of familiar gestures like tapping, and swiping, etc., are relying on several dimensions such as length, area, duration, speed, shape, size, and gesture recognition [31, 32]. Romano et al. [33] conducted a preliminary investigation to understand how blind people perform touch and motion gestures in daily life tasks on smartphones. The results illustrate that factors such as physicality, user background, and mobile context of selected tasks have a high influence on several gesture operations. Buzzi et al. [31] presented a novel system for capturing touch-based gestures on 36 visually impaired people using mobile devices. They analyzed the performance of touch-based gestures for the blind people. The main difficulty reported in this study was the size and shape of the different types of gestures. Furthermore, the size of the touchscreen has a significant influence on using usable gestures during the exploration of touchscreen interfaces. During the experiment, the subjects preferred gestures of short, rounded shape, and single-stroke impression in performing several operations on touch-based interfaces. On the same pattern, Macro et al. [33] conducted a study to investigate how blind people perform touch and motion gestures for performing their daily life tasks on mobile phones. The results are depicting that these gestures are influenced by various factors such as user background, and trends in assigning a particular action and behaviors. Biswas and Robinson [34] have developed a simulator for identifying repetitive tasks in the interface layouts. The simulator is evaluated on visually and mobility-impaired people. Kortum and Sorber [35] measured the usability of applications for mobile phones and tablets through the system usability scale. They have collected responses in four experiments by rating the usability of ten applications of frequent usage on 3575 users. The average usability rating was 77.7% picked from the highest and lowest rated apps. They concluded that the usability of applications operated on a smartphone is higher than on Tablets.

Kane et al. 2008 [2] highlighted issues on positional awareness of objects and non-visual items on touchscreen interfaces. The utility of smartphone-based gestures using screen corners, edges, and multi-touch operations is compared for sighted and blind people. In addition, Wentz and Hochheiser [36] have surveyed 129 blind people on the usability of email applications in performing several activities on different email clients. They highlighted several challenges in performing email-related activities on desktop and web-based email applications [37]. Furthermore, they developed an efficient and usable email system for desktop computers and smartphones for sending/receiving voice messages in their native language (Indian language). The solution was dependent on controlling mouse buttons by associating several functions with mouse clicks. The left mouse click was used for composing email; the middle mouse button was for sending an email message, whereas email list was scrolled through scroll button of a mouse. The mobile version of this application was also designed; however, instead of a mouse click, they divided the smartphone screen into two sections, i.e., one for sending and other for composing an email. The top section was used for composing email, whereas the bottom section was used for inbox, and similarly, the screen was divided into onward activities and sub-activities.

Mahmud and Martens [38] have proposed an iterative interface for people with communication disorders called “Amail”. The most promising feature of this email client was its support for managing emails for people with aphasia. This design supports text-to-speech conversation, vocabulary customization, and locating email address in the contacts list. The system has also reduced the number of steps required for composing an email message. It also provides an opportunity to customize the interface, dictionary, and address book for encountering issues in composing emails. ShrutiDrishi [39] proposed a web-browsing framework integrating features of an automatic speech recognizer and a text-to-speech system to access Internet and emails for blind people. However, this application was developed as part of a research prototype and unavailable for public/commercial use. Unitha and Kalyani [40] have developed the text-to-speech voicemail system which converts a local Indian language “Telugu” through text-to-speech. Using this application, a blind user can send messages from one voicemail application to another and can store their emails permanently. Similarly, Wentz and Lazar [41], have evaluated seven common email applications on 15 blind people. They used Microsoft Outlook 2007, Outlook Express, Thunderbird, Outlook Web Access Light, Gmail, Yahoo, and Hotmail for their evaluation. Three-to-five blind people have evaluated each of the email applications. The participants tested 17 tasks in each application and marked the completion of each task in a percentage of the time. The tasks were related to composing an email, reading emails, adding/deleting email, settings, and folders management. The results derived from this study showed that among all tested email applications, outlook express was ranked 91% out of 100% in all 17 tasks and hence considered the best for blind people. Khan et al. [42] proposed BlindSense, a blind-friendly universal user interface model by adapting user interfaces based on user preferences, context of use, and device usage patterns. They accommodated common applications in manageable logical partitions for effective exploration of non-visual items on a smartphone.

To this end, for the facilitation of blind people in effective usage of the email client, a thorough revision in the interface design is required to improve the user experience in performing email-related activities with ease, independence, and accuracy. Based on the literature, we have come up with a number of issues and opportunities for designing a usable email client. Section 3 describes the issues and opportunities in the existing email clients.

3 Issues and opportunities in the existing email clients

In the past, most of the research work has been carried out in operating touch-based interfaces for blind users. The emergence of touchscreen interfaces has opened new vistas for accessibility-inclusive interfaces for the blind and visually impaired. Blind people confront a number of issues such as placement and selection of non-visual items, layout, search, user interface adaptation, lack of logical order of navigational items and menus, learnability and discoverability, and inadequate mapping of haptic feedback [5, 42]. The issues and challenges in the usability and accessibility of touchscreen interfaces are discussed in the sub-section that follows.

3.1 Usability issues in interface selection for blind users

The advent of the smartphone has a profound impact on the way which people communicate. Mobile assistance helps blind people to reading emails, booking appointments, placing phone calls, sending/receiving messages, and to checking diaries. These applications and services are usually operated by the blind people through accessibility services such as text-to-speech system. The screen readers are necessary tools for blind people to read screen content and navigate the mobile device accordingly. These screen readers read the visual content or layout of an application and respond to the user in the form of voice instruction. These types of applications help a blind person; however, they are confronted with inherited issues like the noisy environment and privacy. The noisy environment badly affects the accuracy of speech recognition in identifying and responding to a set of commands for people with special needs in the outdoor environment [43]. Similarly, Braille devices are serving as alternative interaction devices for blind people. However, Braille devices bear numerous challenges such as cost-prohibitive and literacy constraints. Blind people avoid accessing problematic content [5]. The level of frustration is intensified when they encounter poorly labeled links, forms, and missing text for graphics. The issues in downloading attachments also contribute to frustration [44].

3.2 Potential email consternations for blind users

The emerging ICT-based aids have numerous accessibility and usability concerns. The reason is that most of the ICT-based applications are developed for common users, and thus, blind people remained unprivileged [45]. Furthermore, the non-compliance with accessibility guidelines in the design of a user interface also leads to usability concerns for blind people [46]. The usability concerns in managing and organizing emails for blind people are outlined below:

Email organization takes considerable time and energy for reading, arranging, and organizing a huge volume of emails [21, 22]. Similarly, managing emails in folders, labels, and email visibility approach [13, 23] are some challenges that are yet to be explored.

Spam email is another usability and security concern for blind people. Normally, a blind person has to scroll every unread email in their inbox. Spam email can carry viruses and trojans that may damage the device or data of a blind person. In most cases, spams are common carriers for viruses, trojans, and electronic worms posing a security threat [24]. The available solutions for spamming such as spam-filtering utilities are effective in many cases, but not in all cases [25].

Identification and responding to CAPTCHAs for blind people are a difficult job [47]. Normally, CAPTCHAs are representing images and number, characters, or combination of both. Identifying and responding visual pattern of CAPTCHAs are difficult to recognize and respond [48].

Adaptation of user interfaces is based on user preferences, device logging patterns, and context of use. The user-centric user interface paradigm needs considerable revision [5, 28, 42].

Provision of flexibility in content presentation and emphasis over better control on contents, navigation, and access to information in the way that the blind people want.

3.3 Specific needs of blind users

The design of accessibility–inclusive interfaces needs to accommodate the preferences of blind people. A number of concerns emerge from the use of existing email clients. We have identified a number of limitations from the literature on usability analysis of these email clients. We argue that these points may be considered before the design of blind-friendly user interface design. These include the following:

  • low-cost, cost-effective solutions should be provided to the blind people. The existing solutions are either costly, or cost is associated with usage. Furthermore, commercially available assistive applications are associated with licensing cost.

  • The usability of mobile applications is gradually degraded due to non-availability of tactical and directional controls. The placement of non-visual items on the screen, finding adjacent user interface controls, and inadequate mapping of haptic feedback is contributing to inferior user experience and degraded usability. Besides congested user interface components, cluttered audio-based components, poorly constructive alternative text for images, and complex graphical navigation menu items are also posed to several problems for the blind people.

  • The development of an automatic speech-recognizer system (ASR) is still in infancy stage. Operating mobile applications through ASR in a noisy environment may lead to performance implications and accuracy constraints. Thus, commands generated from ASR may be erroneous and may mislead a blind person in outdoor environment.

  • There are more than 8000 different languages in the world [49] having diversified semantics and syntax. The available solutions are providing coverage to a limited set of languages. Every language has a set of alphabet, features, phonetics, and pronunciation. Thus, applications and technologies have limited choice to adopt features of local and regional languages.

  • The available email solutions are designed for a particular platform and technology, thus arising the need for cross-platform support. These constraints still exclude blind people from the potential of mobile technologies on several platforms, technologies, and operating environments.

  • Blind people follow a logical sequence of actions and activities trying not to miss their path at any stage.

  • A simplified interface should be designed by following easy steps or sequences of commands to perform several activities and tasks.

  • Enabling forward and backward navigation should be easy to learn, while subnavigational items should be made consistent. In addition, advanced application features should have the ability to use standard notations and commands for saving, locating, retrieving, reading, and sending emails. This will provide an opportunity to define/customize user interfaces based on user preferences, and level of understanding and ease.

Based on the limitations discussed in the literature, we propose TetraMail, a usable and cross-platform email client for blind people. We aim to provide a light-weighted alternative to email interface based on blind user’s requirements of accessibility and usability. The proposed solution is described in Sect. 4.

4 TetraMail: a usable email client for blind people

The TetraMail email client has been named after Cavefish or blind fish, a Mexican tetra fish, or blind cavefish, which is a freshwater fish. This fish has no eyes to see and is thus considered as blind. The motivation behind the design of this interface was inspired from the natural aqua ecosystem and the way that the blind fish perform their activities in the deep sea.

We aimed to design a blind-friendly email client to improve the user experience. This may enable blind people to experience a better understanding and enhance the memorizing of a sequence of activities. The consistency in the navigational flow was incorporated into the user interface design by making the sequence of commands memorable and easy to learn for repetitive tasks. The design rationale of the TetraMail email client is based on the principles of accessibility–inclusiveness, usability, and user interaction [10, 45, 50,51,52].

4.1 Framework for TetraMail client

A number of techniques for evaluating the usability of assistive aids are developed in the area of HCI including observations, empirical study, inspections, and modeling or simulations [53]. Interface usability requirements need to be addressed in the design specification and should be compliant with design for all principles and standard HCI models. The HCI models are widely accepted models by researchers, interface designers, and usability experts in modeling an interface development for a variety of users, devices, and platforms. These models include task, domain, dialog, presentation, platform, and user model [54]. The holistic view of the model is depicted in Fig. 1.

Fig. 1
figure 1

User interface modeling in the context of HCI [55]

Full size image

The task model provides a structural representation of activities of an application performed by the blind people. The domain model describes the sequence of interaction and has implemented in the form of screen flipping and changing states. The dialog model describes the interaction between different objects that articulates the interface. The presentation model describes several computational components that render the interface, while the user model describes the characteristics of the user including user profiling, user modeling, etc. This model provides a wide-spectrum coverage for a number of requirements to be accommodated.

Based on the above HCI model, we have developed a framework for an email client for blind people called TetraMail. The interface of TetraMail is designed in such a way that a blind user can easily use it without in-depth knowledge of operating touchscreen interfaces. The design rationale is based on consistency, persistency, reducing semantic loss, enhancing memorization of activities, and actions performed. Consistency and persistency are the key parameters for memorization and quick learning. The consistency is achieved by representing the state of each activity and their corresponding actions in the form of a state-transition diagram (STD). The STD contains three symbols including node, arc, and operation. The Node is represented by a circle representing a particular state of email activities. The arch has devoted to the arrow connecting the activities and actions, and flow of information. The state transformation is represented by arrows. Small squares depict that operations correspond to an integer value. Whenever a specific arc is traversed, an action associated with a transition is to be performed. The TetraMail interface is represented in simple STD, as shown in Figs. 2, 3, 4, and 5. The system is initiated with start state, where it waits for the blind user to initiate an interface activity such as selection, opening, closing, etc. The interface preserves the state of individual activity and their corresponding actions. The structure of email activities is depicted in Fig. 6. The sequence of activities is designed in such a way that it is easy to remember and can be executed in less time. Most frequent actions and activities are kept separate in favorite section and will be populated based on the application usage logs and user’s preferences.

Fig. 2
figure 2

State-transition diagram for email compose activity

Full size image
Fig. 3
figure 3

State-transition diagram for TetraMail

Full size image
Fig. 4
figure 4

Inbox activity

Full size image
Fig. 5
figure 5

Read email activity

Full size image
Fig. 6
figure 6

Structure of TetraMail activities and actions

Full size image

4.2 TetraMail architecture

This architecture is designed on the layer-based approach, whereas all sub-components work closely in a seamless fashion. The interface is designed in such a way that it may improve the learning curve over the time. Furthermore, it may reduce memorization efforts in considerably less time. The degree of understanding and semantic loss is the key parameter for determining the usability of the proposed solution. Every interface screen is divided into manageable partitions by accommodating user activities and tasks accordingly. TetraMail architecture is composed of the following components as depicted in Fig. 7.

Fig. 7
figure 7

High-level architecture of TetraMail

Full size image

4.2.1 Startup module: interface selection

The first interaction of a blind user with TetraMail occurred through the startup model. The user interacts with this screen by touching the TetraMail icon on a smartphone screen. When a user taps a section of the email, five options will be made available including inbox, set up email, compose an email, and general setting. The center of the screen is utilized for five frequently used shortcut features like home, previous state, current state, and forward state, as illustrated in Fig. 9a. Dragging a finger to the left side will activate Go-back feature; dragging a finger to the right side will initiate Go-to-forward; dragging a finger to the top side will be used for Go-home; dragging a finger to the bottom side will be used for reading of the current state of the application, while long tap on the center section will be used for Sign-out from the currently active email account. The STD represented in Figs. 3 and 8 demonstrates different states of this module. The start state initiates the application based on user input activities such as inbox, compose, setting, setup, log out, and error state may be initiated. In case of failure, Error-State will be automatically initiated and will record logging error statement in a text file stored on the smartphone. All these states are stored in the system, and hence, the user has the opportunity to memorize its current state. Furthermore, a user can navigate backward and forward easily.

Fig. 8
figure 8

STD activity for startup module

Full size image
Fig. 9
figure 9

a Application screen. b Application interface for inbox. c Application screen shot for inbox

Full size image

4.2.2 Inbox module: organization of emails

The interface in this module is sub-divided into equally manageable partitions based on our earlier work on universal user interfaces [5, 42]. There are four more sub-components that will be available to the blind users including reading email (visited email), unread email, outbox, and search email. When a user taps read email (visited email) section, again, there will four more sub-components consisting of listing, deletion, reading, and scrolling to up/down (read by sequence number). The text-to-speech system will be used to read aloud the email contents. After that, the user can also opt for other options such as reply, reply to all, and forward and delete email. In this case, the keypad will be activated for the writing of recipient ID and the same procedure will be adopted for a tap in the unread email section as well. When a user taps on search email section, the control will move to the keypad (for text entry of search keywords) and search results of emails will be displayed. The user can control reading emails by scrolling up and down. STD of this operation is illustrated in Fig. 10, whereas application interface is shown in Fig. 9b.

Fig. 10
figure 10

STD activity for inbox

Full size image

4.2.3 Email composer module: composing a new email

The email composer module is comprising of four sub-components including composing a new email, attachment, subject, contents, and another section called TO, CC, and BCC. When a user taps this section, it will read the screen on the following sequence: First TO, Second CC, and Third BCC. Then, control will be shifted to sub-components of the compose email section. After completion of all these processes, the email can be sent by long tapping on the center section. The STD and application interface is presented in Figs. 9c and 11, respectively.

Fig. 11
figure 11

STD activity for compose email

Full size image

4.2.4 Management module: email management and value-added features

The management module displays four sub-components, i.e., folder management, contacts management, signature management, and email account. The folder management is the first component having four sub-components, i.e., new folder, deleting folder, renaming folder, and moving folder. When the user taps on the new folder, the control will be moved to the keypad, and a process cycle will be completed after writing the name for the folder. Similarly, when a user initiates a delete folder option, a list of all folders will be displayed, and by pressing long touch on each folder will delete a folder accordingly. When a user taps the section for renaming a folder, there will be again an option for listing folders, and the user will have to press the long touch resulting in shifting of keypad control for completing the process for renaming folders. Similarly, the same will happen when a user taps on delete contact option. The contact(s) can be the first search through text-entry using the keypad, which will return a list of contacts, by pressing on the desirable contact; the delete option will be activated accordingly. The STD activity and application screenshots are illustrated in Figs. 12 and 13, respectively, for the above process.

Fig. 12
figure 12

STD activity for setting and folder management

Full size image
Fig. 13
figure 13

Application screenshot for folder management and inbox

Full size image

4.2.5 Feedback module: mapping haptic feedback

To use this email client effectively, the user interfaces have been designed in such a way that blind users can perform their required task in a minimal number of operations to send and receive an email. A number of haptic responses are incorporated in the touch-based responses and voice interaction for checking the email and replying email via voice. The process of returning to “Home” option is available in a consistent manner in the overall design enabling blind people to return to the starting point in case of a complex menu structure or content listings or getting lost in an application or menu hierarchy. Similarly, moving backward and forward navigation is also made available through voice interfaces. Specific patterns are designed for common navigational activities like Back, Home, Next, Up and Down, and so on. The application handles control of content through these commands duly associated with temporal parameters. For instance, in case of moving back, the previous state of the screen will be presented after 2 s and similarly for other items, as well. These settings can be modified from the configuration option of the application. Scrolling takes place through voice UP command and reads out the content of the screen until the end. The application provides a special vibration (long vibration) imitating the end of the screen. Consequently, the user again issues a command for the UP or Down navigation. In the configuration settings, the options for continuous reading (for instance in the reading email) are also available, though it depends on user preferences. The scrolling page options are designed based explicitly on finger model and finger placement usage patterns of the blind user providing a high degree of consistency and ease.

5 Results and evaluation

The primary objective of the study was to provide a framework and prototype of an email client for blind people that can be effectively used on smartphones. The proposed solution may help blind people in sending, receiving, organizing, and managing emails. The usability of the design has evaluated using standardized HCI usability and accessibility parameters. We have extended the HCI model in our interface and tested a number of parameters for evaluation. In addition, we have evaluated the timely completion of each task, the degree of accuracy during task completion, the level of easiness achieved duration task completion, and the degree of semantic loss in performing several email-related tasks. Our proposed solution was compared to other email clients such as Gmail and Thunderbird. The results reflect that TetraMail is a better alternative for blind users due to its consistent and blind-friendly interface design. The comparisons of evaluation of task are analyzed in Table 4.

5.1 Materials and methods

A random sample of 38 participants was selected from different institutions and universities of Pakistan and Afghanistan for testing the prototype implementation of TetraMail. The participants had different smartphone usage experiences. Some of them had more than 6 months, while others approximately 1 year. Most of the participants were university students and employees of different organizations. The details of the participants are presented in Table 1.

Table 1 Participant information
Full size table

The study was conducted individually; all participants were provided with the same type of pre-defined tasks. There were two parts of the study, i.e., in-wild capturing (through keystrokes) of responses and semi-structured questionnaire, interviews, and observations. We asked selected participants to perform 14 pre-defined tasks (three times by each user, 14 activities in three iterations). A total of 1696 keystrokes were captured including wrong and missed touches. The tasks were organized in different groups. The average duration exercised on each participant was about 66 min. Two researchers were directly involved in observing the execution of tasks performed to verify the level of understanding and ease of use. The smartphone models used by participants were Samsung S6, Qmobile A12, and iPhone 5. Formal written consent was obtained from the caregivers of these blind users; the institution review board has approved the consent form and procedure before the conduct of this study.

5.2 Evaluation criteria for the usability of TetraMail

The captured responses and keystroke data were collected through our specialized application which stores the following information: (1) number of keystrokes, (2) wrong touches, and (3) missed touches. The degree of easiness is measured by calculating perfect (accurate) touch performed in less time, etc. Similarly, the duration of the activity performed is measured through built-in stopwatch. We have used Keystroke–Level Model (KLM) [56], which identifies operations or activities and assign a timestamp value to each of them. The activities are tracked through KLM. These timestamp values are then added to the final time term as a task-execution time. The KLM was used for validating the responses captured from the questionnaire and interviews.

We have identified the accuracy and easiness in every task from a set of benchmark tasks. For example, for the first time, we let the blind people use the email client and then asked them to get ready for the evaluation [after getting clearance from the first author (study facilitator)]. The benchmark tasks used a “think-aloud” verbal protocol in the observation study only. The think-aloud mechanism was used during the observation section of the study, to verify whether the benchmark tasks meet the required level of satisfaction or otherwise. Thus, they verbalized their comments/remarks on the activity. The survey facilitators observed their flow of interaction and confirmed through verbal statements. We found the values of these parameters from the calculation of task completion time, accuracy, and ease of use during task completion of several tasks. The keystroke-level analysis is used in identifying and calculating the time required for accomplishing a particular task within an email interface. The task-execution time can be estimated with the sum of time needed for all operators, as shown in Eq. 1, Tk (Keystroke time), H (Homing), M (Mental preparing), and R (System response operator):

$${\text{Tk}}={\text{Tk}}+{\text{Th}}+{\text{Tm}}+{\text{Tr.}}$$
(1)

The following parameters were selected for the evaluation of the email applications by the blind users:

  • Time of task completion It is the amount of time required to complete an activity or operation. The time required for completion of the task has further two sub-parameters, i.e., percentage completion of task and time spent on the completion of a particular task;

  • Accuracy during task completion This parameter was used for measuring the accuracy of task completion (through observation and keystroke analysis);

  • The degree of easiness The ease of use during task execution has been measured, to evaluate the usability of the proposed interface. The ease of use can be measured through operating the interface in a user-friendly manner by measuring activities such as navigating between different interfaces. The measurement of this parameter is subjective and co-related to other parameters such as time, accuracy, and semantic loss;

  • Semantic loss The structure of interface should not be complicated to avoid confusion during task execution. The user should have a clear understanding and acknowledgment of his/her current state and next and previous activities. To measure the degree of semantic loss of an interface, we have used Eqs. 2, 3, and 4, whereas M is mean for task completion, A is the accuracy in task completion, E represents easiness in task completion, and SL stands for semantic loss:

$${T_{({\text{total}})}}=A\left( {{\text{task}}} \right)+E\left( {{\text{task}}} \right)+{\text{SL,}}$$
(2)
$${\text{Mean}}=T/3,$$
(3)
$${\text{SL}}=100 - {\text{mean.}}$$
(4)

If a task is completed at full (100%) with 100% accuracy and ease of use, then semantic loss will be 0%. Similarly, if an activity is not completed with 0% accuracy and 0% ease of use, then according to formula 0% + 0% + 0 = 0, 0/3 = mean (0), 100 − mean (0) = 100, there is 100% semantic loss while performing email interface tasks.

5.3 Results

We have captured 532 responses (38 participants performing 14 tasks) in three iterations, resulting in a total of 1596 captures. However, the system captured a total of 1690 responses out of which 1501 impressions were found accurate responses, whereas 189 responses were found some invalid responses. All iterations were not used in the final analysis. We reviewed all captures and iterations and marked down those captures responses which can be used for further analysis. The details of application-captured responses are analyzed in Table 2. Table 4 provides a detailed statistical evaluation of all the usability parameters in all three email applications (TetraMail, Thunderbird, and Gmail) analyzed in this section.

Table 2 Groupwise keystroke analysis
Full size table

The test statistics for the mean of task completion in all three groups (organization, management, and value addition features) on the three email applications were tested using single-factor ANOVA (F Test). ANOVA was used to measure the significant differences between results. The results show that there are significant differences (see the following ANOVA test results). The P values are less than Alpha (α) in most of the cases. Similarly, the calculated F values are higher than the F-crit values in most of the cases, and hence, we conclude that the population means are not equal. The low variation (variance) in TetraMail shows its usefulness. Table 3 presents the single-factor ANOVA for mean of task completion.

Table 3 Single-factor ANOVA for mean of task completion
Full size table

The variance of TetraMail is comparatively low, which indicates a positive sign for a blind person. The variances (σ2) of Gmail are high in most of the cases, which means that blind users faced difficulties in performing their desired tasks using Gmail on their smartphones. Similarly, we tested the values of mean for time using ANOVA test. Results depict that P value (0.009366) < 0.05 (α). In addition, F value is larger than F-critical value, which indicates that the differences in results are quite significant. Mean for time shows a huge variation in the values. This is because some tasks were simple for a blind person, while the difficulty levels of some tasks were considerably high.

The variance of the different email applications shows their usefulness. TetraMail presented low variance (σ2) among all, which shows that TetraMail is a comparatively more useful email client for blind users. Table 4 shows the variance, critical values, and P values of different parameters which we have used; for example, mean for time, accuracy in task completion, ease of use, and semantic loss were tested using single-factor ANOVA (F test). The P values of mean for time, accuracy in task completion, and easiness are less than 0.05 (α). Hence, we may conclude that there are significant differences in the results. In addition, the value of F values indicates the significant differences between results. In case of semantic loss, as shown in Table 4, P value > 0.05 and F value < F-crit. In this case, we may accept the null hypothesis and conclude that the means are equal. However, we observed a very low variation (variance σ2) for TetraMail in this category, which is a definite indicator for blind people and shows that TetraMail is better among all the available email clients.

Table 4 Variances, F values, and F-critical values
Full size table

We have evaluated the F-test values and P values of Mean for time and Semantic loss. The other two parameters, Accuracy-for-Task-completion and Easiness, have generated negligible P values and were consequently not included in the final analysis. Figures 14 and 15 represent the graph for the F test.

Fig. 14
figure 14

F distribution (mean for the time)

Full size image
Fig. 15
figure 15

F distribution (semantic loss)

Full size image

5.4 Analysis

The following analysis provides an insight into the factors influencing the email-related operations from the captured responses. All tasks are classified into three groups to ensure a consistent response from the blind people. The subsections below provide an in-depth analysis of email-related activities by group.

5.4.1 Group I: email organization

This group of activities contains mainly easy-to-operate activities for all participants. The reason behind the easiness in operations was the proper arrangement, organization, and state representation of these activities in our proposed design. Second, these tasks are also common activities in every email application. We also analyzed the collected responses from the observation and structured questionnaire and found that the error ratio (wrong touch, missed touch) in this group is 1.105 per user, whereas all participants have completed eight tasks in this groups achieving an average accuracy of 95% (SD = 5.65). Consistency in the organization of tasks achieved the degree of easiness up-to 92% (SD = 6.24), whereas the level of semantic loss was 5.25% (SD = 5.14). Semantics loss can be reduced significantly with the improvement of haptic and gesture interactions as well as using the current-state-speak feature in the application. The statistical evaluation of this group is illustrated in Fig. 16.

Fig. 16
figure 16

Statistical evaluation of Group I

Full size image

5.4.2 Group II: email management

The email management-related tasks are reported as a moderately difficult task as compared to the email organization. The level of accuracy of tasks presented in this group was relatively less as compared to email organization (Group I), i.e., 93% (SD = 5.71) of accuracy achieved in four tasks in this group. Due to the complex nature of tasks, the degree of easiness in this group is also reported low, i.e., 84% (SD = 6.40) which also leads to increase the semantic loss of 8.5% (SD = 6.0277). In total, 423 responses were recorded as accurate, out of 456. The learning curve can be further improved in this group by integrating with usable touch-sensitive keypad layouts, as mostly the wrong responses are recorded in text entry and text manipulation. The statistical evaluation of this group is depicted in Fig. 17.

Fig. 17
figure 17

Statistical evaluation of Group II

Full size image

5.4.3 Group III: value-added features

The activities in this group consist of two supporting tasks related to email operations. Though the accuracy of these tasks is relatively low in the above two groups, i.e., 91% (SD = 5.65), as well as semantic loss, is increased by 9.5% (SD = 9.19). Out of 207 keystrokes, 20 keystrokes were recorded as missed touch and wrong touches. The statistical evaluation of this group is illustrated in Fig. 18.

Fig. 18
figure 18

Statistical evaluation of Group III

Full size image

5.5 Comparison with similar systems

We have compared our proposed solution with the existing email clients such as Gmail and Thunderbird. The tasks were performed using the above two email clients with Tetramail. The comparison results reflected that TetraMail is a better alternative for blind people due to its blind-friendly user interface design. The comparative evaluation of the existing email clients with our proposed email client is depicted in Table 4.

6 Discussion

The Tetramail email client was evaluated on the four aspects of usability and accessibility on 14 tasks adopted from a survey conducted by Wentz and Lazar [41]. First, most of the 14 tasks were completed; however, a few tasks remained uncomplete due to their complicated workflow; for example, attaching file and moving folders. Second, the accuracy of the tasks performed is estimated in terms of the time duration (in seconds). The third parameter is the ratio of accuracy in completing tasks.

In this analysis, we have represented standard deviation in performing an activity, the highest value of accuracy was 100%, and the low value was 84%. The tasks scored high accuracy mainly due to two factors: (1) larger size of screen division and (2) the activity having short processes. The fourth parameter determined the degree of easiness achieved while performing given task. This parameter is calculated by mean and standard deviation (SD) to find out the easiness of performing the activity during task completion. The participants have reported the most straightforward task was deleting an email (scoring 100% in easiness). Similarly, moving a folder was considered a difficult task, scoring 78%. The main reason behind the difficulty was its complex workflow process to accomplish. The last parameter is the semantic lose; this parameter is very vital for understanding the complexity of the interface as well as for helping capture user behavior in performing activities. This also involves user fatigue in operating a smartphone and particular user interface. The purpose of representing state-transition diagram was to represent different states of activities to minimize the lost path. The app has already provided flexibility as a feature that blind users can know at any time about their current state. We found out the semantic loss parameter from the calculations of values of accuracy and easiness, and concluded if the task is accurate and easy or otherwise.

After this review, we had 1501 valid responses and 95 responses of wrong touches and missed touches (out of which 14 responses were collected as overlapping touches). Each task was evaluated concerning time resulting in an effect on other parameters. On the set of captures and observation, we have performed a statistical analysis, as illustrated in Table 3.

7 Conclusion and future work

The majority of the existing email applications (also known as email clients) are designed for ordinary users and do not explicitly meet the requirements of blind people. Blind people have to find buttons, options, and links by touching and scanning the full screen, as most of the smartphones have no physical buttons to identify the position and location of the non-visual items on the screen. This makes the usage of an email as a challenging activity for blind people. The proposed solution “TetraMail” has addressed this problem by organizing the content of the screen in manageable partitions of five sections and re-arranging the activities of an email in these five sections. The proposed solution is tested through an empirical study. Results showed that this email client helps blind people to send and receive emails with comfort and ease. Furthermore, we observed that providing this simplified email client reduces the cognitive load in awareness of each button, links, and entry points in small size text boxes through small size keys of a QWERTY keyboard with no need for taking long tutorials or training. This consistent screen division enables blind users to memorize the flow of activities and thus reduce the possibility of getting lost in other options of the screen. We evaluated our proposed solution by comparing the performance and usability with the existing email applications/clients such as Thunderbird and Gmail on smartphones. Our preliminary results illustrate that the current solution is better than other available solutions due to the consistent organization of actions and activities, reduced cognitive overload, and improved haptic responses. Future work will be more focused on Internet browsing for blind people; we will fit the functions of Internet browsing in these five-section screen divisions. In addition to the above, the optimization of GUI layouts and elements will be considered in particular focusing on gesture control systems.