3.2.1 Models, Theories, and Practices of HCI

  • John Manning 

Human factors engineering (HFE) and human computer interaction (HCI) are multidisciplinary sciences that seek to optimize the interactions between humans and a given system (Holden et al. 2016). HCI began in the early 1980s as a blend of HFE with software engineering, with the intent of applying scientific principles to address real problems in the software development space (Carroll 2003). HCI assimilates cognitive, social, and behavioral sciences into its frameworks, and members of the HCI community reach far into a myriad of domains including computer science, cognitive psychology, anthropology, mathematics, and communication studies.

3.2.1.1 Models

Many conceptual models exist within HCI, which are utilized both in the design and in the evaluation of a given system. Figure 1-1 provides a visual representation of the common evaluation methods used in HCI. Techniques shown in the analytic approaches (left) are often synergistic with usability testing methods (right), and some degree of overlap does exist among the various categories of methods. For further information on HCI evaluation methods and usability testing, see Sect. 3.2.2.

FIGURE 7-1
figure 1

Classification of Human Computer Interaction (HCI) evaluation methods. Definitions: Goals, Operators, Methods, and Selectors (GOMS); Keystroke-Level Modeling (KLM); Task, User, Representation, and Function (TURF). Adapted from Kannampallil and Abraham (2015)

Full size image

Task Analyses

Task analyses are simple methods used to evaluate an existing system based on the actions that are performed and the motivations/decisions underlying them (Kannampallil and Abraham 2015). Two of the more common variations of task analyses used in biomedical informatics include hierarchical task analysis and cognitive task analysis. Hierarchical task analysis separates large goals into various tasks, sub-tasks, sub-sub-tasks, etc. as desired to achieve an appropriate level of detail (Kannampallil and Abraham 2015; Stanton 2006; Annett 2003). It has been used in the evaluation of medical devices (Chung et al. 2003), clinical workflows (Unertl et al. 2009), and medication errors (Lane et al. 2006). While no theoretical limit exists as to the number of subprocesses included, doing so may limit the overall algorithm’s utility and should be used with caution (Stanton 2006). By contrast, cognitive task analysis focuses more heavily on the internal perceptions and cognition that ultimately result in an observable action (Kannampallil and Abraham 2015; Schraagen et al. 2000). Cognitive task analyses have been utilized in healthcare to assess how primary care providers manage alert notifications (Hysong et al. 2010), and methods for cognitive task analyses frequently employ some of the inspection-based, observational, and feedback techniques also listed in the above figure (Cooke 1994).

Conceptual Models

While technically a task analysis as well as a conceptual model (Kannampallil and Abraham 2015), the Goals, Operators, Methods, and Selectors (GOMS) model is an important foundational model within HCI. The GOMS model separates tasks into smaller components to approximate how much time and effort may be required. At its most basic level, GOMS follows a user through their intended task (Goals/Subgoals); the actions performed to accomplish said goals (Operators); the order in which each action is taken (Methods); and the choice of one method over another similar one (Selectors). Many variations exist within the GOMS family, including the more simplified Keystroke-Level Modeling (KLM) tool , where operators focus more heavily around keyboard/mouse clicks and mouse movements (Card et al. 1980). Like many other HCI models, KLM also tracks the amount of cognitive time required to perform an action and the time required for the system to respond to the user.

The KLM model also takes into account the predictive model Fitts Law , which defines the shortest time interval required to acquire a target (Card et al. 1980; Fitts 1954). When clicking an object on a computer screen, Fitts Law can be applied based on the distance from the mouse to the target and the target’s size (Carol et al. 1978). Another predictive model utilized in HCI is the Hick-Hyman Law , which states that each new choice logarithmically adds time to human processing and selection (Hyman 1953; Hick 1951). This model can be employed to determine the number of objects that a menu should contain, or to guide the display of choices with colors/highlighting to help augment human information processing time (Holden et al. 2016; Kannampallil and Abraham 2015).

3.2.1.2 Theories

As with HCI’s models, cognitive theories played an integral role in shaping HCI techniques (Kaufman et al. 2015). These models typically fall within one of the three basic categories listed in Fig. 7-2.

FIGURE 7-2
figure 2

Classification of Human Computer Interaction (HCI) cognitive theories. Adapted from Kaufman et al. (2015)

Full size image

Human Information Processing

Human Information Processing theories describe in various detail how we as humans absorb, process, and respond to our external environment. Inputs from our senses are distinguished from cognitive methods of processing and from internal storage in short- and long-term memory. One of the foundational theories within Human Information Processing is Norman’s Theory of Action , which separates each mental activity cycle into seven inter-related stages (Norman 1986). Beginning with a Goal (e.g. to order a medication), the three stages within the “Gulf of Execution” (Intent to order a medication, Action sequence of steps, and Execution) are performed internally first, and then the action is actually performed in the real world. Once performed, the steps within the “Gulf of Evaluation” (Perceive the state of the world, Interpret this perception, Evaluate the interpretations) are employed to determine if the goal has successfully been achieved or not.

External Cognition

Just as we interact with the world, so too can the world affect our cognitive states. External cognition is the way in which we employ parts of our external environment to help guide and augment our cognitive behavior (Zhang 1997). These parts—also known as external representations—may exist in the form of memory aids, diagrams, symbols, pictures, or some other abstraction. The largest requirement for something to be labeled as an external representation is that its use must change the cognitive task at hand in some way (Zhang 1997). Some basic examples of external representations are the use of pen and paper to assist with complex math problems; the use of hand-drawn sketches to assist with brainstorming (Visser 2006); and the graphical visualization of a patient’s lab results to understand what trends have occurred over time (Kaufman et al. 2015).

Distributed Cognition

Building off of the groundwork set by external cognition, Distributed Cognition (DCog) shifts the focus from a single person’s cognitive model to multiple people in a “cognitive system” that are collaborating to accomplish a shared goal (Hutchins 1995). Examples include a crew of people working together to operate a ship (Hutchins 1995) and a team of healthcare providers working together to care for a patient (Kaufman et al. 2015). In an attempt to provide an implementable framework for the application of DCog principles in healthcare, Furniss et al. (2015) describe an implementable framework called the Distributed Cognition for Teamwork (DiCoT) which was used to assess a single medical device (glucometer) during its implementation within a health system.

3.2.1.3 Practices

Each of the models and theories mentioned above build upon one another in complexity to help understand how HCI can be used to analyze and improve upon the interactions between humans and technology. As best described by Bederson and Shneiderman (2003), the general reasoning behind learning these principles and their contextual relevance is to:

  • describe plainly and regularly for collaboration;

  • explain and educate;

  • predict performance in current state/ideal state to maximize gains;

  • prescribe guidelines and best practices while warning about concerns; and

  • generate novel ideas.

With such objectives in mind , many of the discussions throughout the rest of this chapter build—at least in part—upon one or more of the foundational principles listed above. Through use of rigorous user-centered design techniques and adoption of best practice designs into interface standards, the hope is to continue to push HCI forwards at a pace that can match and help manage the rapid technology gains seen today.

3.2.2 HCI Evaluation, Usability Testing, Study Design and Methods

  • Laura Kneale 

Human computer interaction (HCI) is increasingly important in the development and deployment of health information systems. HCI is the “study of interaction between people (users) and hardware, software, websites, and mobile devices” (United States Department of Health and Human Services 2017 ). Evaluating HCI influences the design of a system, maximizes the impact the system has on the intended user population, and ensures that a system meets organizational goals. A large component of HCI evaluation is usability testing. Usability is “the effectiveness, efficiency, and satisfaction in which the intended users can achieve their tasks in the intended context of product use” (Middleton et al. 2013 ).

Recent studies have shown that usability is a significant problem in many health information systems. These problems cause frequent user errors that contribute to a range of undesirable outcomes including adverse patient events and low user satisfaction. Lower user satisfaction has been linked to slow adoption of important technology, increased costs associated with training and redevelopment work, and individuals developing complex workarounds to accomplish common tasks (United States Department of Health and Human Services 2017 ; Middleton et al. 2013 ).

For example, one famous set of studies evaluated the implementation of two computerized physician order entry (CPOE) modules in two different pediatric intensive care units (PICUs) . Although the CPOE module was built and installed by the same vendor, the two PICUs experienced vastly different outcomes. Using a pre/post study design, the first PICU that implemented the CPOE module observed a significant increase in mortality following the implementation. A study of the installation was conducted to determine potential causes, and found many issues with the initial design of the CPOE module. Usability problems that contributed to the undesired outcome included the inability to have critical orders available prior to patient arrival on the floor, the increase in provider time it took to prescribe medications, and the incompatibility between the CPOE module and the existing pharmacy systems (Holden and Karsh 2010 ). The other PICU had the opportunity to learn from the first hospital’s implementation, and also evaluated mortality following their CPOE deployment. This second pre/post study found a non-significant drop in mortality rates after implementation. In contrast to the first implementation, the second PICU took extra steps to evaluate the usability of the final design, involve users in the design process, and thoroughly test the system prior to go-live (Han et al. 2005 ).

Because of studies such as the ones described above, the American Medical Informatics Association (AMIA) advocates developing a greater understanding of the importance of usability among its community membership. AMIA recommends formal usability testing during system development and prior to implementation. In fact, in 2013 AMIA released a list of 14 usability principles that all electronic health records should adhere, to in order to ensure that the system follows best usability practices, and the design of the system minimizes adverse events (United States Department of Health and Human Services 2017 ).

Human-computer interaction and usability are often associated with the concept of “ user-centered design .” User-centered design is the process in which the users’ goals, motivations, and environment are considered throughout the design and development phases (Del Beccaro et al. 2006 ). Usability testing can be used to ensure that the assumptions that the system developers made during the design of a system are consistent with the requirements of the users. Figure 7-3 shows one framework for implementing user-centered design during a system development process (Del Beccaro et al. 2006 ). User-centered design is currently considered to be best practice when designing new health information systems.

FIGURE 7-3
figure 3

User-centered design process as modeled on Usability.gov (United States Department of Health and Human Services 2017)

Full size image

3.2.2.1 Types of Usability Testing

Usability testing can occur at any stage of system development. Formative usability evaluations are conducted during the development of the system. Formative usability studies provide developers with early insights into user reactions to design decisions, and help to identify usability problems before the design has been finalized. Because the main goal of these evaluations is to understand user opinions, formative usability evaluations can be conducted on low-fidelity prototypes (including paper prototypes) (Usability Professionals Association 2010 ).

In contrast, summative usability evaluations are conducted after the system has been developed, but before the system is implemented. Summative evaluations are usually the final test to ensure that the system meets the project’s usability benchmarks, goals, and user satisfaction standards. Summative evaluations may be helpful in finding usability issues, such as compatibility with existing workflows and technology, which may only be apparent when evaluating the entire system. In addition, summative evaluations can be used to compare usability of a new system to a benchmark, such as a gold standard or previous system (Usability Professionals Association 2010 ).

Because of the iterative nature of user-centered design, both formative and summative usability evaluations should be completed during the design and evaluation of a clinical information system. Formative evaluations are most helpful in gathering early user opinions, and ensuring that early decisions made by the development team are consistent with user goals and motivations. Summative evaluations, on the other hand, are the last assessment to ensure that all user requirements have been considered and the entire system works as intended.

3.2.2.2 Locations for Usability Testing

Usability testing can be conducted in a laboratory setting or in the user natural environment. Today a usability laboratory may include a wide range of setups. Traditionally, usability testing was conducted in a laboratory that had two rooms: a room where the usability participants conducted the test and an observation room. More recently, usability laboratories often contain sophisticated technology that automatically collects data on the participant as he or she conducts the usability test. These technologies can track participant’s eye movements, calculate time between participant actions, and log user clicks. Conducting usability tests in laboratory environments can be particularly useful if the organization wants to collect a particular type of data on participants using usability software, if the system is expensive or includes a large footprint, and/or if the data in the system is sensitive or proprietary (United States Department of Health and Human Services 2017 ).

Usability testing can also be conducted in the user’s natural environment. This means that the user is testing the system in the same environment that he or she will use the final system. Conducting tests with the user in their natural environment may provide insight into how the new system will integrate with existing tasks and systems. In addition, this method eliminates the need to have a central place to administer the tests, which may be a resource burden on the organization or reduce the organization’s ability to recruit diverse study participants (United States Department of Health and Human Services 2017 ).

Often, depending on the needs of the organization, systems are tested in both laboratory and natural environments throughout the development process. Each location is associated with unique benefits and challenges, and the organization’s goals, budget, and user constraints should be considered before deciding between potential locations.

3.2.2.3 Participants in a Usability Test

There are two groups of people who are needed to conduct a usability evaluation. The first group is the team that develops the usability testing conditions, creates the testing protocol, and administers the study procedures. This group may consist of designers , human-computer interaction experts, domain experts , and developers. The decisions made by this team will affect whether the outcome of the usability tests produce accurate and useful results.

Secondly, an organization will need to recruit people to perform the usability evaluation. Depending on the testing methods chosen, participants may include experts or potential users. Human-computer interaction experts can compare the usability of a new system against a gold standard, previously defined best practices, or help identify common mistakes in the systems’ design. A domain expert can provide insight into user preferences and goals, even if he or she is not part of the group of potential users. For example, when designing a system for a PICU, an experienced pediatric provider may be involved in testing the usability of a system, even if he or she is not currently working in the unit where the system will be implemented.

Although both experts and users can bring valuable insight to the usability testing process, each type of evaluator may also bring biases to the results. For example, the intended users of the system will be able to provide information about their goals and expectations of the system. In addition, they will be able to help identify problems with the system that may be unique to their work environment. Users may not, however, be as good at articulating their needs for the system or be able to help brainstorm more optimal design choices without structured facilitation by the usability testing team. On the other hand, experts will be able to bring insights from similar systems, and help organizations benchmark their system against best practices. Experts will not be able to talk about the nuance expectations of a system’s users, or identify issues that may be unique to a particular organization (United States Department of Health and Human Services 2017 ).

Both experts and users will likely expect compensation for their time. Many usability experts work as consultants and advertise their hourly rate. Internal experts will need to set aside time from their regular duties to conduct a usability evaluation. Recruiting from the intended user population may take even more effort. Depending on the user population, the usability study may need to be advertised to a large audience (e.g. the general public), provisions made to provide travel compensation, and additional compensation will be needed for users who complete the usability test. These costs should be considered when evaluating the different usability testing methodologies. Although it may be a tempting way to reduce costs, it is also important to remember that the people who design the system are not the best candidates for participating in usability studies. People who are closely tied to the system, including developers, healthcare provider champions, and individuals that designed the usability protocol, may unintentionally bring biases to the usability test and reduce the effectiveness of the usability testing data (United States Department of Health and Human Services 2017 ).

3.2.2.4 Usability Testing Methods

Selecting the appropriate method for usability testing depends on the organization’s goals, timeline, budget, and resources. Knowing the organizational goals and constraints will help you select the best method(s) for your usability evaluation. Many organizations use different usability testing methods at different points in the development process to gather a range of data about the system, and provide multiple opportunities to engage with users.

There are hundreds of tools that have been developed to help teams evaluate the usability of a system. Table 7-1 displays a list of commonly used strategies for gathering information from experts and potential users.

Table 7-1 Example usability testing methods
Full size table

3.2.2.4.1 Expert Conducted Usability Methods

As described above, experts can provide insight into how a new system compares to known usability best practices. In addition, domain experts can help system developers understand user needs and goals, even if he or she is not in the user population.

A heuristic evaluation is a process that involves expert evaluators testing a system against a predefined list of potential usability problems (United States Department of Health and Human Services 2017 ). Heuristic evaluations can be completed for general audiences (e.g. Nielsen’s heuristics (Nielsen 1994 )) or for specific user populations (e.g. heuristics for older adult smartphone users).

Heuristic evaluations are most effective when being used to detect obvious usability problems, such as small font or inactive buttons. Completing a heuristic evaluation first may increase the effectiveness of user tests. Eliminating the obvious problems can allow users to focus more on the nuances of the system instead of being distracted by glaring issues. In order to ensure that the heuristics are correctly applied, and the review is comprehensive, it is recommend that multiple experts use the heuristics to review a single system (Martin and Hanington 2012 ).

Cognitive Walkthroughs:

A cognitive walkthrough asks experts to evaluate a system by walking through a series of common user tasks (i.e. “scenarios”) and anticipate how the system should act during each step. Experts using this method are able to tell system designers where the system acts unexpectedly, which in turn may point to areas that may confuse potential users. This method is particularly useful with systems that will provide minimal user training, or where users will be expected to quickly adopt the system. Similar to the heuristic evaluation, this method is most effective when used before user evaluation to allow users to focus on system details during their evaluations (United States Department of Health and Human Services 2017 ; Martin and Hanington 2012 ).

Role-Playing and Simulations:

Sometimes it is difficult or not feasible to recruit actual system users for usability studies due to user availability and organizational resource constraints. In these situations, experts may try to act as the user for usability testing purposes. Using role-playing exercises or simulating a user experience may help experts empathize with user perspectives, and identify potential usability problems for that population. These methods will be most effective if the design team spends time ensuring that the scenarios used in the role-play or simulation is as realistic as possible. This may involve collecting data from users, conducting market research, or validating the scenarios with end-users (Martin and Hanington 2012 ).

3.2.2.4.2 User Conducted Usability Methods

Potential users are often asked to test systems to gain early feedback and identify issues during the formative evaluation stages. Gathering feedback from potential users can be challenging; however, because potential users may not be used to providing constructive feedback on systems. There are several usability testing methods that have been designed to help potential users express feedback.

Focus Groups, Interviews, and Surveys:

The most common way to elicit user feedback is to ask potential users to try a new system, and then ask them their opinion about the experience. Focus groups, interviews, and surveys are all tools that help users to express their opinions.

Individual interviews are a straightforward method for gathering qualitative from a usability test. After a user reviews the system, he or she is asked to sit down with a moderator to talk about the experience. Interviews can be structured (with the same questions asked to each participant), semi-structured (starting with the same questions but allowing the moderator to ask follow up questions), or open-ended (allowing the user to direct the conversation). Interviews can be used alone, or to supplement other testing methods. The advantage of an interview is that the facilitator can individualize the conversation to meet the needs of the participant, and spend the time to gather all the important details of an individual user’s opinion. Disadvantages of this method include a large time investment to schedule and conduct the interviews, and the difficulty in projecting how close one participant’s opinion is to other users (United States Department of Health and Human Services 2017 ; Martin and Hanington 2012 ).

Focus groups are sessions where a group of users (about 5–10) are brought together to talk about their experience at the same time. Examples of focus group participants include a multidisciplinary team of clinical providers, diverse users from the general population, or several people who perform the same job at different organizations. The advantage of conducting group sessions is that users can comment on each other’s opinions. This interaction may highlight similarities and differences of opinions between users, allow users to explain their positions more thoroughly, and develop a consensus about a design choice. Focus groups are more difficult to conduct than individual interviews. Focus group moderators must be sensitive to potential power dynamics that may inhibit users from expressing honest opinions, and lead the discussion in a way where participants aren’t able to derail the group with side conversations (United States Department of Health and Human Services 2017 ; Martin and Hanington 2012 ).

Surveys are a standard way to gather user opinions for a usability test. Surveys can be taken in person or remotely, and can be combined with another usability testing method. Surveys are most helpful when they are “validated,” which means that the survey has been previously shown to produce reliable results over repeated tests. Surveys are relatively easy and inexpensive to administer. There is also little cost associated with increasing the sample size of a survey to reach broad communities of participants. Disadvantages of surveys include the preparation needed to ensure that the data collected from the survey will be comprehensive enough to be actionable, and identifying a survey that has already been validated and will meet your usability testing goals or validating your own survey (Martin and Hanington 2012 ).

Think-Aloud Method

The think-aloud method asks potential users to verbalize what they are thinking, feeling, and doing as they complete a series of assigned tasks. The think aloud method can be either concurrent (the participant verbalizes their thoughts as they complete the tasks) or retrospective (tasks are video recorded and the participant provides commentary at the end of the task sequence). This method allows the team to collect data on what the participant was thinking or feeling as he or she made decisions on how to navigate the system. The advantage of the think-aloud method is that the moderator does not have to guess why a participant made the choices that they did, and to uncover areas of confusion within the system. Not all participants will be comfortable with this method, and a skilled moderator may be needed to know when and how to prompt the user to verbalize their thoughts (United States Department of Health and Human Services 2017 ; Martin and Hanington 2012 ).

A/B Testing

The A/B testing method involves prototyping two different interfaces for the same task or series of tasks. Participants are randomly assigned one of the two prototypes (A or B), and the usability testing team records the outcomes with that interface. After all participants have used either the A or B prototype, the outcomes from the different participant groups are compared. A/B testing is most useful when an organization wants to make a decision between two interfaces, and has specific goals for what the user should accomplish from a particular interface (e.g. purchase a product or complete a task within a set timeframe). A/B testing is not as useful for iterating on design choices or for gathering a nuanced understanding of user preferences .

3.2.3 Interface Design Standards and Design Principles

  • John Manning 

3.2.3.1 Design Standards

Since the late 1980s, The International Organization for Standardization (ISO) has taken an active role in helping define and improve on our understanding of usability, human-centered design/user-centered design (UCD), and the user experience (Holden et al. 2016 ; Jokela et al. 2003 ; Bevan et al. 2015 ). Various ISO guidelines exist to describe and provide guidance on evaluation/implementation of human-computer interaction (HCI) principles . As a general rule, ISO standards are reviewed every 5 years and may be revised or replaced to accommodate current best practices in thinking. A brief summary of the relevant ISO standards for HCI can be found in Table 7-2 .

Table 7-2 Brief overview of the pertinent International Organization for Standardization (ISO) standards for usability and human-centered design
Full size table

The concepts of usability and UCD are critical to HCI discussions. While their current use is covered in more detail in Sect. 3.2.2 , it is worth noting that the each of these concepts has experienced a shift in how it is defined. One simple example is the definition of usability, which has had the following definitions:

  • Nielsen (1994)—a system’s learnability, efficiency, memorability, error avoidance, and recovery (Nielsen 1993).

  • ISO 9241-11 (1998)—the ability to accomplish goals with “effectiveness, efficiency, and satisfaction in a specified context of use” (Holden et al. 2016; International Organization for Standardization 1998).

  • Task, User, Representation, Function (TURF; 2011)—something that is “useful, usable, and satisfying” (Zhang and Walji 2011). Note that the TURF framework specifically focuses on usability in electronic health record (EHR) design.

Design Principles

Various recommendations and design principles exist within HCI. These were covered in part in Sect. 3.2.2 with discussions of Nielsen’s heuristics, with AMIA’s usability design principles, and with Usability.gov ’s recommended UCD process. A summary of the recurring themes seen within these and related principles has been compiled and adapted into Table 7-3 below.

Table 7-3 Compilation of common Human-Computer Interaction (HCI) design principles
Full size table

3.2.4 Usability Engineering

Usability engineering is the process of making software more usable. Although usability engineers often make design and interface recommendations, their focus is on assessing and measuring how well end-users interact with the system. This is done through direct observation, click-counting, written surveys and evaluation of system logs.

A usability test ensures that an application meets user expectations with respect to meeting requirements (effectiveness) easily (efficiently) in a simplistic and satisfying manner. A usable application should be:

  1. 1.

    Easy to use

  2. 2.

    Easy to learn

  3. 3.

    Satisfying to the user

Ease of use is commonly measured by counting the number of minutes, keystrokes, mouse clicks or other interactions that are required for common tasks. One emerging method of analysis is to track users’ eye movements, to see where they spend the majority of their attention. In general, the most common use-cases should benefit from the most optimization.

Ease of learning is accomplished by adhering to common programming paradigms, and having a straightforward menuing system. Consider the following dialog boxes belonging to two very common mainstream applications ( Fig. 7-4 ). In Apple’s Safari, the OK button is on the bottom right of the dialog box. In Microsoft’s Paint, the Cancel button is on the bottom right. A user switching from application to application would be frustrated that the buttons are not where he expects them to be.

FIGURE 7-4
figure 4

Example of two dialog boxes in Safari and Paint, copyright Apple and Microsoft, respectively. Note that Apple has the Cancel button on the left while Microsoft has it on the right

Full size image

User satisfaction is much harder to measure . It is more about the user’s experience while using the application and his/her feeling towards it. This is usually measured by survey or direct observation. Footnote 1 One simple rule is that users want to feel valued. An application’s responsiveness is a key factor. Providing positive feedback for actions and avoiding lag are important. It goes without saying that applications with programming errors that crash frequently can challenge even the most dedicated user.

In commercial web sites, poor usability will result in lost sales. Footnote 2 In the case of an Electronic Health Record, user dissatisfaction will cause users to develop workarounds or revert to downtime procedures when the application doesn’t perform to their expectations. It is up to the usability engineer to prevent this from happening.