Keywords

1 Introduction

The healthcare industry is investing heavily in health information systems to enhance patient outcomes, efficiency, and financial performance (Wickramasinghe and Schaffer 2010). Most recently, Gartner has ranked the healthcare industry as the fifth highest spender on information systems/information technology (IS/IT ) at ~USD108 billion, with an increase of 2.7% compared to 2013 (Gartner 2015). In Australia, we are now witnessing this trend with significant investments being made by various healthcare organizations for various technology solutions to provide and enable better care delivery (Haddad et al. 2015). IT user satisfaction has been shown in various academic and nonacademic publications as a determinant for successful IS/IT projects (Adam Mahmood et al. 2000; Maldonado and Sierra 2013; Dwivedi et al. 2013; Abelein and Paech 2015). The focus of the vast majority of the current literature are on the factors that affect IT user satisfaction. For example, user involvement in systems development, perceived usefulness, user experience, organizational support, and user attitude toward the IS were reported as key factors influencing user satisfaction in general with IS/IT (Adam Mahmood et al. 2000) and that is in agreement with numerous other studies (Dwivedi et al. 2013; Xiong et al. 2014; Bharati and Chaudhury 2006). While examining IS/IT user satisfaction in healthcare has a lengthy history (Adler 2007; Ammenwerth et al. 2006; Cresswell et al. 2013; Nguyen et al. 2015), measuring user satisfaction with clinical information systems lags behind. This examination of the overall user satisfaction with four clinical information systems qualitatively with the use of descriptive analysis identifies the relationship between the overall satisfaction and five aspects of clinical information systems, namely, key functionalities; efficiency of use; intuitiveness of graphical user interfaces (GUI); communications, collaboration, and information exchange; and interoperability and compatibility issues.

2 Methods

An online survey was conducted to collect data on clinical IT user satisfaction at a tertiary, not-for-profit, private healthcare group in Australia. The adopted survey instrument was first tested in another healthcare context to ensure validity of the instrument. As recommended by Miller, three inputs can be used to determine the design of a user satisfaction survey, namely, the objective of the survey, the users’ characteristics, and the resources available (Miller 2004). The objective of this survey was to develop a valid measurement of clinical IT user satisfaction. As the participants are predominately clinicians whose schedules are always busy, the design of the survey took this issue into consideration. The survey is relatively short and enables the users to skip sections that are irrelevant. As the selected case is a large healthcare group with multiple sites and locations, an online survey was the preferred option to collect the data. The respondents needed to click a hyperlink to the online survey prior to answering the questions, and a detailed participant information sheet was presented to the respondents about the purpose of this study and how they can take part in it. A total of 107 respondents answered the questionnaire. Due to missing information and incomplete responses, 76 valid questionnaires were used to present the results on clinical IT user satisfaction in the selected context of this study. The response rate was 38.3%. This rate is approximately 3% greater than the average response rate for studies that utilized data collected from organizations through questionnaire/survey methods as was measured by Baruch and Holtom (2008). The questions were focused on four main clinical information systems (Table 2.1) used by various clinicians at the selected healthcare group.

Table 2.1 The studied CISs in this study and their descriptions
Full size table

3 Results

3.1 CIS User Satisfaction

The respondents were first asked on how often they use clinical information systems (CIS) in their daily work with patients. To avoid any confusion, the survey defined a CIS as “any kind of clinical information and communication technology (ICT) system to support patient care (e.g., managing patient information and paperwork, patients’ medication, diagnostic findings, required investigations, etc.).” 51% of the respondents stated they had used CISs several times per day (Fig. 2.1a).

Fig. 2.1
figure 1

(a) How often CISs are used by the respondents . (b) The percentage of users who use the examined CISs for their daily work

Full size image

The most used CIS in the examined group of systems was SMR with 97% of the respondents answered with Yes on the question whether they use this system in their daily work. RRV was the second common CIS with 47%, followed by CPOE with 13%, and CAT with only 3% of the population said they had used it in their daily work (Fig. 2.1b).

Answering the question on how the participants were satisfied with the four examined systems, RRV was the most satisfying CIS with 63% of the participants satisfied and 6% very satisfied with it as Table 2.2 summarizes.

Table 2.2 The overall satisfaction with the examined CISs in the selected case
Full size table

In order to identify the reasons behind these levels of satisfactions, the respondents were asked to evaluate sets of statements on their use of the examined systems to perform their tasks. From a system functionality perspective, these statements covered providing decision-making support, preventing medication errors, visualizing data and information to facilitate better work flow, improving health outcomes, improving access to important clinical information (lab, radiology, pathology) and documenting these information, improving the quality of information available, and reducing duplicity of effort. As RRV and CPOE were the most and least satisfying CIS, we compared the responses of the statements regarding these two systems. The comparison covered five primary aspects: key functionalities; efficiency of use; intuitiveness of graphical user interfaces (GUI); communication, collaboration, and information exchange; and interoperability and compatibility issues. The summary of this comparison in the area of key functionalities is presented in Table 2.3.

Table 2.3 A comparison between the most and least satisfying CIS in the selected case
Full size table

Similar comparisons showed that CPOE has challenges with efficiency of use, intuitiveness, and supporting information exchange, communication, and collaboration in the clinical space. Although the majority of the users thought CPOE was a reliable system, there is an agreement that the system is not easy to communicate with other systems in a way that enables interoperability . On the other hand, RRV seemed to be accepted by the majority of the respondents in terms of its key functionalities (Table 2.3), efficiency of use compared with using paper to facilitate the daily tasks, intuitiveness of GUI, and supporting collaboration in the clinical space. However, RRV seemed to be struggling in terms of supporting access to information in a timely manner.

3.2 Training and Technical Support Satisfaction

The respondents were surveyed on their satisfaction with IT equipment and systems (hardware and software) in the workplace (Fig. 2.2).

Fig. 2.2
figure 2

The overall satisfaction with IT equipment at the selected case

Full size image

The selected case has an IT hotline in place, the majority of the respondents said they had rarely used this service (79%), and 8% said they had never used it. 29% of the respondents stated that their IT problems were solved immediately over the phone. Similarly, IT-on-call-duty is never used by the respondents. This service relates to IT emergencies and interruptions during the night and on weekends. Asked about the level on onsite support, 50% of the respondents were neutral, 35% were satisfied, and 10% were very satisfied.

The survey then asked on the amount of trainings which the respondents had attended in the last 12 months. The majority of the respondents stated they had received no training at all, and around 87% of them were dissatisfied with IT training.

4 Discussion

This study was performed to qualitatively gain a better understanding of the levels of user satisfaction with four clinical information systems at an Australian healthcare group. Descriptive analysis was also used in this study. The four clinical information systems were of different objectives, the CPOE helps with facilitating electronic drug prescribing, CAT helps create an electronic record for every and each admission to the healthcare group, RRV enables fetching radiology images electronically, and SMR is designed as a system that enables storing all medical records at the selected case in a scanned form. These systems were examined against five primary areas of investigation : key functionalities; efficiency of use; intuitiveness of graphical user interfaces (GUI); communication, collaboration, and information exchange; and interoperability and compatibility issues.

The majority of the participants in this study were satisfied with RRV and dissatisfied about CPOE, and 47% of the participants were satisfied with RRV. RRV is the least expensive system within the examined group of clinical information systems. Yet, it is the most satisfactory system to the majority of its users. The analysis shows that CAT is not widely used at the selected case, and all of its users were neutral about it. This is understandable as the system had been recently implemented at the time of data collection and building conclusions about it might be practically challenging. The most utilized system was SMR with about 97% of the participants using it. The system is seen as a necessary step to EMRs by digitizing all medical records around all admissions that occur at the different sites of the group. Currently, it is used mainly to scan medical records, code clinical episodes, and track paperwork around every admission to all sites of the group.

The system is relatively inexpensive to operate and maintain and is easy to use as described by the majority of the participants. This system, however, suffers from its limited functionality. It is understood that it does not offer the medical records in a way that enables data analytics or business intelligence. This limitation makes this system incapable of coping with today’s digital requirements of healthcare delivery. Further, although the system is used group-wide, it only covers inpatient, leaving outpatients out of its scope. The most satisfying system as the results show was RRV, with almost 70% of the participants satisfied and very satisfied with it. A number of characteristics of RRV significantly contributed to this high satisfaction level as the results show. These include supporting information exchange, communications, and collaboration in the clinical space, intuitiveness of user interface, efficiency of use, and the key functionality of the system in terms of improving access to important clinical information as well as providing the clinicians with quality information that support their decisions around respective care episodes. In addition, documenting clinical information is also easily enabled by using RRV as the results show, which contributed to the high level of satisfaction with using this system.

In contrast, CPOE was the least satisfactory system for the participants with 50% of the participants dissatisfied and 50% very dissatisfied with it. CPOE is a sophisticated system that is used primarily by a limited number of clinicians in the area of cancer care for drug prescribing and patient scheduling, which explains the low percentage of use (13%), unlike SMR, for example, which is used by all clinicians in the selected case. The main factors that contributed to lower levels of satisfaction with this system relate to its functionality, ease of use, technical problems, and intuitiveness of the user interface. Indeed 100% of the participants stated that working on paper is more efficient than using the system that is due to technical problems faced by the clinicians with logging in (takes extended times), entering data, and extracting information of the system. As these activities tend to be lengthy procedures and require a broader bandwidth by the clinicians to deal with, 100% of the participants agreed that the use of this system is distracting them from paying attention to their patients. Further, the studied CPOE does not seem to support information exchange, communication, and collaboration within the clinical domain, with 100% of the participants agreeing that this system does not support delivering information about patients to clinicians within or across healthcare providers.

The level of training and technical support on spot have also contributed to the overall satisfaction of CIS users at the selected case. The results show that the majority of participants were satisfied with the IS/IT equipment they have and thought they were appropriate for the type of work assigned to them. However, the level of training both in-house and external was way below the expectations and needs of the users as the results show. Indeed, both CPOE and CAT received lower satisfaction scores due to lacking a proper training that tracks the progress of their utilization of the system and realizing its benefits. The overall satisfaction seems also to be affected by the level of technical support provided on spot. Although all of the participants were happy about the level of help desk provided to them, this support is limited to normal technical issues. With more complex enquiries about sophisticated systems, the technical support seemed to struggle to meet the actual needs of users. Two of the main facilities available for users to use to receive technical support were barely used. These are the IT hotline and IT-on-call-duty services. It is not clear from the results why these are not utilized by the users, which needs a further investigation.

The implications of this study cover both theory and practice. Theoretically, the survey instrument may be used by various types of hospitals and healthcare organizations in general to understand the overall user satisfaction with their clinical information systems. This is particularly timely with the ever-increasing trend to implement electronic medical records (EMRs) in Australia and globally. One of the factors that make this survey valid for different contexts is its coverage to various aspects around the usefulness of clinical information systems in today’s healthcare delivery. This includes the systems compatibility with clinicians’ tasks in terms of core functionalities, efficiency of use, intuitiveness of user interfaces, accessibility of information, support of collaboration, and interoperability , compatibility, and reliability of the studied systems.

Practically, the results of this study help decision makers and top management at hospitals to better understand the actual needs of clinical information systems’ users to better utilize CIS as contemporary assets (Fichman and Melville 2014; Davern and Kauffman 2000). This is crucial with the increased investments in IS/IT in healthcare. Today, healthcare is ranked fourth in investing in IS/IT after retail, banking and securities, and education (Gartner 2015). The study also shows that CIS users are likely to be satisfied if the systems are intuitive and easy to use and enable better access to medical information in a timely manner. This agrees with numerous studies in the literature. See, for example, (Adler 2007; Ammenwerth et al. 2006; Cresswell et al. 2013; Haddad et al. 2014; Nguyen et al. 2015; Wickramasinghe et al. 2014). The results also show that decision makers will need to pay attention to training and technical support channels . The amount and quality of training are key aspects of user satisfaction as the results show. While it is not clear why the vast majority of the participants in this study did not use the IT hotline and IT-on-call-duty services based on the data, further investigation on this matter is likely to clarify this behavior and how to minimize its impact on the overall user satisfaction with clinical information systems.

This study has several limitations. First, the sample size is relatively small and only covers one healthcare organization. Second, the structure of the survey is predominantly qualitative and meant to evaluate the overall user satisfaction with their clinical information systems.

5 Conclusion

This study set out to evaluate the overall user satisfaction with clinical information systems at an Australian tertiary, not-for-profit, private healthcare group. Different constructs were considered to evaluate the user satisfaction. The results show that intuitive, easy-to-use, and collaboration enabling systems are more likely to satisfy their users. The level of technical support and training seem to play key roles in determining user satisfaction in the clinical domain. Future research directions include fine-tuning the survey to quantitatively determine user satisfaction based on its constructs in this study, i.e., systems compatibility with clinicians’ tasks in terms of core functionalities, efficiency of use, intuitiveness of user interfaces, accessibility of information, support of collaboration, and interoperability, compatibility, and reliability of the studied systems. Also, examining the impact of user satisfaction on the business value of IS/IT in healthcare and moderating role of proper training, coaching, and change management practices on this relationship is planned to be the second phase of this study. A comparison between different healthcare providers is also beneficial and planned to be conducted in the future. Finally, this study highlights the benefits of a survey methodology. The survey used was designed and tested in specific healthcare contexts to establish its validity. A survey methodology can be an appropriate and very meaningful approach to capture key insights from various users, but often further follow-up research is needed to fully understand and unpack key insights gained.