Keywords

Introduction

Australia and Germany have many similarities when we compare healthcare delivery. Both countries have two-tier health systems, that is, a mix of public and private, and are designing national e-health solutions ; for Australia, it is the personally controlled electronic health record (PCEHR) and, for Germany, it is the e-health card. In addition, these countries are embracing information technology (IT) solutions to effect better healthcare delivery to try to stem escalating cost pressures. These similar health environments provide an exceptional opportunity to investigate healthcare issues, simultaneously in similar contexts to leverage the potential for shared learning and the transfer of expertise. In any application of IT, key success factor now becomes the level of user satisfaction and the rate of adoption of the various technology solutions. In healthcare , given the multiplicity of users and their respective needs of a system, high user satisfaction is always challenging. To date, good metrics which serve to correctly capture user needs and satisfaction have yet to be developed for healthcare contexts and proxies are used from other industries which naturally at best lead to suboptimal solutions. This chapter attempts to address this important issue by designing and developing a suitable evaluation criterion for measuring user satisfaction with IT in healthcare contexts in both German and Australian hospitals and then providing a systematic framework to address the key issues identified. Given the cost to healthcare of poorly designed technology solutions and low user acceptance , this study is especially timely and its results are most beneficial and far reaching. Moreover, given the similarities between the two healthcare systems in the respective countries, such a study can leverage lessons within and between the two systems, which not only make the study truly unique but also make the findings much more robust, useful and useable in order to develop superior technology solutions that, in turn, will serve to support and enable the realisation of excellent healthcare services that address all current challenges for both countries.

Background

The healthcare sector is an information-intensive area with high information demands (Wickramasinghe and Schaffer 2010). IT provides timely and accurate information to make sure that physicians, nurses and other care professionals obtain the complete and extensive information they require to provide high-quality care (Wickramasinghe et al. 2009; Haux et al. 2003). On the other hand, hospital IT also ensures fulfilling managerial needs while improving the hospital’s effectiveness and efficiency needed to support such care (Brand et al. 2012; Simon 2010). Given the rapid development of healthcare IT, many hospitals have aggressively increased their IT expenditures. Today, hundreds of information systems are used in hospitals (from widely used electronic patient records, computerised order entry systems, to modern radiology information systems, including speech recognition technology). A series of IT support, maintaining, consulting and training services usually provided by a hospital’s IT department is now in high demand. Although the technology-related benefits are obvious in theory, it seems that they are not clearly associated with the operating situations in hospitals. The evidence is strong: A high number of reports are published about system flaws, poor IT usability and insufficient relation to work activities (review in Viitanen et al. 2011). Previous research indicates that a key factor affecting the successful adoption of hospital IT lies in user acceptance , perceived usability and satisfaction (Viitanen et al. 2011; Chen and Hsiao 2012; Bundschuh et al. 2011; Smelcer et al. 2009; Ash et al. 2004; Stürzlinger et al. 2009; Bleich and Slack 2009; Ludwick and Doucette 2009). Despite the increasing trend towards user satisfaction issues, relatively little systematic data have been gathered on user satisfaction with hospital information services in a comprehensive way. Usually, the successful implementation , adoption and daily use of hospital IT depends on the quality of the information systems themselves as well as on the various IT services provided by the hospital IT department (i.e. IT hotline, on-site service, on-call duty, training and advising for user, and project management, etc.) . As the literature shows, previous publications focus mainly on a single project or healthcare information system (e.g. Smelcer et al. 2009; Stürzlinger et al. 2009; Bleich and Slack 2009; Ludwick and Doucette 2009; Bürkle et al. 2001; Khajouei et al. 2009; McKinlay et al. 2010), are conducted in a specified use context (e.g. Ash et al. 2004; Ludwick and Doucette 2009; Röhrig et al. 2007; Kuosmanen et al. 2010; Ammenwerth et al. 2002; Oroviogoicoechea and Watson 2009; Sánchez 2004) and involve rather a small number of participants or include only one health profession (often only physicians; e.g. Viitanen et al. 2011; Stürzlinger et al. 2009). Furthermore, whether in Germany or Australia, no national reference data exist. In addition to the data issues in current studies, there is nearly no information about the psychometric properties of the used or developed instruments and scales (i.e. validity and reliability). Hence, the quality of such study results must at least be considered as uncertain.

Aim

Physicians and nurses are key providers of healthcare services in hospitals and are main users of hospital IT. The implementation, appropriate usage and user satisfaction can improve the quality of care that health professionals provide and influence the hospital performance and outcome quality for patients. The aim of our investigation is to measure and validate user satisfaction with hospital information services in a comprehensive manner. This study will be one of the first investigations in this research area in Australia and Germany. The research outcomes will not only answer the senior hospital management’s question, “How good is our hospital information service?”, but will also provide hospital IT researchers and practitioners for the first time with valid and useful measurement instruments (best practice standard questionnaire).

Objective

The objective of this investigation is to measure and validate user satisfaction with hospital IT services, specifically to:

  1. 1.

    Understand the current state of user satisfaction with IT services in the context of clinical environments in hospitals, and provide a descriptive picture of the present situation from the subjective perspective of health professionals in Germany and Australia

  2. 2.

    Provide first reference values/data on user satisfaction in both countries for senior hospital management and chief information officers (CIOs), as well as a basis for benchmarking and efforts for developing quality and efficiency of IT performance in hospitals

  3. 3.

    Develop and implement empirical analysis and statistical validation of appropriate metrics and scales for healthcare contexts (to provide validated and appropriate standard measures)

  4. 4.

    Identify starting points for increasing the quality of hospital IT systems as well as IT support, and to support a more detailed evaluation processes in the future

Rationale

Healthcare costs in Australia and Germany are increasing exponentially. To address this and other healthcare challenges including the aging population and increases in chronic diseases, both countries are investing heavily in information systems (IS)/IT . Nowadays, nearly all clinical and administrative processes depend on IT systems and related services. However, without proper metrics that are designed specifically for the known challenging, multilayered dynamic environment of healthcare contexts, neither is it possible to evaluate the benefits of these technology investments nor design them to optimally meet the user needs. Thus, a key void is the existence of appropriate metrics and instruments to measure and validate user satisfaction with hospital IT services —a void that this research will address.

Achievement and Measurement of Aims

To achieve the stated aim, we have already developed a comprehensive questionnaire based on literature research and expert focus group interactions in Germany. A pretest and a preliminary study proved the validity and appropriateness of the questionnaire in Germany successfully (n  = 106, Hospital Nuernberg). Before administering the questionnaire in Australia and extending our study to the intended multicentre investigation, we will run pretests in a second pilot study (including translation/retranslation of the questionnaire). Our main purpose is to collect a sufficient national sample in Germany and Australia to analyse the current status of user satisfaction , and provide valid and useful measuring instruments and scales for hospital executives. Figure 5.1 illustrates the research design. In addition, unstructured interviews will be conducted at all data sites. Second, but equally important, aspects of this project include achieving a high level of knowledge transfer between the two countries as well as significant industry participation. The development of the comprehensive questionnaire and its subsequent administration represents a key sharing/transfer of expertise and knowledge, which will be further enhanced through the focus group discussions with key experts. Industry partners, namely the respective public and private hospitals, who have agreed to participate in this study will not only benefit themselves but also represent significant industry support and participation in the study. Finally, transfer of skills and expertise especially between Early Career Researcher (ECR) and senior project team members within and across countries will occur. Given the existing track record and collaboration to date between Professors Simon and Wickramasinghe, we are confident that not only will the knowledge transfer within and between the research groups be successful but also the whole project will be completed so that all aims will be achieved and all key success factors will be met.

Fig. 5.1
figure 1

Proposed research design highlighting key project steps

Full size image

Data Collection

Data will be collected in multiple ways including site visits, briefing and debriefing of CIOs , senior hospital management as well as medical and nursing directors, and the administering of questionnaires completed by IT users (health professionals). The central focus of the planned investigation will be on the main occupational groups within hospitals: physicians and nurses. All data collection will be guided by healthcare and IT professionals. Visits to the respective data sites by all members of the project team (i.e. German and Australian) will ensure that replication is being conducted in a similar fashion and will also serve to facilitate learning across both data sites.

Recruitment Strategy

In order to replicate and extend the size of the pretest study, a sample size of 500 participants each in Australia and Germany will be regarded as sufficient. According to the project’s vital importance, the German Association of Hospital Chief Information Officers (KH-IT is the highest body of German hospital CIOs and represents more than 400 members) has already approved full support. In fact, given that certain hospital CIOs have already agreed and wish to participate in the study, the recruitment process will not be a big challenge. A selection process will be conducted to cover the main hospital size categories (e.g. large, middle and small) and the public and private healthcare sectors. The multicentred, cross-sectional study in each hospital will include participants in the following inclusion criteria:

  1. a.

    Staff members in a clinical department, that is, physician or nurse

  2. b.

    Working for the hospital for more than  3 months

  3. c.

    Regular and active user of IT systems

  4. d.

    Willing to participate in the study

In order to achieve the intended sample size of participants, a response rate between 10 and 30 % at each investigation site (hospital) will be regarded as sufficient. The participation will be totally voluntary and anonymous for the IT users (Fig. 5.2).

Fig. 5.2
figure 2

Four-module questionnaire study

Full size image

Study Methodology

We will apply a triangular approach to measure user satisfaction with hospital IT. Triangulation defines a research method that combines various study measurements and concepts as well as theories in one investigation. It is increasingly used to cover complex investigation objects in qualitative as well as quantitative studies (Flick 2011). Our comprehensive concept includes the measurement of four questionnaire modules (see Fig. 5.2):

  1. 1.

    Module Quick Check: general user satisfaction (4 items), three open questions, overall satisfaction grade

  2. 2.

    Module IT-Service: appropriateness of IT equipment and frequency of use , IT hotline (10 items), IT on-site service/support (7 items), IT on-call duty (at night and weekends) (7 items), IT training for users (8 items)

  3. 3.

    Module IT-Application: suitability for the task (15 items), suitability for learning (8 items), conformity with user expectations (8 items)Footnote 1 (see Fig. 5.3)

    Fig. 5.3
    figure 3

    Module IT application

    Full size image
  4. 4.

    Module Statistics (socio-demographic variables)

The questionnaire will be administered as an online version (applied in Questback Unipark, EFS Survey, version 8.0). Closed and open variables will be included (e.g. user satisfaction, importance, satisfaction grade and open questions, i.e. appraisal, critical issues, statements and hints).

Data Analysis

Statistical analyses will be performed with the Statistical Package for the Social Sciences (SPSS), version 20. Descriptive statistics, including mean, standard deviation and frequency, percentage will be calculated. The differences between the subsamples from Australia and Germany will be compared using the t-test should the data prove to be normally distributed. Psychometric evaluation will be conducted to validate all developed instruments. The reliability of the scales (internal consistency and split-half reliability) and the validity will be reported (exploratory factor analysis). Significance will be set at the 5 % level (p < 0.05). Qualitative content analyses (Flick 2011) will be used for the participants’ responses to the implemented open questions.

The Expected Outcomes

This is a very rich study that attempts to address a key void in the current research and practice for healthcare assessment in both Australia and Germany . Thus, a major expected outcome is to fill this key void for both countries. In so doing, we shall develop a tested systematic evaluation tool for both Australian and German healthcare organisations to clearly and accurately measure the level of user satisfaction with hospital IT services and thereby enable the hospitals in the respective organisations to immediately address any/all issues with their existing IS/IT solutions, as well as be able to inform the design and development of new solutions. Thus, the value of the outcomes of this study is far reaching and potentially very large, especially in view of the millions of dollars and euros spent on developing IT solutions for healthcare delivery in the respective countries.

Innovation and Significance

According to our preliminary research and pilot project, most of the hospital CIOs in Germany and Australia do not measure user satisfaction . Among the small number of hospital CIOs with relevant data, the majority use self-developed, hands-on questionnaires with poor empirical quality. Moreover, as outlined previously, there is nearly a complete lack of national studies on user perception on hospital IT service quality in Germany, Australia or elsewhere. Hence, valid reference data covering the entire national hospital sector are not available. This research project will be one of the first, if not the first, investigations in this field. Given the huge amount of money spent on healthcare IT in both countries as well as the increasing escalating costs of healthcare expenditure for both countries, it is not possible to overstate the importance and significance of this study.

Conclusions

This chapter has served to outline a longitudinal study to investigate user satisfaction of hospital information services. Consumers of healthcare information are many and varied. In a hospital context, this includes physicians, surgeons, nurses and allied healthcare professionals, as well as various levels of healthcare managers. Clearly, all these user groups have different needs and requirements from the information they access and use. Understanding all these different user perspectives is an essential first step in the design and development of technology solutions that are, in fact, user/consumer centric. Our proposed study then serves to outline a systematic and rigorous approach to assist in gaining a better understanding of consumer needs of health information services and systems within hospital context. We conclude by calling for more research in this key area.