Introduction

International health IT benchmarking initiatives

Multinational health IT benchmarks have become a common instrument to measure IT indicators that give an account of the readiness for health IT in a country and to stimulate cross-country learning [1]. The OECD eHealth model survey is a methodological approach to define relevant indicators in terms of availability and use of a broad range of systems and functionalities in different health care settings from the perspective of different stakeholders [2]. Parts of the OECD model survey formed the foundation of the Survey of the European Commission where hospitals from 30 countries responded to questions of the availability and use of systems, health information exchange, IT infrastructure, context and governance variables [3].

Cross-country learning that draws on benchmark facts allows politicians to find out whether the eHealth strategy in their country meets the initial expectations, to learn from best practice examples and to align the eHealth strategy accordingly [4]. With the many differences in healthcare systems around the globe that potentially affect the adoption of eHealth, benchmarking among countries with a similar healthcare context seems promising to identify eHealth specific facilitators and inhibitors [5]. In 2012, the Nordic countries therefore launched an initiative to benchmark the availability, use and usability of eHealth systems across their countries [6]. Another example of health IT benchmarks in similar healthcare system environments was the comparison between Germany and Austria [7], which was conducted in 2007 and published in 2010.

Against this background we decided to repeat the Austrian German health IT benchmark using relevant OECD indicators. Knowing that both healthcare systems are shaped to a large degree by national regulations [8, 9], laws have the potential to exert a strong influence on the general health IT climate and on the monetary conditions of the health care organisation [10, 11]. In addition, other factors with a potential impact on health IT adoption, in particular the perceived innovative power of the organisation could make the difference between adopting and non-adopting organisations. Some case studies hint at the importance of organisational innovativeness [12, 13]. Finally, there are other facilitators and inhibitors on the level of the hospital demographics known from the literature, which need to be taken care of for adjustment, first and foremost “size” of the organisation [14, 15], ownership [16], teaching status [14, 17] and system affiliation [18].

Healthcare systems in Germany and Austria

Germany and Austria both have an insurance-based healthcare system with the majority of the population insured in the statutory health insurance (Table 1). Whereas Germany has higher expenditures per capita, life expectancy at birth is marginally lower. Austria furthermore shows a higher physician- and nurse-to-bed ratio with regard to acute care facilities, which hints at a better staffed acute care system. Austria also shows a shorter average length of stay.

Table 1 Selected indicators describing the healthcare systems in Austria (AUT) and Germany (GER)
Full size table

Research framework

In order to benchmark Austria and Germany with regard to health IT, we propose a research framework, which describes the environment and potential forces in this field (Fig. 1). The benchmark object in our study was the availability of IT functions in the hospital information systems. The availability was judged by the clinical stakeholders as the experts, who are familiar with the IT functions and IT systems that shape the clinical processes. We thus speak of “perceived technical availability”, which can differ from the “technical availability” as seen through the eyes of a chief information officer [21] and from the actual “use” of these functions.

Fig. 1
figure 1

Research framework of the eHealth benchmark between Germany (GER) and Austria (AUT): variables in black font were observed in this study, black arrows mark the integration of the variables into the regression analysis, light grey arrows mark environmental forces that could affect the results but were not included into the regression analysis

Full size image

In our framework, we assume that IT adoption of hospitals is exposed to two major potential forces: The top-down force “country specific forces, in particular the legal-financial environment” and the bottom-up force “innovative power of the organisation”. This research framework also integrates existing knowledge about hospital demographics exerting a potential influence on IT adoption as discussed above. The framework draws on existing models, particularly on the socio-technical-material framework [22], in which the material environment, e.g. laws, financing schemes and other forces, that cannot be changed easily, was integrated.

This framework allows the following research questions to be derived:

  1. 1.

    Do German and Austrian hospitals differ with regard to their “perceived technical availability” of IT functions?

  2. 2.

    Do demographic factors of the organisation play any role to explain possible differences?

  3. 3.

    Does perceived “innovative power” of an organisation contribute to the understanding of potential differences between the two countries?

Country specific context forces will be used to discuss the results rather than to phrase research questions.

Methods

Study design and measurement instrument

We conducted an observational cross-sectional study in acute hospitals in Austria and Germany. We hereby chose directors of nursing as representatives of clinical stakeholders to answer the questions. Due to their dual role as experts in the clinical field and as board members or as high-ranking executives they oversee the area of interest. Furthermore, they represent the largest group of healthcare professionals in a hospital, who are exposed to IT systems in their daily work.

IT adoption was operationalised by measuring the “per-ceived technical availability” of 27 IT functions that were taken from the OECD eHealth model survey and previous surveys within the framework of IT Report Healthcare [23, 24]. These functions covered the six clusters: “documenta-tion”, “order entry and results reporting”, “decision support”, “patient safety”, and “supply chain functions” and “interface functions”. The implementation status of each of these IT functions had to be rated on a 4-point Likert scale (1 = “no implementation planned”, 2 = “implementation started/resources provided”, 3 = “implemented in at least one unit but not in all”, 4 = “implemented in all units”).

In addition, the hospital demographics “country”, “location”, “size”, “ownership”, “system affiliation”, “teaching status” and “surgery available” were included into the questionnaire. “Innovative power” of the organization was rated on a 10-point scale with 1 denoting no power and 10 the highest possible power. The entire questionnaireFootnote 1 is shown in Appendix Table 4.

Data management and statistical analysis

One thousand seven hundred fifty four email addresses of German and 169 of Austrian directors of nursing in hospitals could be identified by Internet research. They represented 96.1% percent of the German acute hospitalsFootnote 2 and 95.5% of the Austrian ones. The questionnaire was made available to them between November 2013 and February 2014 [23] utilising the online survey tool Unipark (http://www.unipark.com).

All data were analysed using R (Version 3.2.1). Statistical significance was set at alpha = 0.05. To account for multiple testing, p-values were Bonferroni adjusted. In order to describe the two samples, we tested for differences with regard to the demographic variables using logistic regression analyses with the criterion country (Austria as reference). The samples were also contrasted with the population in each country regarding the “size” of the hospital (see Appendix Table 5).

In order to compare both countries, we used a sub-score within the Workflow Composite Score (WCS) [26], an aggregated score, which measures the IT potential of hospitals to support clinical workflows. WCS provides four descriptors which are represented by sub-scores: (1) data and information, (2) IT function,Footnote 3 (3) integration of IT functions and (4) distribution of data and information. These sub-scores, which are reflective of points given to IT features that relate to the particular descriptor, are expressed as normalised sum values [26]. In this study these values could range from 0 to 100 points.

To evaluate the IT adoption of both countries we utilised the sub-score "IT function". It is a highly reliable score (split half reliability r = 0.89) that integrates the 27 IT functions addressed in this survey [26]. The sub-score “IT function” served as criterion in a stepwise forward multiple linear regression analysis, into which “country”, the demographics variables and the variable “innovative power” were entered as predictors. The final model was tested for non-multi-collinearity, homoscedasticity and normal distribution of the residuals. The significant predictors of this model were used in subsequent logistic regressions to test the country differences on the level of the 27 individual IT functions. To this end, the implementation status of the 27 IT functions was dichotomised, which were then analysed as criterion in univariate logistic regressions with “country” as predictor and adjusting for demographic variables.

Results

Sample

A total of 464 German and 70 Austrian directors of nursing took part in the survey, which corresponded with a response rate of 26.5% in Germany and 41.4% in Austria. Hospitals of all “size” categories and federal states participated in both countries. We compared the sample data to all population measures that were made available by the responsible federal offices (portion of for-profit hospitals, bed-size and teaching status in Germany and portion of for-profit hospitals as well as the bed-size in Austria). Significance testing revealed no significant differences between the sample and the population in both countries (see Appendix Table 5). Additionally, we compared the sample data of both countries: There were no country specific differences with regard to “size”, “system affiliation”, “teaching status” and availability of a “surgery theatre” (Table 2). Only “ownership” turned out to be significantly different with an odds ratio of 2.24 (95% CI 1.31–3.78), i.e. a 2.24 greater chance to find a for-profit hospital participating in the survey in Austria compared to Germany (Table 2).

Table 2 Summary of hospitals demographics (95% CI in brackets) and the results of univariate logistic regression analyses for hospital demographics (predictor) and country (criterion) of the sample
Full size table

IT adoption: “Perceived technical availability”

Figure 2 shows the frequency distribution of the adoptionFootnote 4 of the 27 IT functions in the two countries. This descriptive approach revealed nine IT functions with nearly equally distributed adoption rates (difference less than five percentage points). Eleven IT functions had higher adoption rates (> five percentage points) in Austria whereas seven functions showed higher adoption rates in Germany. The highest difference in favour of Austria was found for “nursing documentation” (Δ = 35.8 percentage points), the highest difference in favour of Germany for “identification of samples” (Δ = 23.0 percentage points).

Fig. 2
figure 2

Adoption rates (implemented in at least one unit) for 27 IT functions sorted by size of difference with the largest positive difference between Austria and Germany at the top

Full size image

The aggregated WCS sub-score “IT function”, which provided an integrated view on all IT functions, yielded an arithmetic mean of 57.9 (SD = 18.8, n = 70) for Austrian hospitals and of 52.3 (SD = 12.6, n = 464) for German hospitals. The two countries differed significantly with regard to this score (p = 0.027). This indicated a higher adoption level of IT functions in Austrian hospitals compared to German ones.

In order to explain the variance of this score a stepwise forward multiple linear regression analysis was performed. “Innovative power” had the strongest effect with the highest beta coefficient (p = 0.000) on the aggregated score. Furthermore, the variance of the “IT function” sub-score could be also explained by the demographic variables “hospital size” (p = 0.000) and “hospital system affiliation” (p = 0.015). These results indicated that larger hospitals and those hospitals belonging to a multihospital system had higher “IT function” values. The final model with the four significant predictors could account for 42.8% of the total variance in “IT function” (Table 3).

Table 3 Final multiple linear regression model resulting from stepwise forward selection with, IT function sub-score” as criterion (all models see Appendix Table 6)
Full size table

The effect of “innovative power” on the aggregated WCS sub-score “IT function” was similar for both countries: more innovative hospitals had higher scores than less innovative ones as shown in the two univariate linear regression analyses (Fig. 3). However, Austrian hospitals had significantly higher innovation values (\( \overline{x}=6.9 \), SD= 2.1; n = 60) than German hospitals (\( \overline{x}=5.9 \), SD = 2.1; n = 409). A univariate logistic regression analysis with country as criterion resulted in a significant OR value of 1.25 (95% CI 1.09–1.44).

Fig. 3
figure 3

Scatterplot “innovative power” of the organisation perceived by the directors of nursing versus WCS sub-score “IT function” for both countries

Full size image

IT adoption: Adjusted “perceived technical availability” for individual IT functions

Based on the knowledge that “innovative power”, “hospital size” and “hospital system affiliation” could significantly explain the variance of the aggregated “IT function” sub-score (WCS), the computation of the OR values of the 27 individual functions were adjusted for the influence of these two demographic variables. Knowing that the two samples differed significantly with regard to hospital “ownership” this variable was included as third demographic factor for the adjustments (Fig. 4).

Fig. 4
figure 4

Adjusted OR values and 95% CI for demographic variables. Austria served as reference category in the logistic regression: OR > 1 indicates a greater chance that an IT function is implemented in Austrian hospitals than in German ones. For example, having implemented a nursing documentation system was 5.98 times more likely in Austria than in Germany

Full size image

The adjustment for demographic variables (all values of the adjustment see Appendix Table 7) led to significant differences between the countries. Only for “identification of samples” (OR = 0.39) Germany showed a higher IT adoption. Whereas the adjustment resulted in five IT functions with significantly higher perceived availability for Austria, i.e.

  • “nursing documentation” (OR = 5.98).

  • “intensive care unit (ICU) documentation” (OR = 2.49) and

  • “medication administration documentation” (OR = 2.48),

  • “electronic archive” (OR = 2.27),

  • “medication” (OR = 2.16),

Discussion

Sample and research questions

This study is based on a large sample of hospitals in Austria and Germany with a good response rate. A wide coverage of different hospitals from different regions in both countries and of different size classes participated.

There is a higher “perceived technical availability” of IT in Austrian hospitals compared to German hospitals. “Nursing documentation”, “ICU documentation”, “medication administration documentation”, “electronic archive” and “medication” show a significantly greater availability in Austrian hospitals. In comparison, only one IT function, i.e. “identification of samples”, was more often available in Germany.

These results partly resemble the findings of the 2007 study, which was published in 2010 [7]. At that time “nursing documentation”, “electronic archive” and “picture archiving and communication system” showed a significantly higher availability in Austria than in Germany. This indicates that the differences between both countries still persist with regard to the first two applications and now include more functions than in 2007. The significantly higher composite sub-score “IT function” in Austria than in Germany matches with a comparable sub-score (AUT = 0.653 versus GER = 0.502), which was developed in the context of the European Hospital Survey [25].

“Hospital size”, i.e. the number of beds, and “system affiliation”, i.e. whether the hospital is working on its own or in a multihospital system, were found to significantly influence the variation of the sub-score “IT function” but could not explain the difference between the two countries. The association between hospital demographics and IT adoption matches other findings with regard to “size” and “system affiliation” [14, 15, 18] but contradicts the literature with regard to the effect of “ownership” and “teaching status”. The correlation between the latter two variables and “hospital size” may explain this result [14]. In both countries, teaching hospitals and not-for-profit hospitals tend to belong to the group of larger hospitals.

“Innovative power” of the organisation as perceived by the directors of nursing exerted a forceful effect on the variation of the WCS sub-score “IT function”, which was not only significant but yielded the highest beta coefficient in the model. “Innovative power” worked uniformly in both countries with regard to fostering IT adoption, but was significantly higher in Austria than in Germany. Thus the overall potential impact of “innovation power” was stronger in Austria than in Germany.

Research framework

The research framework, which underlays this study, assumed two main forces: the bottom-up force “innovative power” of the organisation and top-down forces, in particular the legal-financial environment. At first glance, the influence of “innovative power” seems trivial or tautological. However, we contend that the strength of this factor was not predictable, even though we expected some positive correlation with IT adoption. Our findings affirm studies which emphasize the important role of “innovative power” and “organisational culture” [12, 26] but did not demonstrate it empirically. Innovation always entails some sort of risk to be associated with the implementation of the innovation [27]. Organisations with strong innovative power often venture forth on unchartered territory also at the costs of failure.

“Innovative power” could therefore be also associated with the top-down acting context factor legal-financial environment. In Germany, shortcomings in the reimbursement of investment costs within the G-DRG system are discussed as a strong inhibitor of innovative changes with mid- or long-term return of investment [28] such as health IT. IT often needs time to unfold its potential and contribute to a positive cost-benefit ratio because of a complex implementation and integration process [29, 30]. Unlike Austria, Germany shows an ongoing trend to shorten the length of stay (LOS). This difference may explain a higher pressure to act in Germany [31] and to curb costs, e.g. cutting nursing staff (see lower nurse-to-bed ratio in Germany in Table 1) instead of investing in new technology. Comparing the spending of the statutory health insurance per hospital bed in both countries also reveals a more favourable situation in Austria than in Germany (161,482 Euro in Austria versus 127,482 Euro in Germany in Table 1). Assuming similar cost structures, these figures point to the fact that there is more money in the health care system in Austria than in Germany.

The top-down force legislation had been discussed in the 2007 study to account for the higher adoption rates of “nursing documentation” in Austrian hospitals. Austria had passed a law, the Austrian Healthcare and Nursing Act [32], already in 1997 that stipulates the documentation of the full nursing process including the nursing diagnoses [7]. It was argued that it took some time before this law got manifested in corresponding IT adoption rates of “nursing documentation” systems. The effects of this law can still be seen. In Germany, the Nursing Complex Intervention Score (German: Pflegekomplexmaßnahmen Score PKMS) of the Hospital Financing Reform Law of 2009 [33] could potentially stimulate the uptake of “nursing documentation” but became effective in 2012 only. This circumstance may have made it difficult to measure its effect in particular given a slow acceptance of the PKMS.

Legislation seems to be most effective if it stipulates health IT and at the same time helps building enough free space to let health IT emerge or to give direct incentives for health IT adoption such as the Meaningful Use Program in the United States of America [34].

Limitations

The limitations of this study are related to the research design as an observational cross-sectional study that does not allow any causal relations to be derived. We assume that “innovative power” comes prior to IT adoption and thus may influence adoption behaviour. However, it could also be the other way round. Because organisations had implemented novel IT functions they felt they were innovative. Even more complex, feeling innovative and behaving innovative may be intertwined in a self-reinforcing process [35], i.e. because an organisation judges its “innovative power” as high, it adopts innovative technology and because it has adopted innovative technology it judges its “innovative power” as high.

Apart from “innovative power”, other factors may have a potential influence on health IT adoption, e.g. “management of the IT implementation process” and “user support” [36], “commitment of the top management team” [13, 27] and “participation of clinical end users” [37]. They should be considered in future studies. In addition, “innovative power” itself needs further clarification in particular facilitators and inhibitors, e.g. factors acting behind the scenes such as IT governance and centralisation [38].

Cross-sectional studies are prone to the self-selection bias. There are statistical techniques such as the propensity score that aim to remediate this bias [39]. We did not employ any of these methods because we did not find significant differences between the samples and the corresponding populations with regard to a selected number of variables. Equally, we did not have access to specific variables that might differentiate between responders and non-responders and thus could be entered into propensity analyses.

Another limitation arises from making the questionnaire only available to directors of nursing. Even though they represent the largest user group of health IT in hospitals, this study lacks the perspective of other medical professions such as physicians or therapists that would complement the user’s viewpoint in hospitals.

Conclusions

This study is the first to empirically demonstrate the effect of “innovative power” in hospitals pursuing a regression approach in a bi-national health IT benchmark. We recommend including the financial situation of healthcare organisations into future regression models. On a political level, measures to stimulate the “innovative power” of hospitals should be considered to increase the digitalisation of healthcare.