Keywords

Introduction

Improving healthcare is a goal across the world. In order to reach this goal, it is necessary to develop criteria, indicators and instruments to assess quality. Nearly fifty years ago, Sanazaro and Williamson noticed that not much work had focused on the development of objective criteria of performance (Donabedian 1966). For this reason, they conducted a study to create a classification—based on episodes of care provided by physicians—of what constitutes effective and ineffective performance (Sanazaro and Williamson 1970).

Since that time, a vast amount of literature has been published to understand better what quality care is and to find the most appropriate criteria and tools for its measurement and improvement (Arah et al. 2006; Brook et al. 1996; Campbell et al. 2000; Donabedian 1988, 1990; Institute of Medicine 2001; World Health Organization 2006). Major trends that have originated in the management field—such as Total Quality Management, Quality Assurance, Continuous Quality Improvement, Lean or Six Sigma—have also been applied to healthcare. In addition, publications such as those from the Institute of Medicine 1999, 2001), and associations such as the Joint Commission International, the American Society for Quality, the National Association for Healthcare Quality, the International Society for Quality in Health Care and the Agency for Healthcare Research and Quality have emphasized quality problems and their improvement.

Given this ‘quality revolution’ (Maguard 2006), we replicated Sanazaro and Williamson’s (1970) design about fifty years later, using a sample of healthcare managers, to compare our results to their suggested classification, identifying differences and similarities between physician and managerial perspectives and discussing the evolution of quality dimensions over time.

Methods

This study is part of a larger project (Amati et al. in preparation) to develop an empirically informed taxonomy of quality of care, grounded in Donabedian’s structure, process and outcome framework (Donabedian 1996, 1998). We refer to that paper (Amati et al. in preparation) for a more detailed description of the methods used.

We replicated a revised version of the critical incidents technique adopted by Sanazaro and Williamson (1970), who collected 9115 episodes of patient care —describing effective and ineffective performance—from 2342 physicians. Our sample comprised 236 top managers in executive positions, middle managers and directors, who had completed the Masters of Science in Healthcare Administration programme at Trinity University (San Antonio, Texas) from 2004 to 2013.

Sanazaro and Williamson’s (1970) classification system first divided quality statements into process (i.e. what physicians do to patients) and outcome (i.e. effects of physicians’ performance on patients). In addition, they identified specific subcategories of both process and outcome, such as ‘arriving at diagnosis’ or ‘improvement of physical abnormalities’.

Each episode of care from our study was analyzed using this classification, in order to ensure a comparison of the data. Moreover, we used an inductive exploratory approach to examine those parts of the texts that did not belong to any of Sanazaro and Williamson’s subcategories, leading to the identification of new dimensions of quality care (Amati et al. in preparation). Finally, after the percentages for each subcategory were calculated, we modified three tables published in Sanazaro and Williamson’s (1970) work to compare our results to theirs. The comparison was made by looking at ranks and means and did not use formal statistical analysis.

Results

Sample Characteristics

A total of 135 episodes of care were collected from 74 managers (response rate = 33%). Fifty-three percent of the respondents were female and the average age was 35 years old, with a mean of eight years of experience in healthcare management. Professional titles ranged from ‘Executive/Vice President’ (24%) and ‘Director/Manager’ (32%) to ‘Assistant/Associate Administrator’ (16%) and others, such as ‘Consultant’ and ‘Analyst’. Concerning organizational settings, 56% of the respondents worked in private not-for-profit hospitals, 19% in public hospitals, 17% in private for-profit hospitals, whilst the rest worked in other types of healthcare organizations (e.g. health insurance companies or outpatient clinics).

Process Subcategories

Sanazaro and Williamson’s Subcategories

Table 1.1 reports the top fifteen process subcategories of effective and ineffective performance most frequently reported in this investigation, compared with those from the original work (Sanazaro and Williamson 1970). Overall, Sanazaro and Williamson’s (1970) process subcategories were replicated by our data. However, the ranking and percentages were quite different from the original study. Since Sanazaro and Williamson’s (1970) investigation used physicians to describe quality of care, their derived taxonomy was very detailed about certain elements of the delivery of care (e.g. use of instruments, X-ray, EKG, caesarean section , etc.), which were not as prominent in our study.

Table 1.1 The top fifteen process subcategories of effective and ineffective performance most frequently reported by the healthcare managers in our study in comparison to Sanazaro and Williamson’s study (1970), expressed in percent
Full size table

Concerning effective performance, seven subcategories appeared in the top fifteen list of both studies (i.e. Surgical treatment , Use of facilities , Professional manner , Patient education, Arriving at diagnosis, Drug treatment and Laboratory ). However, some differences could be found: four subcategories (i.e. Arriving at diagnosis, Drug treatment, Patient education and Laboratory) were ranked higher by physicians in Sanazaro and Williamson’s work. Furthermore, three additional subcategories (i.e. Use of health team, Follow-up and Physician availability) were part of Sanazaro and Williamson’s (1970) overall classification, but did not belong to their top 15 list, whereas in the eyes of our managers they assumed more importance. In particular, Use of health team was the most reported subcategory of effective performance in our study.

Concerning ineffective performance, five out of the fifteen subcategories most frequently reported by physicians in Sanazaro and Williamson’s (1970) investigation also belonged to the top fifteen of our study (Professional manner, Patient education, Surgical treatment, Use of facilities and Drug treatment). However, whilst Professional manner was reported more frequently by our managers, Drug treatment was reported much less frequently than in Sanazaro and Williamson’s (1970) work. Five subcategories—which had been identified by Sanazaro and Williamson but that did not belong in their top fifteen list—assumed more salience in our study (i.e. Use of health team, Physician availability, Professional responsibility, Procedure and Follow-up).

The subcategory Physician availability in our investigation included the availability of other healthcare professionals. Overall, in the episodes of care that we collected, five of Sanazaro and Williamson’s (1970) top fifteen subcategories appeared as contributors of both effective and ineffective performance (i.e. Surgical treatment, Use of facilities, Professional manner, Patient education and Drug treatment). Arriving at diagnosis and Laboratory were amongst the top fifteen only under effective performance, whilst Professional responsibility and Procedure appeared only under ineffective performance.

New Subcategories

Six new subcategories were identified from our episodes of care (Amati et al. in preparation). Four of them ranked amongst the top fifteen of both effective and ineffective performance: Staff-patient-family communication, Timeliness, Inter-staff communication, and Adherence to guidelines/ protocols . Patient-centredness was a new subcategory under effective performance and Consistency/Continuity of care was a new subcategory under ineffective performance.

The subcategory Inter-staff communication included more specific communication aspects that were not covered in the subcategory Use of health team—which only referred to ‘coordinating services of other physicians, nurses, auxiliary workers; promoting, facilitating communication among professionals’ (Sanazaro and Williamson 1970, p. 301)—such as handoffs, communicating wrong information, conflict management, alert, documentation, debriefings and ‘speaking up’. The subcategory Staff-patient-family communication included aspects of Patient-centredness —defined by the Institute of Medicine as ‘providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions’ (2001, p. 40); Patient education—i.e. ‘instructing, educating; explaining; preparing patients. Primary purpose is increased patient knowledge and understanding of condition and regimen’ (Sanazaro and Williamson 1970, p. 302); Professional manner—i.e. ‘establishing or maintaining rapport; physician behavior/attitudes in dealing with patient’ (Sanazaro and Williamson 1970, p. 301); and Psychologic support—i.e. ‘Reassuring; alleviating concern; expressing interest in patient, family. Goal is improved emotional state’ (Sanazaro and Williamson 1970, p. 302).

The above subcategory referred not only to a unidirectional type of communication from the healthcare staff to the patient and the family, but it also emphasized a mutual type of relationship, stressing the importance of the patient ‘speaking up’, of the quality and timeliness of the information and the manner in which it is exchanged. Furthermore, whilst in Sanazaro and Williamson’s (1970) work the subcategory Patient education specifically referred to treatment, in our study communication was also about navigating the patient and their family through the healthcare system and the process of care.

If we group all these subcategories under two broad dimensions named Inter-staff communication and Staff-patient-family communication, the former one would cover 19.2% of all subcategories related to effective performance and 17.5% of all subcategories related to ineffective performance. The latter one would account for 28.8% of all subcategories related to effective performance and 32% of all subcategories related to ineffective performance. Therefore, overall, in this study communication aspects would account for 48% of all subcategories related to effective performance and 49.5% of all subcategories related to ineffective performance.

Outcome Subcategories

Beneficial Outcomes

Table 1.2 reports the top thirteen most frequent beneficial outcomes of our study, compared to Sanazaro and Williamson’s (1970). Out of their study’s top thirteen beneficial outcomes, six were confirmed in the top thirteen of our results (i.e. Attitude towards M.D., care: Positive; Physical abnormalities: Complete recovery ; Individual function: increased; Life saved; Physical abnormalities: Prevented; and Physical abnormalities: Improved). In half of the cases, the ranking was even similar (Physical abnormalities: Complete recovery; Individual function: increased; and Life saved).

Table 1.2 The top thirteen most frequent beneficial outcomes reported in our study, compared to Sanazaro and Williamson (1970), expressed in percent
Full size table

Amongst the main differences, the subcategory Attitude towards M.D., care: Positive was the beneficial outcome most frequently reported by the managers in our sample. Furthermore, Physical abnormalities: Improved, which was in the third position in Sanazaro and Williamson’s ranking, was not as prominent in our study, whilst Physical abnormalities: Prevented had a higher ranking.

In our investigation, Unnecessary risk: Avoided or reduced, Psychological symptoms: Partially relieved and Hospitalization: Avoided or reduced assumed more relevance. In Sanazaro and Williamson’s (1970) work, these were not listed in the top thirteen subcategories of beneficial outcomes. Furthermore, four outcomes on our list represented a new contribution: System adjustments; Process outcomes: Care received; Efficient utilization of resources; and Accommodation of patient/family needs: positive (Amati et al. in preparation).

Detrimental Outcomes

Table 1.3 compares Sanazaro and Williamson’s (1970) top thirteen most frequent detrimental outcomes with ours. Nine subcategories corresponded, five of which also had the same ranking, with similar means (i.e. Physical abnormalities: Caused, exacerbated ; Attitude towards M.D., care: Negative; Psychological symptoms: Caused, exacerbated; Physical symptoms: Caused, exacerbated; and Cost: Increased). However, in our investigation Hospitalization: Unnecessary and Unnecessary risk: Incurred ranked much higher, whilst Death caused and Physical abnormalities: Prolonged, unimproved ranked lower.

Table 1.3 The top thirteen most frequent detrimental outcomes reported in our study, compared to Sanazaro and Williamson (1970), expressed in percent
Full size table

Four new subcategories emerged: Did not return to the same facility; Death not attributable to providers; Perception/Reputation of the facility: Negative; and Inefficient utilization of resources. However, unlike the new process subcategories—which were at the top of the ranking—the first five most frequently reported subcategories belonged to Sanazaro and Williamson’s (1970) original categorization, and three of them ranked exactly as in Sanazaro and Williamson’s (1970) table (Physical abnormalities: Caused, exacerbated, Attitude towards M.D., care: Negative and Psychological symptoms: Caused, exacerbated).

Discussion

Assessment is necessary for improving healthcare and the literature offers numerous examples of ways to measure quality (Griffey et al. 2015; Rushforth et al. 2015; Carinci et al. 2015). Amongst these efforts, Sanazaro and Williamson (1970) developed a classification based on physician reports of effective and ineffective performance in relation to patient outcomes. Our study replicated their design, but used a sample of US healthcare managers instead of physicians.

The findings showed that Sanazaro and Williamson’s (1970) subcategories re-emerged in the episodes of care collected in this study, indicating that their suggested framework holds over time, and despite nearly fifty years of progress in quality improvement since their investigation, many issues are still relevant from the point of view of the healthcare managers. In this paper, we have presented the top fifteen effective and ineffective process subcategories and the top thirteen beneficial and detrimental outcome subcategories. In numerous cases, the ranking was quite different and new ideas were identified. There are two possible explanations for the differences: (1) contemporary healthcare managers might have different perceptions about the dimensions of quality care than do physicians; or (2) the dimensions of quality have evolved over time for both managers and physicians.

Process Subcategories

In Sanazaro and Williamson’s work (1970), the most reported subcategory of effective and ineffective performance was Arriving at diagnosis , which emphasizes the importance attributed by physicians to identifying a condition or disease in relation to a beneficial or detrimental outcome of care. On the other hand, in our study contemporary healthcare managers seemed to identify aspects related to good teamwork (i.e. Use of health team) as key for the attainment of good quality care, whilst poor quality care was critically determined by poor communication amongst healthcare staff, patients and families (i.e. Staff-patient-family communication).

In our study, eight subcategories of effective performance and ten of ineffective performance did not even appear in the top fifteen list produced by Sanazaro and Williamson (Table 1.1). Some of them represented new contributions of our study (i.e. Timeliness, Patient-centredness, Adherence to guidelines/ protocols , Inter-staff communication and Staff-patient-family communication ), while others were already present in Sanazaro and Williamson’s (1970) investigation but were not reported very frequently. For example, healthcare managers seemed to attribute more importance to aspects such as Use of health team, Physician ( and nurses) availability or Professional responsibility, whilst they rarely discussed issues related to Drugs, biologicals, electrolytes, fluids or Laboratory.

Timeliness was not even considered as an attribution of quality by Donabedian (1990), but it later became one of the six dimensions identified by the Institute of Medicine—defined as ‘reducing waits and sometimes harmful delays’ (Institute of Medicine 2001, p. 40). As defined by the Agency for Healthcare Research and Quality, timeliness in healthcare is the ‘system’s capacity to provide care quickly after a need is recognized’ (Agency for Healthcare Research and Quality 2016). Today, advancements in medicine and technology make it possible to intervene in and potentially solve extremely complex clinical cases; however, timeliness has become even more fundamental. For example, research shows that lack of timeliness can result in emotional distress, physical harm and higher treatment costs (Boudreau et al. 2004), whereas appropriate care delivered in a timely manner can reduce morbidity and mortality for chronic conditions such as kidney disease (Kinchen et al. 2002) and affect stroke patients’ long-term disability and mortality (Kwan et al. 2004). Moreover, clinical outcomes can be improved by timely antibiotic treatments (Houck and Bratzler 2005). The relevance of timeliness was indeed confirmed and highlighted by our data.

Another notion that has drawn the attention of contemporary healthcare managers is Patient-centredness, which has been integrated into many quality definitions (Institute of Medicine 2001; World Health Organization 2006; Arah et al. 2006). There is substantial ambiguity related to its meaning and the best method to assess it (Mead and Bower 2000). We view patient-centredness as a partnership between the provider and the patient, and not a mere accommodation of patients’ needs and expectations (Street et al. 2003). Consequently, patient-centredness and communication are intrinsically tied to each other: there is no patient-centredness without communication, but at the same time, there is no effective communication without patient-centredness.

Communication aspects were not absent in Sanazaro and Williamson’s (1970) categorization, but they were mainly considered as part of the delivery of a service, such as instructing the patient or sending comfort messages, and not as an interplay amongst all parties involved. Contemporary research attributes to provider—patient communication historical functions such as exchanging information or responding to patients’ emotions, but it also sheds light on additional ones, such as fostering healing relationships, managing uncertainty, making decisions with the active involvement of patients and families, and enabling patients’ self-management whilst advocating for patients and supporting their autonomy (Epstein and Street 2007). In this investigation, communication—with its different facets—accounted for almost 50% of both effective and ineffective performance, confirming the growing awareness of its importance in healthcare (Agarwal et al. 2010).

Finally, the emergence of the subcategory Adherence to guidelines/protocols suggests that it is an increasingly important topic , as it has been shown that in the USA only 55% of patients receive care as recommended in the guidelines (McGlynn et al. 2003). Research studies are trying to uncover the barriers that hinder the implementation of guidelines in clinical practice (Lugtenberg et al. 2011).

Outcome Subcategories

Concerning beneficial outcomes of care, in both investigations the second most discussed beneficial subcategory was Physical abnormalities: Complete recovery. However, in Sanazaro and Williamson’s (1970) work, the first one was Individual function: increased, whilst in the episodes provided by our participants it was Attitude towards medical doctors and care. This denotes contemporary healthcare managers’ awareness and concern that the quality of care affects more than physical and psychological patient outcomes. In fact, amongst the new beneficial outcome subcategories we found Accommodation of patient/family needs, whilst amongst the new detrimental ones we found Not return to the same facility and Perception/ Reputation of the facility .

In both studies, the most frequently reported detrimental outcome was Physical abnormalities: Caused, exacerbated. Contemporary healthcare managers are concerned—as were physicians fifty years ago—that the care provided may not improve patient health, but instead it may prompt or worsen physical abnormalities, diseases, conditions and their complications. Surprisingly, despite the increasing attempts to contain healthcare costs (Schnipper et al. 2012; Minogue and Wells 2016), there was no qualitative difference in the ranking of Cost: Increased. Whilst we typically expect managers to factor in costs in their assessment of quality of care, the respondents in our study did not emphasize financial aspects very much.

On the other hand, the importance of Sanazaro and Williamson’s (1970) subcategories Unnecessary risk and Hospitalization are perfectly in line with current management concerns. This was also emphasized by the emergence of new subcategories such as System adjustments , Utilization of resources and Perception/Reputation of the facility. In fact, much research has been conducted to investigate and address issues such as rehospitalization (Hansen et al. 2013), misuse of resources (Bulger et al. 2013), or hospital reputation (Mira et al. 2013).

Limitations

The limitations of our investigation mostly pertain to sample size and that it included alumni from only one US graduate programme in Healthcare Administration, who mainly work in the same geographic area in which they earned their degree. The response rate was 33%, which limits the validity of the results, even though it is similar to that achieved by other surveys of healthcare managers (McDonagh and Umbdenstock 2006; Vaughn et al. 2014). Finally, we compared contemporary managers with physicians. Different stakeholders account for diverse perspectives and findings. For this reason, further research is needed to focus on contemporary physicians in order to investigate the evolution of quality dimensions in relation to this specific group of stakeholders.

Conclusion

This study replicated Sanazaro and Williamson’s (1970) design to investigate qualitatively how the dimensions of quality have evolved over time and how the perceptions of managers might be different from those of physicians. Our findings confirmed the existence of the subcategories identified about fifty years ago by Sanazaro and Williamson (1970) in relation to the process and outcomes of care, suggesting that those dimensions of quality are still valid nowadays. However, several subcategories gained more importance, and new dimensions emerged from the data. This suggests that the multifaceted concept of quality care has evolved over time, and for this reason, it is imperative to take into account a wide spectrum of dimensions when assessing it, and to potentially change priorities in the process of continuous quality improvement.