Event reporting systems are widely prevalent across healthcare organizations and are used as tools to learn about a variety of negative outcomes and near misses. As such, they are artifacts of the traditional approach to safety, to learn from how things go wrong based on specific episodes or incidents. These systems typically involve self-reporting by staff involved in the incident, in compliance with policies and guidelines on reportable events. Generally, however, the effectiveness of such tools in improving safety has been limited. One of the primary reasons is that this approach to organizational learning primarily focuses on errors, near misses, and adverse events, all of which represent things that go wrong. Among the workforce, this can result in fear of blame, reprimand, and associated social and socio-legal consequences (Anderson et al., 2013; Ashcroft et al., 2006; Sujan, 2015; Waring, 2005). From a learning perspective, retrospective analysis of negative occurrences or outcomes is fraught with hindsight bias, where the adverse or potentially adverse consequence leads to a tendency of the analyst to undervalue the contextual factors that influenced or necessitated the course of actions taken prior to the event (Cook et al., 1998; Wears & Cook, 2004). As a corollary, there is an underappreciation of the contextual factors that influence adaptive responses. As a result, there is a widening gap between work as done in actual operational contexts and work as imagined by policy makers and system designers (Hollnagel, 2015, 2016).

In contrast, the resilience engineering (RE) approach is to learn from how things go well in everyday work. This is based on the premise that things go right and wrong for, essentially, the same reasons, that is, variability in performance within a variable environment. However, there is a lack of formal mechanisms and tools to operationalize such learning in organizations. Much of the current empirical literature on resilience has involved research investigators observing, interviewing, or surveying domain stakeholders. These efforts have largely been in the context of research for academic purposes. There are few, if any, examples of concerted efforts by organizations to implement frameworks for proactive learning about everyday work with the lens of resilience engineering. This chapter focuses on the development and efforts to implement a self-reporting tool for frontline caregivers at hospitals – the Resilience Engineering Tool to Improve Patient Safety (RETIPS). RETIPS is designed to enable caregivers to share narratives of adaptive performance in their everyday work. In contrast to traditional incident reporting systems, a key feature of this tool is that it aims to elicit examples of successful adaptive performance in the context of specific events as well as ‘normal’ routine functioning when there are no ‘events’. This marks a shift toward proactively learning about normal work as it happens, including how and why performance in daily routines varies, and why it usually succeeds (when there is no event). The chapter will provide a summary of the development, initial implementation and results. The chapter will also dwell on the authors’ experience in their attempt to implement it at a large multispecialty hospital.

1 Developing the Original RETIPS

The project had its origins in an interview-based knowledge elicitation technique designed to learn about how things go well in everyday clinical work. The interview was semi-structured, based on the format of the Critical Decision Method (CDM) by Klein et al. (1989). The questions were adapted to focus on situations with positive outcomes in terms of patient safety, as well as formal and informal practices, routines and adaptive measures employed in everyday work. Questions were also derived from Hollnagel’s Resilience Analysis Grid (RAG) and adapted to suit the healthcare domain and clinical areas to which participants belonged. An initial study developed and conducted interviews of frontline caregivers. Qualitative analysis findings were used to develop a self-reporting form, RETIPS, which retained the essence of the interview protocol in terms of its knowledge elicitation goals. However, the structure was adapted to self-reporting, that is, a combination of free text and multiple-choice questions. Feedback was sought from experts in human factors and safety. Additionally, feedback was sought from domain stakeholders, including clinicians. The feedback at this stage is mostly related to the semantics and relevance of the content. There was not much emphasis on making the tool pragmatic, such as its length. A detailed description of the original version, including its development from the interview results, has previously been published (Hegde et al., 2015).

2 Development of RETIPS 2.0

The initial version of RETIPS was reviewed by anesthesiologists at a large multispecialty hospital as a potential tool for lesson-sharing in anesthesia. Iterative feedback was used to refine the tool, resulting in RETIPS 2.0 – a much more concise version of the original, adapted to anesthesia residents. The clinicians acknowledged the conceptual basis of the tool as relevant and were supportive of implementation on a trial basis. The feedback at this stage mostly focused on the design of the tool for practical use in clinical settings. Specific feedback included:

  • Conciseness: The clinicians, almost unanimously, agreed that the original version of RETIPS was too long, which would be a deterrent for potential respondents given the highly busy environment in which they work. In order to make the tool more practical, the general suggestion was to make the tool as short and concise as possible. One clinician provided a specific guideline: “it should take no longer than 10 minutes to submit a response”.

  • Focused narrative: Several clinicians suggested customizing the tool for specific clinician groups, such as anesthesia residents, and focusing on specific safety and quality issues, such as difficult airway management. This approach would drive more focused recall and narrative. The guiding examples, cues, and response choices should be tuned accordingly. This strategy would have the added benefit of enabling analysis of patterns related to each group and issue through multiple reports. On the flipside, it could entail significant time and effort to develop separate versions of the tool for the various issues.

  • Clear purpose: It was important to clearly communicate to the respondent how the information provided would be used to enhance patient care. This could be done both during dissemination of the tool and in the introduction section of the tool itself.

3 Tool Description

RETIPS-AnRes consists of multiple sections, described below in sequence.

Introduction: A short paragraph is included at the beginning of the tool to define ‘resilience’ in a health care context and the purpose of the tool.

Case Selection: The respondent is asked to think of examples from their own work practice that relate to resilience in terms of preventing patient harm. This question field guides the respondent to think of either of two types of examples: a specific instance or a generic routine or process.. The examples were chosen so as to be relevant to anesthesia workflows and, therefore, more relatable to the respondent.

Detailed Narrative: This field is provided for the respondent to describe in detail the ‘resilience’ example they considered in the previous section. The following thematic cues are provided to guide the respondents with their descriptions: Key Challenges and Concerns; Adaptive Responses; Anticipation; Preventive Measures; Monitoring Behaviors (checks, reviews etc.); Resource Availability; Policies and Standard Practices; Communication; Cooperation; Patient/Family Involvement. There is no suggested word limit to the narrative description.

The remaining sections of RETIPS consist of checkbox-type responses intended as probes on various aspects (e.g., success factors, challenges, resources) of the reported case that are relevant from a resilience perspective. Each response field includes a text box to allow respondents to elaborate or describe other factors not listed that may have been involved in their example.

What Went Right: This field is designed to highlight the factors that contributed to success or the factors that were favorable to the goals inherent in the resilience example being related. Response choices include experience and knowledge of co-workers; culture and attitudes; standard practice/policy; shared understanding; cooperation between co-workers; and leadership.

Challenges and Concerns: This section more specifically probes the issues that challenged or threatened patient safety, or impeded successful intervention. The six response choices are: patient condition or behavior; communication issues; complexity of the situation; uncertainty or ambiguity; limited resources; and policy issues.

Resources: This section asks the respondent to check off those resources that were useful in the situation(s) they previously described: adequate time; technology/equipment; co-workers/consults; information; and procedural guidelines.

Area of Practice: The respondents are asked to indicate the specific clinical area of practice related to their example, such as surgical, preoperative, and postoperative anesthesia.

4 Implementation of RETIPS 2.0

Pilot Implementation: After multiple revisions, RETIPS-AnRes was administered twice in a two-year period to consecutive batches of anesthesia residents in their internship year (first postgraduate year, or PGY-1). The residents served as a representative group of the potential user-population, that is, frontline caregivers. The tool was implemented as part of a week-long course on Quality Improvement (QI) for each cohort of anesthesia interns. RETIPS-AnRes was welcomed by the residency program directors as it could serve as a tool that would allow for self-reflection on challenges and triumphs during the residents’ workday. Such reflection is key for a number of professional milestones outlined by the Accreditation Council for Graduate Medical Education (ACGME): self-directed lifelong learning (ACGME Anesthesia PBLI Milestone 3: Self-directed Learning) as well as System-based Practice Milestone 3 (Patient Safety & QI), and Professionalism Milestones 3 (Commitment to Institution Department and Colleagues) and 4 (Receiving & Giving Feedback). These milestones are a way in which the ACGME determines who can practice medicine at a defined level of proficiency through a competency-based model (Holmboe et al., 2016). Therefore, RETIPS-AnRes was incorporated into the QI course curriculum as a self-reporting exercise for the interns. The tool was made available in an electronic format through REDCap, a Health Insurance Portability and Accountability Act (HIPAA)-compliant web-based application used to build and manage online surveys (Harris et al., 2009). In order to encourage use of the tool, the department’s residency program director offered extra credits for submission of responses. Responses to the tool were anonymized. Reports were reviewed by authors, SH and CDJ, for overall response patterns and alignment of responses with the purpose of the tool.

Additionally, the tool was also made available to all anesthesia residents at the hospital. Dissemination strategies included emails to the residents introducing the tool, its purpose and potential impact. The emails were endorsed by the chief residents as well as the residency program directors. Unfortunately, however, no response was received.

In year one, nine reports were received from the six residents, and in year two, four reports were received. As the reports were anonymous, it was not possible to tell whether all six residents responded in year one or whether any participant submitted more than one report in year two. Participants briefly described lessons learned, and indicated the success factors, challenges, and resources pertinent to their examples. Four of the responses were categorized by the participants as ‘specific’, four as ‘generic’, and one as ‘both’. However, upon further examination of the responses given, we found that three of those categorized as ‘generic’ were, in fact, specific instances of routine workflows. Some examples of the responses collected are given below:

  • Exemplar Response 1: “During this robotic assisted case, despite a low probability of requiring blood products during this OR case, the anesthesia resident made sure to have blood readily available in room in the event that an adverse event occurred during which the daVinci robot could not be moved out of the surgical field quickly. The rationale was that in case a vital artery (specifically, the pulmonary artery in this lung wedge resection) is injured during the robot assisted surgery, blood could be given rapidly using units in the room as a temporalizing measure rather than having to wait for blood products to be transferred from blood bank while the robot could be moved away from the surgical field.”

    • Category: Specific.

    • Success Factors: Experience and knowledge of co-workers; standard practice/policy; cooperation between co-workers; shared understanding; culture and attitudes.

    • Comment: “Communicating with the circulating nurse, OR nurse and blood bank allowed for blood units to be available for this case, which followed from the close conversation the resident had with the anesthesia attending and with the surgical team. This allowed us to plan for and foresee potential catastrophic outcomes prior to start of the case.”

    • Challenges: Communication issues; complexity of situation. Comment: “Getting blood products into the OR seems to be a critical but oftentimes time-delayed process. Having blood on hand was an important portion of the anesthesia plan as detailed above, however despite the best efforts of the OR team blood took over 40min to make it into the room.”

    • Resources: Adequate time; procedural guidelines.

  • Exemplar Response 2: “During my overnight calls (especially when I am cross-covering patients from other services), I like to touch base with each one of the nurses about our patients before doing my own rounds. In this brief meeting, I like to address their concerns for the night. I feel that gathering this information before seeing the patients helps me to have more effective and productive rounds. In this interaction, I also inform the nurse about my plans during the night and I make myself available for anything during the night. I feel that this practice improves our communication and their trust level in me as the intern on service.”

    • Category: Generic.

    • Success Factors: Experience and knowledge of co-workers; cooperation between co-workers; shared understanding; leadership.

    • Challenges: Communication issues; complexity of situation.

    • Resources: Adequate time; co-workers/consults; information.

  • Exemplar Response 3: “On positioning of patient in a prone position, all available members in the OR assist in flipping the patient. The anesthesia resident is the one in charge of communicating and directing members of the team. I observed my senior anesthesia resident clearly giving instructions on how to flip the patient and explicitly stated the order to put the monitors back on the patient in order of importance. The patient was not flipped until all members of the team were ready. Care was taken with all IV lines and with the ET tube. The patient had become hypotensive on induction, so care was taken to closely monitor blood pressure after patient positioning. Care was also taken to relieve areas of pressure points. Arms were placed in the neutral position.”

    • Category: Specific.

    • Success Factors: Experience and knowledge of co-workers; cooperation between co-workers; shared understanding; culture and attitudes; leadership.

    • Challenges: Limited resources.

    • Comment: “One of the arm boards of the bed didn’t seem to be working and thus the patient was not positioned appropriately. This was recognized and instructions were given to nursing to help to retrieve another arm board. The surgical team assisted with patient positioning and ultimately the patient’s arms were placed in a satisfactory position.”

    • Resources: Adequate time; technology/equipment; co-workers/consults.

  • Exemplar Response 4: “If there are no surgeries booked at a certain time cutoff the acute care surgery operating room becomes available for other services so that surgeries such as transplants can be performed. This assures prompt attention for these patients minimizing ischemic times, etc.”

    • Category: Both.

    • Success Factors: Experience and knowledge of co-workers; standard practice/policy; cooperation between co-workers; shared understanding; culture and attitudes; leadership.

    • Challenges: Uncertainty or ambiguity in the situation; limited resources; policy issues.

    • Resources: Information.

5 Reflections on the Pilot Implementation Experience

Based on the responses submitted to RETIPS 2.0, it seems that the expectations of the tool were generally understood by the respondents. This was generally reflected in the fact that the responses ranged from specific incidents or episodes to generic routines free of any episodic context. However, all responses illustrated elements of variability in the environment and performance. Participants were able to explicitly articulate elements of variability in their everyday work in a variety of contexts. Therefore, the tool seems to be effective in terms of its knowledge elicitation objective. That said, given the limited number of responses, their usefulness in informing the identification of any systemic patterns of adaptation cannot be verified yet. Aggregating a sufficient number of examples of variability in a specific area could allow for such patterns to be identified.

However, the larger challenge that became evident from the exercise related to the uptake of the tool in operational settings. The healthy response rate – 13 reports from 12 participants – from the cohort of residents who were part of the quality improvement training can be attributed to the fact that RETIPS was included as part of their curriculum with extra credit for submitting reports. There was a clear purpose and incentive for fulfilling course goals, within a time-bound context. In sharp contrast, there were no responses received from the larger resident population despite a formal endorsement by faculty who were residency directors, and follow up circulars encouraging residents to submit responses. This is not surprising given the many competing priorities for their time, including clinical duties and fulfilling educational requirements. Another reason may be that there was no clear perceived direct and immediate benefit to the respondents. For instance, it would not have been apparent to the resident whether submitting the reports would help be followed by improvement actions or policy changes in the near term, or not.

6 Development and Towards Implementation of RETIPS-Airway Management

In line with the feedback provided by experts and clinical leaders earlier, a third version was developed which focused on a specific issue in anesthesia – airway management or difficult intubation. The previous version of the tool was adapted by replacing examples with those relevant to airway management and modifying questions and response choices to be more specific to the clinical issue. These modifications were made by involving anesthesiologists and nurse anesthetists over multiple iterations to ensure relevance and coherence of the content.

The original strategy of the authors was to incorporate the RETIPS-Airway management form into the hospital’s existing event-reporting portal. The portal consists of separate forms for various safety issues and clinical areas, such as patient falls and blood transfusion. The idea of RETIPS as a way to learn proactively about how things go well in everyday work was supported by the Vice President for Health Care Quality at the hospital. However, an important question that emerged in terms of implementation was what resources would be required to administer, maintain and process reports once they would be generated, and how would such a project be funded. The authors assured the leadership that no additional resources would be necessary if the tool were to be incorporated within the existing portal infrastructure, and that reports would initially be used primarily for analysis and research purposes. Additionally, a significant logistical issue was encountered in that the portal had a set template for forms, which could not accommodate the structure and full content of RETIPS. Therefore, as a compromise, the authors had to relinquish the idea of including RETIPS-Airway Management as a separate form within the portal. Instead, we included two key questions from RETIPS to the portal’s existing airway-management form meant for adverse event or near-miss reporting. The questions focused on expanding beyond the specific event to describe how the process usually goes well: (1) “Context beyond this incident, what usually goes well?” and (2) “Please use the text box to describe workflows, decisions, and factors that enable effective airway management and risk prevention under the usual circumstances.” Technical constraints inherent to the portal’s design meant that responses to these questions could not be made mandatory for form submission. Only five responses to the questions were received out of 45 reports over the span of 9 months, July 2018 to April 2019. None of the responses provided information about care under usual circumstances or what usually goes well, but instead amplified information about the event being reported.

7 Reflections on the Overall Experience of Applying RETIPS in a Hospital: What Went Well and What We Learned

As mentioned earlier, the design of the tool itself was effective in terms of knowledge elicitation about everyday variability in performance, and the key factors that contribute to the challenges and successes of performance.

Buy-in from leadership, engagement with key stakeholders: A strong and sustained intent from the hospital’s clinical and safety leadership is important for driving a new initiative, more so if it involves a shift in thinking. Therefore, it is necessary to communicate, not just the idea of RETIPS, but the approach to learning that it represents, to various levels of leadership in the hospital. In our effort to implement RETIPS hospital-wide, we met with the Vice President for Health Care Quality at the hospital and described the idea of a tool designed to learn proactively about ‘normal’ work, rather than just adverse events; they were immediately agreeable to exploring how the tool could be implemented. This meeting led to further discussions with other key stakeholders in the organization, such as the Clinical Manager for Perioperative Education, who reports to the Associate Chief Nurse for Perioperative Services. The endorsement of high-level administrators and managers enabled us to reach out to other key stakeholders, such as technical and administrative staff, whose support was imperative to operationalizing RETIPS. We do not have an answer yet to the best possible way to get staff to engage on a wide scale in the learning process. However, communicating the ideas demonstrated in the tool to stakeholders at multiple levels could be a useful way to start. We have found that the idea of RE and learning from how things go well is a simple yet compelling concept. There was no disagreement encountered at the conceptual level. However, this does precipitate questions on the ‘how-tos’ regarding operationalizing the concept.

Using Organizational ‘hooks’ to operationalize the tool: Our approach was not to propose a replacement to existing reporting and learning systems at the hospital, but to influence existing workflows and learning pathways in the organization. To this end, we identified existing processes in the hospital and department as organizational ‘hooks’, to ‘latch’ RETIPS onto. Again, communicating to key stakeholders, the purpose of the tool and what we were looking for in terms of implementation, was crucial in identifying such hooks. For instance, in order to pilot the tool, we spoke to the residency supervisors in the Anesthesia department. In turn, they suggested the quality improvement course as a forum to introduce RETIPS to the residents through their curriculum. At the hospital level, the existing event reporting portal was identified as a platform to introduce questions from RETIPS. Similarly, appropriate organizational hooks could be identified for integration of the tool or its parts in concert with clinical and administrative groups at departmental and hospital levels.

Incentive to respondents: In our experience, offering extra course credit for submitting reports seemed to help generate a relatively high response rate (13 reports from a total of 12 residents) within the course participation. In general, however, obtaining a healthy response rate for meaningful analysis remains a significant challenge. When implementing the tool more widely, beyond the ACGME course work, it may be useful to offer a suitable incentive to targeted respondents, especially initially, in order to drive responses. Promotion by senior leadership or supervisors is essential.

Confidentiality: RETIPS reporting should be confidential in nature so as to protect the identity of respondents. Confidentiality is important to help respondents feel secure about discussing any potentially sensitive aspects of their work, including risks and informal workarounds. Confidentiality, rather than anonymity, would also enable analysts and investigators to follow up with the respondent to gather additional details related to their report, and engage them in any subsequent improvement efforts. This might have contributed to responses from residents. We did receive more than the minimum number.

Follow-up, analysis, and feedback: A key factor in sustaining engagement of the workforce with reporting is communicating the outcome or impact of the reports to the respondents. When the analysis and its ensuing decisions are made visible to the organization, staff are motivated to continue to report as they see their responses as being impactful on their environment. This feedback loop should, therefore, be an essential part of the larger organizational learning framework of which RETIPS itself would be a part. Furthermore, the exemplar responses submitted through RETIPS do not provide all details to understand a work practice or the environment, but can be used as triggers for further investigation by safety administrators, senior management, hospital analysts, and even clinicians. In addition, these examples could be used to seed a survey asking for an expansion of the list of examples to a wider audience. This has the virtue of not requiring someone to complete it immediately after cases and may result in richer input since respondents should not be as time constrained. While we have not done this in our implementation so far, the visibility of information from individual reports back to the professional community has been acknowledged as one of the main factors underlying the sustained success of the Aviation Safety Reporting System (ASRS) (Cook et al., 1998).

In addition to the above reflections, we offer the following takeaway from our experience: In order to successfully implement RETIPS, it was important for us to identify networks of people and agencies at various organizational levels, who can influence and/or may be influenced by the change. Identifying the network also entails understanding the relationships in these networks. In this regard, relevant questions to consider include: who works for whom (hierarchical and lateral dynamics), what are their usual responsibilities and scope, how are resources shared, what would it require people to do in order for the proposed changes to be implemented, and are there perceived benefits? Engaging with various levels of the workforce helped us not only understand the system and its networks better, but also to communicate our ideas more effectively. Among other benefits, this process of continuous engagement and dialogue enabled the identification of the aforementioned organizational hooks, which are key to implementation. This approach follows from the core RE principle of learning about normal work to implementing changes that blend with the flow of normal work.

8 RETIPS in the Context of a Larger Organizational Learning Framework

RETIPS is designed as an artifact of the ‘new thinking’ towards proactive learning in resilient systems. The organizational hooks mentioned earlier are but a few ways in which RETIPS was introduced into the hospital’s learning processes. Going forward, RETIPS could be part of a larger organizational learning framework, complementing other information-sharing pathways by emphasizing the focus on learning proactively about how things go well in everyday work. Other organizational hooks that could be opportunistically leveraged include:

  1. 1.

    Departmental Morbidity and Mortality (M&M) Conferences: These could be used to describe and discuss examples of everyday performance variability, including those identified through RETIPS.

  2. 2.

    Combined Safety Grand Rounds: This is a forum where the entire Perioperative community (periop services, surgery, anesthesia, ortho surgery, OB/GYN) come together for a lecture or activity on patient safety. Some of these could be devoted to clinical resilience, and may be a way to incentivize staff to provide ‘stories of resilience’ that would be called out in this forum.

  3. 3.

    Simulation: Debriefs during simulation exercises could be focused more on what went well rather than just areas for improvement in the technical and nontechnical work so that individuals and teams can discuss aspects of their actions and workflows which contribute to safe patient care.

  4. 4.

    Lecture: Numerous opportunities are available, especially at teaching hospitals, with residents, staff, and students to introduce them to concepts of resilience and how they relate to everyday clinical work, quality of care, and patient safety.

As mentioned in the ‘Reflections’ sections earlier, the leadership would have to be involved in facilitating reporting and conversations with RETIPS through the existing learning pathways. Appropriate incentives can be identified based on the type of staff (e.g., residents, attendings, nurses, technicians), area or clinical specialty, organizational level, and other factors. The course credit for residents used in our pilot implementation is an example. Other incentives can include formal recognitions and awards. Data from various areas of the hospital can be analyzed under a common analytic framework with a ‘resilience lens’. Such analysis would involve identifying patterns, including adaptations, resource usage, and communication. These patterns further inform the recognition of resource needs, process redesign requirements, policy changes, etc. for various concept-driven goals, such as reducing brittleness and improving adaptive capacity.

9 Conclusion

RETIPS, a previously developed lesson-sharing tool based on Safety-II and Resilience Engineering principles was revised and reconfigured, based on feedback from clinicians, in terms of relevance and practicality within the hospital setting. The revised version of the tool, RETIPS-AnRes, was disseminated on a pilot basis to anesthesia interns as part of the curriculum of a one-week course on quality improvement. The implementation validated the design in that the responses were aligned with the purpose of the tool, which was to learn about how things go well in everyday clinical work. Further, the tool was adapted for a specific clinical issue – difficult intubation and airway management. Reflecting on the overall experience of implementing RETIPS, we summarize key takeaways for operationalizing the tool in hospital settings. This work demonstrates the potential for RETIPS as a means for proactive organizational learning in healthcare, widening the focus beyond adverse events and near misses. The potential for wider and longer-term implementation of RETIPS within a larger organizational framework for learning about resilience in frontline medical work is also discussed.