Keywords

1 Introduction

The current global emergency SARS-CoV-2 (Covid-19) has put enormous pressure on intensive care units (ICU) in all health systems around the world, due to the rapid and exponential growth of new patients requiring intubation and mechanical ventilation.

The contribution of pulmonary ventilators in an ICU is crucial for a high level of care for a critical patient and, now more than ever, in light of the current global COVID-19 emergency.

This is the reason why intensive care units are rapidly populated with a wide variety of pulmonary ventilators that differ from each other in terms of type of model, graphic interface, commands, management modes and functions.

For this purpose the usability of the medical device and patient safety are essential requirements, but many errors in healthcare occur because MDs are difficult to use [1] or do not respond to the user’s mental models [2].

The major scientific contributions on error in healthcare pointed out that deficiencies in the design of the human-machine interface, both physical and digital, can create the conditions for errors to occur, especially in emergency and unusual conditions [3,4,5]. Furthermore, incorrect procedures, stress, fatigue and a wide range of medical devices used by the operator during his/her working day can further affect the individual performance.

Scientific evidence [6, 7] shows that one of the most serious consequences of the spread of COVID-19, together with the daily discomfort that health workers are called to face (e.g. management of emergencies, stressful shifts, availability, staff shortages, situation of extreme distress) is that pandemic is putting a strain on all health professionals, whatever their task is. They are the first line defense against an emergency of enormous magnitude, which significantly affects workloads, physical fatigue and psychological health [8].

The motivations above led the researchers to conduct a usability study, using the Post-Study System Usability Questionnaire (PSSQU) method, for 3 models of pulmonary ventilator most commonly used in ICUs of some hospitals in Tuscany.

As suggested by ISO 9241-210:2010, the usability informs about how effective, efficient and satisfactory the use of a device is for the user [9], even under emergency or unusual conditions.

2 Methodology

A focus on the methodology approach is shown in this paper.

The study involved 30 anaesthetists of varying age, experience and gender, contacted through a telephone recruitment campaign and by e-mail (see Table 1).

Table 1. Characterization of subjects
Full size table

The hospitals at issue are: AUSL Toscana Sud Est, AUSL Toscana Nord Ovest, AUSL Toscana Centro, Azienda Ospedaliera Universitaria Careggi e Azienda Ospedaliera Universitaria Pisana.

For this study, we considered the 3 most commonly used types of pulmonary ventilator in ICUs (Servo I Maquet, Gentinge AB; Hamilton C6, Hamilton Medical AG; Hamilton G5, Hamilton Medical AG), 1 of them recently introduced (Hamilton C6).

It is necessary to specify that many ICUs are equipped with different models of pulmonary ventilators, each one with its own management, assembly and interaction, and that many of the users reported to use several medical devices simultaneously during the working day (see Table 1). This can lead to errors and operational inefficiency by even the most trained and experienced staff. Consequently, considering the total of 30 subjects who took part in the survey, 14 used Hamilton C6, 13 used Hamilton G5 and 18 used Servo I.

Usability testing is probably one of the most common methods to assess the usability of a product [10]. Nevertheless, social restrictions, travel limitations and containment measures imposed by COVID-19, made it impossible to observe the users interaction with pulmonary ventilators directly. Not to mention the significant workload that each operator had to and still has to face every day to ensure high care levels. For all these reasons, a case study was conducted using an online questionnaire.

The PSSQU was submitted to the operators using the Google Forms platform, in anonymized form, from June 2020 until September 2020.

In order to receive reliable answers, each participant was asked to fill the PSSQU after using the pulmonary ventilator during his/her work shift.

This tool allowed to assess user satisfaction regarding 4 different dimensions of usability: Overall PSSQU (user satisfaction, the average scores of questions 1–19), SYUSE (system usefulness, the average scores of questions 1–8), INFOQUAL (information quality, the average scores of questions 9–15) and INTERQUAL (interface quality, the average scores of questions 16–18). At the end of the questionnaire each user was asked to describe what they would like to change about the ventilator(s) in their possession.

2.1 Post-study System Usability Questionnaire

To quantify the usability levels of medical devices the Post-Study System Usability Questionnaire (PSSQU) was used [11].

In scientific literature several questionnaires have been conducted for usability measurement of product/service from the user point of view, such as the Software Usability Measurement Inventory (SUMI) [12], the Website Usability Measurement Inventory (WAMMI) [13], the Computer System Usability Questionnaire (CSUQ) [11], Post-Study System Usability Questionnaire (PSSUQ) [11], and the System Usability Scale (SUS) [14].

These questionnaires help researchers and human factors experts to understand the usability of medical devices, revealing users’ perceptions of outcomes and interaction [15] with it.

Retrospective usability evaluation requires less expanse and less effort from both users and researchers. Users are not required to complete any tasks but rather they are expected to integrate all their previous experiences with the device and provide their own evaluation.

This retrospective method can be conducted with any validated usability questionnaire, such as the PSSQU [16,17,18].

The PSSQU tool developed by Human-Computer Interaction expert of IBM, originally called System Usability Metrics (SUMS), is a questionnaire composed of 19 questions [16], where the user is asked to answer each question on a Likert scale from 1 to 7 (1 representing completely agree while 7 completely disagree, and one point not applicable (N/A) outside the scale), where the lower is the answer, the higher is the subject’s satisfaction with the analysed system. It is also useful in order to measure the usability of the system, including performance, usability problems and end-user satisfaction.

3 Results

Results are reported below.

Overall, the PPSQU score global values (OVERALL) show a high general usability. The results therefore indicate a positive reliability of the pulmonary ventilators on object, even if the PPSQU score global value is less than 60% for 4 subjects out of 30 interviewed. Respectively the score is 6.32 (11.40%) for the first subject, 4.26 (45.61%) for the second subject, 3.84 (52.63) for the third one and finally for the forth one the score is equal to 3.42 (59.65%), (see Table 2).

Table 2. Users and ventilators: the PSSQU score global values.
Full size table

As it can be observed in Table 3, the interface quality (Infoqual) of the three lung ventilators analysed had the worst results, considering the independent subscales. In fact, at the end of the PSSQU, the participants were asked the following question “What would you like to change of the pulmonary ventilator if you could?”. 12 out of 30 answered to this question and they all declared that the most critical issues are related to the digital human-machine interface.

Table 3. Ventilators: the PSSQU score global values.
Full size table

Users suggest that a clearer, more efficient, less cluttered interface would allow easier interaction with the device. They also complained about the lack of detailed error messages. Moreover, weight and dimensions of these products, and some of their auxiliary functions led to difficulties in using the equipment.

4 Conclusion

In this study we evaluated how the PSSQU could be a replicable tool for different ventilator models currently in use in some Tuscan hospitals, also effective for measuring the usability of pulmonary ventilators (performance, usability problems, user satisfaction).

Feedbacks from physicians, although positive on average, highlight the need for clearer and more detailed error messages, more usable and less cluttered graphic interfaces, lighter and more intuitive mechanical ventilators during assembly and disassembly of components.

Our study aimed to test the PSSQU as a tool for evaluating the usability of medical devices in the anesthetic and resuscitation fields for possible ergonomic interventions.

In conclusion, usability studies such as the methodology reported in this article are necessary to evaluate the design deficiencies of the device. This will reduce user errors and potential damage, saving time, money and resources in the long term. The reliability of the PSSQU tool, collected from the various studies [16,17,18] and from the present case study, suggests that the adoption of such a tool in risk management processes can highlight any usability problem and constitute an important part of the production process of future medical devices. Prevention is in fact obtained intervening in the project before the harmful event occurs, using appropriate forecasting methodologies [19,20,21].