Introduction

With increasing evidence on their benefits, electronic systems aimed at remotely monitoring and/or managing treatment-related symptoms seem to have acquired a solid place in the supportive care of patients with cancer. These systems have shown to improve patient care, e.g., communication between healthcare professionals (HCP) and patients and symptom management, and some studies have even demonstrated they positively affect patient outcomes such as quality of life and survival of patients with advanced cancer [1,2,3,4].

A systematic review of electronic systems to report and manage side effects of cancer treatment showed important variation regarding the features offered by these systems, with some being primarily focused at symptom monitoring and others at improving symptom management and/or communication [5]. To improve symptom management, some systems provide self-management advice while others (also) send e-mail alerts to HCP when patients report a severe and/or worsening symptom. Basch et al. demonstrated important clinical benefits of a Symptom Tracking and Reporting tool (STAR) for symptom monitoring and hypothesized that nurses’ early responsiveness to patients’ severe and alert-triggering symptoms prevented adverse downstream consequences and might (partly) explain the positive impact on patients’ quality of life and survival [1, 2].

While up to 17 out 41 systems reviewed by Warrington et al. offer a similar alert-triggering feature [5], there is limited report on the feasibility of the feature and the relevance of remote symptom management. Addressing the cost and workload of the alert-triggering feature of mobile symptom-monitoring tools, Basch et al. described only 1.7% of symptom self-reports concerned severe or disabling symptoms generating an alert trigger to a healthcare professional [2]. Alerts mostly concerned fatigue, pain, anorexia, dyspnea, neuropathy, and nausea. Nurses provided telephone counseling in the vast majority of cases. Sometimes, their management included supportive medication initiation/change, referral to the ER/hospital, chemotherapy dose modification, or imaging [2]. In a study of the mobile phone-based Advanced Symptom Management System (ASyMS©), nurses expressed mixed views on the alerting facility of the system. While the nurses in the study had experienced the role of alerts in early detection and prevention of symptoms, some alerts were considered too mild and unnecessary [6].

Some features of remote symptom-monitoring systems may have conflicting objectives or may create contradictory user expectations. Several systems, like the digital diary used in our hospital, are primarily focused at improving patient self-management, offering patients (automated) advice on symptom management. In this spirit, severe symptoms trigger automated feedback to patients advising them to take contact with a healthcare professional (HCP). While the feature of alerting HCP is known to be reassuring from the patient perspective [7], the feature might impact the self-management objective of a remote symptom-monitoring tool, getting patients to lean on the monitoring and management provided by the system or their HCP.

This ambivalence is challenging for HCP and cancer centers considering the implementation of a remote symptom-monitoring system and an alert-triggering feature in particular. While Kessel et al. found a majority of German HCP to be in favor of the use of mobile apps in oncology, opinions about the alert-triggering feature were divided [8]. Legal uncertainty regarding medical responsibility and data privacy were the largest concerns of those HCP not in favor of apps for cancer care. When a remote symptom-monitoring system was implemented in our hospital, the alert feature raised concerns about workload, feasibility, and utility. Therefore, we set up a study in the early phase of its implementation to explore the uncertainties of the alert feature. First, we aimed at exploring the process and feasibility of the alert feature and alert management. Second, we aimed at exploring the clinical relevance of real-time symptom management as perceived by oncologists and oncology nurses, in other words the acceptability of the alert feature and the alert management intervention.

Methods

Design

A prospective process evaluation was set up to explore the process of alert management as a complex intervention. By examining implementation, mechanisms of impact, and context, the purpose of process evaluation is to provide more detailed understanding of a complex intervention and ultimately, to inform policy and practice [9]. Key questions at this stage of implementation were the following: what is the incidence of alerts and what symptoms trigger these alerts, to what extent do patients reporting severe symptoms adhere to the automated feedback to contact a HCP and to what extent is HCP-initiated real-time symptom management needed, what type of symptom management is needed in case of alerts, is nurse-led alert management feasible?

Additionally, we set up a cross-sectional survey to explore HCP opinions on the clinical relevance of real-time symptom management. By collaboratively engaging these key stakeholders at this stage, we aimed at better estimating their support for this alert feature and at maximizing the credibility of the intervention [10].

Setting

The study took place between January and September 2019 at two oncology daycare centers and two oncology wards of the University Hospitals Leuven, a large academic hospital in Belgium. At the time of this study, all patients with lung cancer, multiple myeloma, a tumor from the gastro-intestinal tract, or non-metastatic breast cancer who were starting their first ever systemic therapy (i.e., chemotherapy, immunotherapy, targeted therapy, but not antihormonal treatment) were offered to use the mobile tool at the start of their treatment.

The tool

Based on our paper-pencil diary, we implemented a mobile and web-based diary for remote monitoring symptoms during systemic anticancer treatment (i.e., chemotherapy, immunotherapy, targeted therapy, but not antihormonal treatment). The digital diary enables (daily) self-monitoring of treatment-related symptoms but unlike the paper-pencil diary, the symptom set presented to patients can be adjusted to patients’ treatment. The basic set of displayed symptoms comprises the following: nausea, vomiting, anorexia, diarrhea, constipation, fatigue, pain, skin rash, dyspnea, psychosocial burden, and fever, but nine other symptoms (oral mucositis, peripheral neuropathy, tearing eyes, hearing loss, hand-foot syndrome, itch, myalgia and arthralgia, cough, hypertension) as well as the intake of oral anticancer medication can be added upon patients’ treatment. The diary is available in the mynexuzhealth application, a secure web application for patients to access their medical record (reports, appointments, invoices, radiology images, questionnaires, and diaries) online via the website and/or mynexuzhealth app (https://www.nexuzhealth.be/en/mynexuzhealth).

Next to the symptom-monitoring feature of the digital diary, the digital diary has some additional features. First, it is integrated in the electronic patient file. A summary and all completions are visible in the nursing module and the mynexuzhealth module of the patient file to facilitate HCP’s use of patient-reported outcome measures (PROMs) for symptom management. Second, self-reports trigger automated feedback with self-management advice for patients. This feedback is displayed immediately after the patient’s self-report and shows standard self-management tips and the invitation to read more at the hospital’s website. In case of severe symptoms (grade 3), the feedback calls patients to take immediate contact with a HCP. Finally, severe symptoms also trigger an automated alert to a healthcare professional at the hospital who can contact the patient for real-time symptom management. These alerts are the subject of this study. The selection of symptoms triggering alerts was made by the researchers on the basis of the relevance of real-time management to prevent critical toxicities and/or to improve quality of life. Next, fourteen doctors and seventeen advanced nurse practitioners from a variety of oncological disciplines participated in this selection process by making a clinical estimation on the relevance of an alert function for the digital diary’s symptom set in an online survey. As a result, the following symptoms triggered automated alerts: fever (≥38°), hypertension (blood pressure higher than 160/100 mmHg and/or an increase of diastolic blood pressure by 20 mmHg), an aggravation of dyspnea and/or cough by two grades, and two consecutive days with grade 3 of nausea, vomiting, diarrhea, constipation, pain, myalgia and artralgia, dyspnea, oral mucositis, and cough. A project team of one clinical nurse specialist (AC) and two nurse consultants (KM and EV) who were, at the time of the study, in charge of the implementation of the tool, the clinical management of the alerts, and the conduct of this study received an e-mail notification about the alerts. On top of that, all alerts landed at a login-protected overview module of the electronic patient record (EPR) system. The nurses handled the alerts during day-time working hours, 7 days a week. Late-night or night-time alerts were managed the next morning.

Sample

To evaluate the prevalence and characteristics of the alerts, we prospectively registered and collected data from all alerts generated by the system between January and September 2019. To evaluate the clinical relevance as perceived by HCP, we invited all medical staff members (n=12) and advanced practice nurses (n=10) clinically involved with the patients under the scope of the tool.

Data collection

A structured data collection form for reporting the characteristics and management of every alert was constructed for the purpose of data collection in this study. This form included basic characteristics and context of the alert: the symptom, the oncological discipline of the patient, the execution of a telephone contact (and reasons not to), information on treatment protocol, date of latest treatment administration, date of alert, date of next hospital appointment. Next, it provided description of the nursing assessment and management of the alert: the severity, frequency, and interference of the symptom and any related symptoms; the patient’s self-management and response to the alert (pharmaceutical, non-pharmaceutical, contact with HCP); the nursing management of the alert (e.g., referral to ER, referral to GP, pharmaceutical and non-pharmaceutical advice, time spent on the alert management). Most items were set up as multiple choice questions to facilitate data collection and analysis. Free-text space was provided to allow rich description of the assessment and management of the alerts. For every alert in the data collection period of this study, the nurse in charge of the alert completed the form and reported the alert in the electronic patient record.

Data collection for the process evaluation of the alert management was entirely based on this form, exploring the number and characteristics of the alerts, and the scenarios for patient self-management and nursing management of the alert. To evaluate the clinical relevance of the real-time management of the alerts, as perceived by the medical and nursing staff, online surveys were set up presenting clinical reports of the alerts and its management. Oncologists and advanced practice nurses in oncology were invited to participate in these online surveys by e-mail. A reminder was sent after 2 weeks. Each online survey presented several anonymous cases. Online surveys were set up per oncological discipline, presenting only cases from their own patient population. Information for the case descriptions was extracted from the structured data collection form and the nursing report. Case descriptions included characteristics and context of the alert, patient self-management of the symptom, and the nursing assessment and management of the alert. The selection of alerts was based on chronological order although the research team decided to skip cases when necessary to avoid overlap and to have sufficient variety regarding the symptoms triggering the alerts. Per case, respondents rated their level of agreement with 7 self-constructed statements on the clinical relevance using a 5-point Likert scale (from 1 = totally agree to 5 = totally disagree). The main statement was the following: “To what extent do you find the real-time management of this alert (by a telephone consult) was clinically relevant?” Four statements questioned HCP about the role of the alert management on the patients’ course of treatment and quality of life. Finally, two statements concerned their estimation of the symptom being potentially critical and the preventability of the alert.

Data analysis

Process evaluation data were analyzed using descriptive statistics, i.e., numbers and proportions of the alert characteristics and alert management scenarios. Clinical relevance scores were analyzed descriptively using mean (SD) and median score (IQR) for the seven statements. We also analyzed mean relevance scores (SD) per type of HCP, per oncological discipline, and per symptom to better understand any differences important for the future improvement of the alert management intervention. In order to explore whether perceived clinical relevance significantly differed between these groups (i.e., type of HCP, type of oncological discipline) or between symptoms, we compared the mean relevance scores between groups and symptoms using unpaired t-tests (in case of 2 groups) and one-way ANOVA tests (≥ 2 groups). We used IBM Statistical Package (SPSS v19) to perform data analysis.

Ethics

The study protocol was approved by the Ethics Committee of UZ/KULeuven. The dataset and questionnaire of the alerts only contained pseudonomyzed data. Only the researchers had access to respondents’ identity and individual code. HCP responding to the questionnaire did not sign a formal paper-pencil informed consent, but checked their consent and voluntary participation as part of the online survey.

Results

Characteristics of the alerts

During the period of data collection, a total of 253 alerts were notified. During the same period and based on the EPR data, a total number of 17,972 self-reports were submitted by 507 unique patients. Thus, patient-reported toxicity triggered an alert in 253 out of 17,972 reports, bringing the incidence rate of alerts at 1.4%. The characteristics of all alerts are reported in Table 1. Two-thirds (60.5%) of the alerts concerned patients with a primary tumor of the gastro-intestinal tract. Notified alerts were mostly about pain (n=68), fever (n=57), dyspnea (n=44), and nausea (n=41).

Table 1 Characteristics of the alerts
Full size table

Process evaluation of alert management

Not all alerts were answered with a telephone call from the project team: more than half of the alerts (56.1%) did not require real-time management, e.g., because symptom burden was known and had been responded several times before with no new symptom management strategies possible (n=42, 30.0%) or because the alert concerned a delayed symptom report and more recent self-reports showed symptom relief (n=29, 20.4%). Other reasons for not responding are reported in Table 1. On the other hand, 111 out of 253 alerts (43.9%) were responded with a telephone call by the project nursing team. In 7 cases, the nurses learned that the patient had already contacted a healthcare professional without track of the contact in the patient file. Five patients were surprised by the call and said the high severity of the symptom was reported by mistake.

Of 99 patients called for real-time symptom management, 44 (44.4%) expressed no intention to contact a HCP at all, despite the automated feedback to do so. Eighteen (18.2%) had contacted their general practitioner and ten (10.1%) expressed their intent to talk about the symptom during their next hospital visit. More than half of the patients (n=58, 58.6%) had taken their prescribed medication to manage their symptom.

In 69 of these 99 alerts (69.7%), real-time symptom management consisted of the provision of self-management advice and 45 (45.5%) patients who expressed some extent of resistance to the given advice were coached using motivational interviewing. Thirty-nine patients (39.4%) were referred to their general practitioner and 15 (15.2%) to the emergency department. Adjustment and start of supportive medication were advised in 17 (17.2%) and 12 (12.1%) of the cases respectively.

A telephone consult was considered to suffice for the management of the symptom in 62 (62.1%) of the cases. In 25 (25.3%) of the cases, a clinical examination was considered necessary. Twelve cases (12.1%) could be managed by telephone but required a medication prescription.

Regarding the time spent to the real-time management of the symptom, this was between 5 and 15 min for the majority of the cases (61.6%). In one out of four cases (27.3%), the nurse spent 15 to 30 min to manage the alert.

Clinical relevance of alert management

Ten out of 12 invited oncologists and all ten invited oncology nurses participated in the online survey, resulting in a response rate of 90.9%. A total of 31 cases were presented to the clinical teams using online survey. The team of digestive oncology was asked about 14 cases, followed by the team of respiratory oncology, assessing 10 cases. Most cases concerned fever (n=7), dyspnea (n=6), nausea (n=5), and pain (n=4). There were 156 assessments in total.

Relevance scores for the main item are reported in Table 2. Overall, the clinical team (totally) agreed with the clinical relevance of the real-time management in 148 of the ratings (95.1%). The mean relevance score attached to the cases was 4.51 (±0.80). Comparing the ratings of nurses and physicians, nurses rated the relevance slightly higher than physicians (4.66 compared to 4.30) and this was statistically significant (p<0.001). There was no statistical significant effect based on oncological discipline or type of alert.

Table 2 Global clinical relevance of real-time alert management as rated by oncology professionals on a 5-point Likert scale from 1 = totally disagree to 5 = totally agree (156 ratings for 31 cases)
Full size table

Clinical relevance ratings of all items assessed by the HCP are shown in Table 3. The statement that the real-time management improved quality of life of the patient got the highest level of agreement (4.38, ±0.77). This was closely followed by the statement that without real-time management of the alert, the symptom would have a negative impact on the patient’s quality of life (4.36, ±0.95) and the statement that real-time management of the alert promotes the favorable course of treatment (4.28, ±0.91). The statement that the alert could have been prevented by well-adjusted supportive care got the lowest level of agreement (2.72, ±1.16), followed by the statement that without real-time management, the symptom would be critical (3.72, ±1.42).

Table 3 All items indicating clinical relevance of real-time alert management as rated by oncology professionals on a 5-point Likert scale from 1 = totally disagree to 5 = totally agree (156 ratings for 31 cases)
Full size table

Discussion

In considering the implementation of remote symptom-monitoring systems, the feature of triggering alerts to healthcare professionals may at the same time be an opportunity and a concern. Less than half of current applications dispose of this functionality [5] but overall, there is limited report on the clinical impact and utility of the feature. In order to contribute to the insights into this particular feature of electronic systems for patient support, this study explored the clinical characteristics and process of the alert feature as well as the clinical relevance of alert management as perceived by HCP. Overall, the majority of alerts triggered by our mobile tool for remote symptom-monitoring during cancer treatment could be managed by a telephone nursing consult and real-time management was evaluated as (very) relevant by the majority of clinicians.

During the study period of 9 months, 507 unique patients used the digital tool and self-reported 17,972 times. Alerts were triggered 253 times during this period (1.4%). This is in line with the results of the randomized study of Basch et al. [2] that showed less than 2% of symptom reports triggered an alert. Furlong et al. [11] reported averages of 1.25 to 2.4 alerts per patient, dependent on cancer diagnosis. Given our observation that today, only half of the patients engage in self-monitoring using the electronic tool (Coolbrandt et al. in progress) [15]; over time, the number of alerts could potentially double.

Pain, fever, dyspnea, and nausea were the most common symptoms triggering an alert in our study. It is important to note that symptoms triggering alerts differ between different tools. While fatigue, anorexia, and neuropathy were among the most prevalent alert-triggering symptoms in the study of Basch et al. [2], these symptoms (alone) do not trigger an alert within our electronic tool. Instead, they are managed at the next hospital visit or during patient-initiated telephone consultation. While remote symptom management support can be valuable for any bothersome symptom, workload concerns prompted us to carefully develop the alert management protocol. The alert algorithm reported in this study was defined after consultation of HCP, i.e., doctors and nurses, and these three symptoms failed to reach the threshold for triggering alerts and associated real-time patient contacts. In their ASyMS intervention, Furlong et al. used not only red (i.e., severe or life-threatening) but also amber alerts. Amber alerts were used when patients’ symptoms were bordering on becoming problematic and were responsive to early preventative interventions. [11] Remarkably, the management of amber alerts took on average 38 min compared to 15 min for red alerts, as in our study. Further study, providing more empirical data and exploring the perspectives of all stakeholders could serve at determining a gold standard alert protocol. While patients have been consulted in the selection of symptoms for self-monitoring [12], they have not been involved in determining our alert algorithm.

Cancer diagnosis seems to markedly impact the prevalence of alerts. Alerts were mostly generated for patients with colorectal cancer, and three times less in patients with breast cancer and lung cancer. This analysis does not take into account the total numbers of patients using the mobile tool. However, our monitoring of patients’ acceptance of the tool (Coolbrandt et al., in progress) [15] showed a similar number of patients with colorectal and breast cancer using the tool. Contrary to our findings, Furlong et al. [11] observed about the same number of alerts (i.e., almost 2.5 per patient) among those with colorectal cancer and breast cancer, while patients with hematological cancers accounted for half of this number [11]. As this may substantially influence the workload of alert management among different oncological disciplines, further research into the incidence of alerts in relation to cancer diagnosis is recommended.

Self-management support and coaching were the most frequently applied nursing interventions to handle an alert in our study, followed by referral to the GP. Two-thirds of the alerts could be handled by a nursing telephone consult alone. Basch et al. [2] reported that almost 80% of the alerts were handled by telephone counseling [2]. As for the study of Basch et al. [2], initiation or adaptation of supportive medication as well as referral to the emergency department was less frequently applied. Assuming the availability of intervention protocols and sufficient clinical expertise, our results support the nurse-led management of symptom alerts.

Regardless of the system’s automated feedback to contact a HCP, only a minority of patients had consulted a HCP at the hospital or in primary care. Mobile health technology may contribute to patients’ autonomy, but paradoxically it can at the same time add to patients’ level of dependence [13, 14]. While the alert feature enables to intercept cases of severe symptoms that otherwise would remain unmanaged (before getting even more critical), the ultimate goal is to improve self-management. The ratio of patient- versus nurse-initiated telephone consultations can be considered an interesting parameter for evaluating a tool’s efficacy in improving self-management. As in the study of Judson et al. (2013) and for ethical reasons as well as for legal reasons, patients in our center are informed that self-reported symptoms are reviewed and managed at the subsequent visit and that patients should contact a HCP when symptoms are worsening, persisting, or concerning. Further research is needed to understand patients’ non-compliance to automated feedback urging them to take contact with a HCP and improve patient self-management of severe symptoms during cancer treatment.

Our process evaluation study did not include any data on cancer type, treatment type, cancer stage, and patient characteristics. Therefore, we cannot relate the number and type of alerts to any of these characteristics. A fully digitalized electronic form is now being used for the routine management of the alerts and will facilitate further study on this topic. The major limitation of the study is that the clinical relevance of the alerts was assessed by a relatively small sample of clinicians from one hospital and those oncological disciplines who were in favor for the implementation of the mobile tool. It is recommended to set up such evaluation at a larger scale and for an extended number and variety of symptoms.

Conclusions

A large number of remote symptom-monitoring systems have been reported in cancer care, with great variety in the features they offer. While almost half of these symptoms trigger alerts to HCP for serious symptoms, there is little known about the feasibility of this feature. Based on our study, we conclude that the majority of alerts can be managed by a telephone nursing consult and that real-time management is evaluated as (very) relevant by the majority of clinicians. While symptoms triggering an alert to HCP also generate automated feedback to patients to contact a HCP, this advice is mostly not followed. This adds to the relevance of clinician-initiated real-time symptom management but at the same time calls for actions to improve adequate patient-initiated consultation.