Introduction

Aneurysmal subarachnoid haemorrhage (aSAH) is a form of haemorrhagic stroke that occurs due to the rupture of an abnormal blood vessel in the brain. It is distinct from other forms of stroke in terms of the population it affects (younger and more likely to be female), its pathophysiology, and long-term sequelae [1, 2]. There is a mortality rate of 35% (range 20–67%) [3] and of those who do survive, most are left with long-term morbidity [4, 5].

A lack of consistency in the outcome measures used to evaluate aSAH treatments directly impacts researchers’ and clinicians’ ability to compare and aggregate trial results which, in turn, reduces research efficiency [6]. Over the past 20 years, 285 unique outcomes have been reported in 129 randomized controlled trials involving SAH patients [7]. Multiple definitions and measurement instruments are used to characterize frequently reported domains such as assessment of function (> 75% of included studies) where 10 distinct instruments were used. Patient-reported quality of life measures were rarely reported (< 10% of included studies) but, again, seven different instruments were employed to measure it. Finally, trials measure and report these outcomes at variable time points rendering it difficult to reliably assess recovery [7].

There are also mismatches in the outcomes used by researchers relative to what is considered important by patients, family, and health care providers [8, 9]. Indeed, studies have shown discrepancies between what those with lived experience of aSAH and health care practitioners consider important when determining the quality of a patient’s outcome following aSAH, and between members of different health care specialties themselves [10, 11]. Patient engagement initiatives have emphasized the importance of aligning the priorities of patients, clinicians, and researchers in the promotion of stroke recovery [12, 13].

To address the lack of consistency and help align outcome selection with stakeholder preferences, an international consortium of people with lived experience of aSAH, researchers, and health care professionals is developing a core outcome set (COS) for aSAH [14]. A core outcome set is a limited set of outcome measures that all studies in a particular clinical area are recommended to report [15]. COS development is informed by rigorous methodology developed by the Outcomes in Rheumatology Consortium (OMERACT) and related organizations such as the Core Outcomes Measures in Effectiveness Trials (COMET) initiative [16]. Key to the development of a COS is understanding which measurable aspects of health (referred to as domains) are most important. In this study, consistent with recognized COS development methodologies, we have used an international Delphi survey to build consensus towards a set of core domains in aSAH [17, 18].

Objective

To develop consensus among the various groups with a vested interest (people with lived experience, health care professionals, and researchers, funders, and industry professionals) in which domains should be measured and reported in aSAH research.

Methods

Ethics

Ethical approval for this study was granted by the Ottawa Health Science Network Research Ethics Board (reference: 20210028-01H). These results are reported according to the Equator Checklist for Reporting of Survey Studies (CROSS) [19]. The project to develop a COS in aSAH has been registered with the COMET database [20].

Study Design

We conducted an international, online, Delphi-based questionnaire study with three stakeholder groups (people with lived experience of aSAH [people who had an aSAH and/or their family members]; health care professionals; and a group consisting of funders, researchers, and industry professionals). The Delphi survey was completed using the web-based Delphi Manager (The COMET Initiative, Liverpool UK) and consisted of four rounds. The Delphi technique is a methodological approach for reaching consensus anonymously that provides all participants an opportunity to have an equal voice, while also allowing consideration of how their view aligns with others [17, 21, 22].

Participants

We approached potential participants with lived experience via email through the Brain Aneurysm Foundation. We also posted invitations with moderator approval to private online forums such as the Subarachnoid Haemorrhage Group set up by the Brain and Spine Foundation UK. We identified health care providers through local contacts, national organizations, and international collaborative networks. Eligible health care providers included nurses, physiotherapists, occupational therapists, dieticians, and medical doctors involved in the management of aSAH from acute care to rehabilitation. We also invited funders, researchers, and industry professionals involved in research funding. Given the rarity of this condition, we further identified those with relevant professional or personal experience through snowball sampling, where we asked eligible participants to identify others who may be interested in participating. Through this process, we identified 253 potential participants who were invited via a unique link to an online Survey Monkey (SurveyMonkey Inc., San Mateo, CA, USA., www.surveymonkey.com) form where they provided basic demographic details. Each participant consented to participate upon registration using the web-based Delphi Manager and we informed participants that they could withdraw at any time. After potential participants registered their interest, we sent an introductory email that explained the overall aims of the project and the Delphi survey process (Appendix 1).

Sample Size

Based on previously published COS development we aimed for a sample size of 100–200 participants [23,24,25].

Candidate Domains

We used a mixed methods approach to generate an initial broad range of candidate domains. This included a literature review on previous outcomes in aSAH research, qualitative interviews with people who have lived experience of aSAH (unpublished), and a multi-stakeholder workshop conducted at the international aSAH research meeting (see Appendix 2 for full domain list) [7, 14, 26]. These domains were classified according to four core areas (pathophysiological manifestations, life impact, resource use, and death) in accordance with the OMERACT process [27]. Members of the research team (CA, SE, JS, MM), including patient partners (MLM, PT, XZ), then removed duplicates and grouped overlapping domains to include 62 domains for rating [28]. During Round 1, participants were also offered the opportunity to suggest any domains missing from the initial 62 and these were added in Round 2.

Data Collection

Consistent with standard practice, in Round 1, participants rated the importance of each domain (on a 9-point Likert-like scale). In Round 2, participants received their Round 1 rating for each domain along with the median rating from Round 1 and the rating distribution across all participants. Participants could then re-rate domains or retain their initial rating. To determine consensus, Delphi Manager software dictates that two groups are needed. We grouped participants into a lived experience group and a combined researcher and health care professional group. Consensus for any domain was achieved when greater than 70% of participants in both groups rated it as critically important (rated as 7–9) and less than 15% of participants rated the same domain as of limited importance (rated as 1–3). The domains that achieved consensus were automatically removed after Rounds 2 and 3 by the Delphi Manager program. In Round 3, participants again received their rating for each remaining domain along with the rating distribution and the median rating for that domain from both groups and were able to re-rate domains if they wished.

To gain further insight into the relative priority of domains considered critically important, we included, a priori, an additional (fourth) ranking Round. This ranking was introduced to aid development of a core domain set given that the large number of identified domains meant it was possible many domains would be endorsed as critically important. In this situation, ranking the domains introduces a mechanism to assess participants’ priorities. In this final Round, participants were asked to rank the domains that achieved consensus in Rounds 2 and 3 into their top ten in order of importance. A score of 10 was allocated to the top position with descending scores until the 10th position which scored 1. The summed scores for each individual domain reaching consensus provided an overall ranking based on the highest overall score.

Participants were also able to provide free text comments during Rounds 1 to 3 with the prompt ‘Provide feedback’ after each domain in each of the three initial Rounds. After Round 4, there was also an opportunity to provide free text with a prompt ‘Comments’. A descriptive analysis of the free text entered by participants was performed using a single coder. The survey was initiated on 11 January 2022.

Results

Participant Characteristics

Figure 1 summarizes participant characteristics. From the 253 potential participants identified, 175 (70%) registered in the survey. Of the registered participants, 104 (59%) identified primarily as belonging to the aSAH survivor or family stakeholder group, 13 as researchers or industry (7%), and 58 (33%) as health care providers (see Appendix 3). Most (65%) respondents self-identified as women and the most common age range was 46–55 (31%). Respondents from 25 different countries and all regions of the world took part (Fig. 1).

Fig. 1
figure 1

Participant demographics and depiction of the countries they lived in

Full size image

Round 1

Round 1 was open from 11 January 2022 until 14 February 2022. It was completed by 154 (87.5%) of the registered participants (see Fig. 2). The mean scores and the percentage of participants rating each of the 62 domains as critical (7–9) for Round 1 are provided in Appendix 4. Eleven additional domains suggested by participants were added to Round 2 for ranking (see Fig. 3) and are listed in Appendix 5.

Fig. 2
figure 2

A flowchart displaying the number of participants who participated in each round of the Delphi survey

Full size image
Fig. 3
figure 3

The four voting Rounds undertaken in this survey with the number of participants completing each round from a maximum of 175 registered participants. Note: After Round 1, 11 additional domains suggested by participants were added. After Round 2, 25 domains reached consensus and progressed to the ranking round. A further 7 domains reached consensus after Round 3 and also progressed to the ranking round. The 32 consensus domains that were ranked are illustrated in Fig. 4

Full size image
Fig. 4
figure 4

The 32 domains considered to be critically important by survey participants categorized according to the OMERACT process [15]

Full size image

Round 2

Round 2 was open from 15 February 2022 until 28 March 2022. It was completed by 127 participants from the 154 who completed Round 1 (82.5%, Fig. 2). Of the 73 included domains, 25 met the pre-defined consensus criteria and progressed directly to the ranking round (see Fig. 3). The full results from this round are in Appendix 6.

Round 3

Round 3 was open from 29 March 2022 until 25 April 2022. It was completed by 124 participants from the 154 that completed either the first or second rounds (80.5%, Fig. 2). This represented an overall response rate of 70.8% (124 of 175 registered participants). Participants rated the 48 remaining domains that had not yet reached consensus (Appendix 7). An additional 7 domains reached consensus after the third round. The full list of domains that achieved consensus is listed in Fig. 4.

Round 4 (Ranking Round)

Round 4 was open from 26 April 2022 until 23 May 2022. Participants ranked their top ten domains from the 32 domains that reached consensus. The summary results for Round 4 are presented in Table 1. Cognitive and executive function was the highest ranked domain overall and was also ranked in the top two by each stakeholder group. Delayed cerebral ischemia was the highest ranked domain for health care professionals whereas those with lived experience ranked this domain 24th. Measures of survival had the three highest mean Delphi scores after round 2 but, when ranked overall, they rated 14th, 23rd, and 32nd.

Table 1 Results from the ranking round (Round 4) of domains achieving consensus and the rankings according to the average Likert scale score after Round 2 when all 73 domains were assessed by participants. The results from the ranking round are also presented by stakeholder group
Full size table

Key Themes in Comments by Participants Within Rated Domains

During the domain rating exercise, 13/175 participants made comments, often on multiple domains: 8/104 from the lived experience group, 1/13 from the researcher group, and 4/58 from the health care professional group.

  • Domain Theme 1: Situating mortality: For the survival domains, there were three comments including ‘if you die all the rest are not important’ and ‘Mortality is obviously a crucial outcome, however not as important as functional/cognitive outcomes’.

  • Domain Theme 2: Clarifying views about resources: In the resource domains, 5 participants made comments. One participant referred to issues with financial incentives in respective health systems such as ‘there is a financial reason for patients rehabilitation length of stay’. One participant expressed uncertainty with respect to the overall measure of quality-of-care domain commenting ‘Guideline concordance? Patient satisfaction? My response could be influenced in either direction depending on which’. For the discharge destination after hospital a participant commented that ‘[the] ultimate destination is more important than initial destination following discharge’.

  • Domain Theme 3: Qualifying views about specific pathophysiological manifestations: There were comments from 10 participants with respect to pathophysiological domains. With respect to aneurysm obliteration, one participant commented ‘this is an early and important milestone. However, the aneurysm does not necessarily have to be COMPLETELY obliterated. It is not uncommon for there to be a small residual component to the aneurysm and for it to be still secure’. Another commented ‘cerebral infarction is irrelevant as the sole outcome parameter if there are no symptoms or function restrictions after rehab’. One participant wrote ‘Like…how important are vasospasms. Well; my wife had them. It was terrible. But they controlled them. So, of course you don’t want them; but in the end if she had them and was ok upon discharge; who cares if she had them as part of the recovery/treatment process.’

  • Domain Theme 4: Clarifying perspectives on life impacts: With respect to life impact domains, there were comments from 3 participants. One respondent commented after the memory changes domain ‘the ability to process information is also important. I can no longer read a book’. Following scoring of the anxiety and/or PTSD domain a participant commented ‘I expect perspective of clinicians and patients may differ here. Like all the things listed here it is important. However, I’m not sure PTSD is a bigger problem with SAH than with other critical care conditions in general!’. One participant commented on the quality of life as reported by the survivor domain with the following ‘Getting others to understand that I may look the same as before and on surface seem fine; but there is lots going on that can’t be seen’.

Key Themes in Comments by Participants at End of Survey

In the free text section at the conclusion of the survey, 50/175 participants provided comments related to the survey: 39/104 from the lived experience group, 3/13 from the researcher group, and 8/58 from the health care professional group. Two main themes emerged from those responses:

Overall Comments Theme 1: Delphi process and interface: Sixteen participants conveyed their appreciation for the opportunity to participate in the Delphi process. Generally, the comments were favourable regarding the experience with one participant commenting ‘[The Delphi process] shows how sometimes the caregivers, healthcare providers and researchers share priorities and the areas in which they differ. Fascinating.’ Several respondents expressed difficulty with the survey interface with comments such as ‘[I] struggled to see the font used in completion’ and from another respondent ‘I used my phone to answer the survey and had a hard time reading it’. One participant indicated that ‘their level of comprehension has impaired my response’.

Overall Comments Theme 2: SAH Experience descriptions: Many participants took the opportunity to provide firsthand perspectives on their experiences after a SAH. Comments included ‘I truly believe that I have recovered extremely well with virtually no physical deficits (apart from ongoing fatigue +  + +)’ and ‘I went on to complete my Master and do a PhD after my recovery’ as well as ‘Having an aneurysm is a life changing experience’. Some participants emphasized challenges such as ‘new problems are developing; especially balance and vestibular issues like vertigo’ and ‘[My SAH] makes it difficult for me to read; understand and retain information’ as well as ‘Based on my experiences; the trauma and mental health piece was never addressed’.

Discussion

In this international, online Delphi survey, stakeholders (including an important proportion (59%) of participants with lived experience) identified 32 domains considered to be of critical importance to measure following aSAH. The highest ranked domains were Life Impact domains including cognitive and executive function, the ability to walk, and basic activities of daily living. Pathophysiological outcomes including aneurysm obliteration, cerebral infarction, delayed cerebral ischemia, and vasospasm were also highly ranked. The quality of life as reported by the survivor and changes to mood and emotions rounded out the top ten.

Our results are consistent with those of an international q-sort survey of aSAH stakeholders from 25 different countries which found similar domains prioritized [29]. Results from both surveys show a mismatch in the existing aSAH RCT literature between the outcomes selected by researchers and those prioritized by key stakeholders including patients and their families. Patient-reported quality of life and assessments of mood and depression were among the highest rated domains by participants in this study, yet they are rarely reported in the literature, with only 8.5% of aSAH randomized trials reporting a QoL measure and less than 2% reporting assessments of mood or depression [7]. The comment from one participant about what is going on that cannot be seen is especially pertinent in this context. This demonstrates the importance of asking stakeholders’ priorities directly rather than relying on past publications when selecting outcome measures.

The addition of a fourth round to enable ranking was relatively novel when compared to other Delphi surveys that have been used to prioritize outcomes in COS development [23,24,25]. We noted some distinct differences between the ranking of some domains depending on whether the Likert scoring method was used or an overall ranking process. This was most striking with respect to the survival domains which were scored highly for importance on Likert scales but ranked low when participants were asked to identify a top ten. This finding may have important implications for other COS developers who do not have the additional ranking round used in this survey and is worthy of further investigation in other contexts.

Our study has strengths. Firstly, we placed those with lived experience of aSAH at the centre of this work. We have had patient research partners involved in every step of the research process and almost 60% of the survey participants have direct lived experience as either survivors of aSAH or as a family member. We had high completion rates and maintained a high level of engagement across the four rounds of the survey. Additionally, we have used systematic and rigorous methodology. Delphi surveys are a well-accepted method; the iterative process encourages the development of consensus while allowing equal voice to all participants.

There are limitations to our work. Many domains were considered critically important (32/73), more than can be included in an eventual COS. This was mitigated by the a priori planned additional ranking round to help inform subsequent stages of the COS development. There is a risk of survivor bias in participants from the lived experience group, and this was commented on by one participant with the statement ‘if you die all else are unimportant’. We have included caregivers in the lived experience group to reduce this risk, but survivor bias should be considered in subsequent stages of the COS development.

The results of our survey are critical in the development of an aSAH COS and help inform clinicians, researchers, and policy makers what aspects of health are most important when designing and evaluating new interventions in aSAH. The next step in the COS development process is a consensus meeting to finalize a core domain set as per standard OMERACT methods. Further stakeholder engagement will address the uncertainty on the optimal time points to measure the consensus domains. Finally, we will evaluate the feasibility, truth, and discrimination of specific measurement instruments to characterize the chosen core domains and finalize the aSAH COS [15].

Conclusion

In this international Delphi survey, we have identified a range of domains considered to be of critical importance to a mixed group of stakeholders, of which the majority have direct lived experience with aSAH. Our results confirm that there is a mismatch between the outcomes reported in aSAH research and those prioritized by patients, family members, caregivers, health care providers, and researchers. Developing a COS in aSAH will help address this mismatch, improve research efficiency, and aid knowledge translation. The results of our study are a key step in the process for developing an aSAH-specific COS.