Introduction

Patient-reported outcomes (PROs) are direct reports from patients about any aspect of their health, including disease symptoms, treatment side effects (e.g. fatigue, pain), psychosocial issues (e.g. anxiety, depression, fear), functioning (e.g. physical, cognitive, social, role), and more complex concepts, such as health-related quality of life (HRQOL) [1]. Other information, such as health care experiences and health behaviours can also be self-reported by patients. PROs offer the patient’s perspective of the impact of disease and treatment. Information about the types of studies that collect PROs is needed to understand how patient-centred evidence is being used to inform patient care and health policy decisions. Only a small proportion (~20%) of cancer trials which collect PROs publish these data [2], thereby representing a significant barrier to the use and translation of PRO evidence into practice. This alarmingly low publication rate also represents a waste of limited PRO research funding, resources, and patients’ valuable time, as the PRO data are not disseminated and cannot be put to use [3].

Increasingly, clinical trials and observational studies are being registered with national, regional, and international trial registries. These registries allow patients to search for potentially suitable trials, and enable researchers and health professionals to keep abreast of research activities and medical advances that may impact clinical practice or policy. Methodological researchers advocate for trial registration as a strategy to reduce research waste [4]. The Australian New Zealand Clinical Trials Registry (ANZCTR) was established in 2005 and is one of 16 Primary Registries in the World Health Organization Registry Network. These 16 Primary Registries, along with Clinicaltrials.gov, feed trial information into the International Clinical Trials Registry Platform (ICTRP), a global trials portal. The ANZCTR is the fifth largest trial registry internationally, with 21,330 registered trials as at January 2018 [5]. It includes intervention trials and a small proportion of observational studies (this review discusses both intervention trials and observational studies, as registered in ANZCTR, and uses the term “trial” to refer to all registered studies except where otherwise stated). There have been no published studies of the proportion of ANZCTR-registered trials that include PROs as primary or secondary endpoints.

A recent review of ClinicalTrials.gov (the largest international trials registry, managed in North America) determined that 27% of trials registered between 2007 and 2013 included a PRO as a trial endpoint [6]. The ClinicalTrials.gov review did not report any geographic information regarding trial sponsorship or recruitment and, thus, it is unclear how many ClinicalTrials.gov studies were sponsored by ANZ organisations or recruited patients from ANZ. The ANZCTR and ClinicalTrials.gov registries have a small overlap of approximately 1.9% [7]; however, it is unclear how many of these overlapping studies were included in the review by Vodicka and colleagues [6].

Understanding where PRO research is being conducted is important for several reasons. Firstly, quality of life and other PROs can systematically differ by geographic region [8, 9]. Secondly, there may be regional differences in clinical practice guidelines for specific clinical areas, the development of which may be based on local PRO research data. Finally, information on regional characteristics of studies with PROs as key trial endpoints can help clinicians, policy makers, funders, and other key stakeholders determine where to target efforts to close patient-centred evidence gaps. To better understand the use of PRO endpoints in clinical trials globally and within ANZ, we aimed to identify ANZCTR-registered studies with PRO endpoints and describe key features of these studies according to data recorded in the ANZCTR. A review of study endpoints reported by proxies (such as clinicians or family caregivers) was conducted simultaneously, the results of which will be published separately.

Methods

Search strategy

Upon registering a study with the ANZCTR, the investigator completes a registration form with ten sections, including details on the clinical area, interventions, trial outcomes, recruitment regions, sponsor, and contact people [10]. For this review, the “outcomes” and “statistics” fields of the ANZCTR database were systematically searched from registry inception (2005) to 31 March 2017 using an updated version of the validated search used in the ClinicalTrials.gov review [6] (See Online Appendix 1). In summary, the search included the full titles and acronyms of PRO measures, as well as generic PRO terms (“quality of life”, “health-related quality of life”, “QOL”, “patient-reported outcome”, “health status”, “Likert scale”, “numeric rating scale”, “patient satisfaction”, “psychometric”, “self-report”, “Visual analog* scale”). All PRO measures listed on the Patient Reported Outcomes Quality of Life Instrument Database (PROQOLID™) by Mapi Research Trust and Grid-Enabled Measures (GEM) were included. PROQOLID™ provides descriptive information on over 1562 instruments and aims to enhance knowledge about relevant use of clinical outcome assessments for the scientific community [11]. GEM is an interactive database of “behavioural, social science, and other scientific measures organized by theoretical constructs” and contains information on 1447 measures (not all of which are patient-reported) and 483 constructs [12]. Our search strategy excluded caregiver measures, measures available exclusively in languages other than English (as the ANZCTR is in English), and measures of exclusively demographic or geographic information. The search thus comprised all 2448 validated search terms used in the review of ClinicalTrials.gov [6] and 202 new terms: names of new PRO measures and corresponding acronyms released since the ClinicalTrials.gov review (2650 terms in total).

Search results were exported into Microsoft Excel by a representative from ANZCTR. The Excel spreadsheet included information about each trial’s primary and secondary endpoints, assessment time points, sponsor, stage of progress, phase, clinical population, recruitment countries, and funding source, as entered into ANZCTR by trial investigators at the point of trial registration.

Trials identified by the search were divided among members of the research team. Researchers each screened and coded trial endpoints to determine whether each trial actually included at least one PRO endpoint. The coding process comprised three steps: (1) coding of all trial endpoints (PRO endpoint, not a PRO endpoint, or unclear); (2) categorising of individual PRO endpoints only, using a coding system adapted from Ahmed et al. [13] and summarised in Table 1; and (3) coding of each trial (includes/does not include a PRO endpoint). Any trial that included at least one PRO endpoint was included in the final analysis. Some trials listed the PRO domain or concept being measured as the PRO endpoint, whereas others listed the name of the PRO measure (i.e. questionnaire or tool). We coded PRO endpoints as they were recorded in the database and report domains and names of measures together in this analysis.

Table 1 Coding system for patient-reported trial endpoints
Full size table

Prior to the commencement of coding, all reviewers completed a pilot coding exercise of approximately 200 trial endpoints, which was checked by RMB or DW for consistency and accuracy. In addition, 10% of final coding was double checked, as were any individual endpoints or trials flagged as unclear by any reviewers. Prior to analysis, we checked the final dataset for face validity, for example, to ensure that any trial with at least one PRO endpoint had not wrongly been excluded at the trial level.

The final database of studies with PRO endpoints was analysed using descriptive statistics in Microsoft Excel and SPSS, Version 24.0. Armonk, NY: IBM Corp. Trial characteristics of interest included primary and secondary endpoints, clinical category, intervention type, sponsor type, stage of progress, phase, recruitment countries, year of registration, and funding source. To assess potential trends over time in prevalence of PRO use in trials in the ANZCTR, the relationship between year of trial registration (trials registered 2006–2016; the years for which data were available for the full calendar year) and percentage (%) of registered trials with a PRO endpoint for each year was calculated using Pearson correlation.

Results

On 31 March 2017, 13,666 trials were registered on the ANZCTR database, 5154 of which were excluded immediately as the search determined they did not include a PRO. The remaining 8512/13,666 trials, comprising 51,443 trial endpoints in total, were screened by the team. Of these, 6168/13,666 trials (45.1%) included at least one PRO endpoint. There were 17,961 individual PRO endpoints in total across the 6168 trials; in 4721/17,961 (26.3%) of cases, the PRO was the primary trial endpoint. A further 397 endpoints were considered “unclear if patient-reported”, but most of the corresponding trials could be included or excluded from the analysis based on other endpoints. Only four trials were classified as “unclear”, as none of the endpoints could be clearly classified as patient-reported (or not patient-reported) based on the information provided by the investigators in the ANZCTR form. Of the 6168 trials with PRO endpoints, 93.1% were intervention trials and 6.9% were observational studies (specific details on the intervention trials are reported in Online Appendix 2). Approximately 30.3% of all studies with PROs were active and recruiting, and 34.7% were completed, stopped early, suspended, or withdrawn (see Online Appendix 2).

Trials with PRO endpoints are described below, according to key data fields [10] within the ANZCTR.

Condition category

Mental health trials were most likely to include a PRO, with 1837 of 1840 trials in this category including a PRO endpoint (99.8% of all mental health trials; 29.8% of the 6168 trials with PROs) as shown in Table 2. Other condition categories in which PRO endpoints were included in over 50% of trials in that category were stroke (83.7% of 301 stroke trials), physical medicine/rehabilitation (65.6% of 1053 trials), alternative and complementary medicine (64.7% of 408 trials), musculoskeletal (63.5% of 1256 trials), public health (63.1% of 1273 trials), injuries and accidents (60.5% of 453 trials), neurological (55.9% of 956 trials), and cancer (54.2% of 1511 trials). Overall, blood trials had the lowest percentage of studies including PRO endpoints (17.1% of 258 trials).

Table 2 Characteristics of 6168 studies with PRO endpoints
Full size table

Interventions

Some 79.3% of registered behaviour trials, 70.5% of rehabilitation trials, 65.0% of lifestyle trials, and 57.5% of treatment—‘other’ trials included PROs (Table 2). The intervention category with the lowest percentage of PRO endpoints was Diagnosis/Prognosis, with 20.8% of 452 trials including PROs.

Trial sponsors

Trials with PROs were largely sponsored by universities; 55.7% of university-sponsored trials included PROs, 45.9% of trials sponsored by individuals included PROs, and 37.4% of trials sponsored by hospitals. Only 30.7% of industry-sponsored trials included PRO endpoints (Table 2).

Trends over time

Figure 1 shows the number of trials registered on ANZCTR over time with PROs and overall. There was an upward trend in both the number and proportion of ANZCTR studies with PROs registered over time (r = 0.74, p = 0.009). At the time of this review, the year with the largest number of trials registered overall, as well as the largest number of trials with PROs, was in 2016, with 882/1389 (63.5%) including PROs.

Fig. 1
figure 1

Trials registered on ANZCTR over time, including proportion of trials with PRO endpoints

Full size image

Recruitment countries

Trials with PROs were recruiting from 95 unique countries. The proportion of studies with PRO endpoints in ANZCTR was highest among studies recruiting within Australia (75.8% of 6168 trials with PROs, 48.3% of 9672 trials recruiting within Australia) (Table 3). New Zealand followed second, with 15.4% of all 6168 trials with PRO endpoints recruiting from New Zealand (46.3% of 2057 trials recruiting within New Zealand). Over half (52.5%) the 244 ANZCTR studies recruiting within UK and Ireland and 51.6% of the 376 ANZCTR studies recruiting in North America included PRO endpoints.

Table 3 Recruitment locations of 6168 trials with PRO endpoints
Full size table

Among the 6168 trials with confirmed PRO endpoints, 72.7% were recruiting from Australia only, 3.1% were recruiting from Australia and outside Australia, and 16.0% from outside Australia only. Location of recruitment data was not available for 8.2%. Recruitment within Australian states and territories is displayed in Table 3.

Types of PRO endpoints

Table 4 shows how the 17,961 individual PRO endpoints used in the 6168 trials were categorised. The majority of PRO endpoints were evaluating patient-reported symptoms, functional outcomes, or condition-specific QOL (65.6%). Generic QOL (13.2%), patient-reported experiences (9.9%), and self-reported behaviour (7.9%) outcomes were the next most common types of PROs assessed. Finally, 2.9% of PRO endpoints included multiple PROs as a single outcome within the ANZCTR database, each assessing different types of outcomes, for example: “Questionnaires on self-efficacy, self-concept, physical activity and inactivity, lifestyle information relating to nutrition, smoking, physical activity and diabetes knowledge;” and “Questionnaires—Habitual exercise (PASE)—WOMAC index (pain, stiffness and functional ability)—Depressive symptoms (Depression Scale)—Quality of life (SF36)—Confidence performing physical activity (Ewart)—Demographics”. In these cases, the PRO was poorly defined and could have been coded in more than one category. Relatively few PRO endpoints were insufficiently defined to permit classification (0.5%, n = 97), for example: “To assess the effect modification of clinico-demographic factors using health and demographics questionnaires (Questionnaire has been designed specifically for this study)”.

Table 4 Types of PRO endpoints across the 6168 trials with PRO endpoints identified in this study
Full size table

Discussion

This study determined that 45.1% of the 13,666 trials registered on the ANZCTR from 2005 until 31 March 2017 included a PRO as a primary or secondary trial endpoint. Additionally, the percentage of trials registered in ANZCTR that include PROs increased over time. The majority of trials including PROs were interventional, as compared to observational. Further, intervention trials of behaviour, rehabilitation, and lifestyle were most likely to include PRO endpoints (79.3, 70.5, and 65.0% of trials, respectively). Mental health, cancer, public health, and musculoskeletal studies comprised 69% of all 6168 trials with PRO endpoints, yet these categories of trials account for only 43.1% of the trials registered on the ANZCTR. Mental health, cancer, and musculoskeletal conditions are associated with a relatively high level of symptom burden, which may explain the high proportion of studies with PRO endpoints. The most common types of PRO measures used were measures of symptoms and functional outcomes (65.6%), followed by generic QOL and utility measures (13.2%).

Our findings suggest a high level of PRO research activity within ANZ and among studies registered with ANZCTR as compared to studies registered in other international clinical trial registries [6, 14]. For example, only 27.2% of all trials registered on ClinicalTrials.gov between 2007 and 2013 included a PRO [6]; considerably lower than the 43.4% of trials registered on ANZCTR with PRO endpoints in the same period: 2007–2013. Vodicka also determined that 29% of 13,584 oncology trials registered on ClinicalTrials.gov between 2007 and 2013 included a PRO endpoint [6], while the figure for ANZCTR oncology trials was almost double that (54.2%). Further, within ANZCTR, studies recruiting in South America, Africa, the Middle East, and Oceania were less likely to include PRO endpoints (fewer than 35% of studies within each region included PROs). This may reflect low English literacy levels in these regions, or a tendency of ANZ-based investigators, or investigators of studies in mental health, stroke, rehabilitation, and other categories with high PRO use, to collaborate with researchers based in North America, the UK, and Europe (regions with a higher proportion of studies with PROs, based on our findings) due to common use of the English language. Additionally, guidelines for the use [15] and reporting [16, 17] of PROs exist in English and may not be translated (or accessible) to other languages. Some regulatory guidelines that address PROs are specific to certain regions [1, 18] and may have promoted use of PRO endpoints in these regions as compared to other regions. Collectively, our findings indicate regional variation in the use of PRO endpoints, as well as the importance of evaluating multiple international registries to get a comprehensive picture of the use of PROs internationally.

Our findings provide additional evidence that the use of PROs in clinical trials is on the rise globally, a trend that was identified in both this present study and the review of ClinicalTrials.gov [6]. It is worth noting that the proportion of ClinicalTrials.gov trials (2007–2013) with PROs had increased from 14% to 27.2% between the publishing of an earlier review conducted of trials registered between 2004 and 2007 [14] and these rates may be increasing still. This is not surprising, as more and more professional and clinical organisations are endorsing the inclusion of PROs and offering guidance for their use. The American Society for Clinical Oncology, the European Society for Medical Oncology, the American Academy of Orthopedic Surgeons, and the Organisation for Economic Cooperation and Development have described the clinical benefits of PRO assessment in recent years [19,20,21]. The U.S. Food and Drug Administration and the European Medical Association have produced comprehensive guidance for the use of PROs [1, 18].

A larger proportion of ANZCTR-registered studies with PROs were sponsored by industry (30.7%), as compared to the ClinicalTrials.gov review (22.5%) [6]. These relatively low proportions of industry-sponsored trials with PROs may reflect the number of industry-sponsored, earlier phase trials, or drug development trials, in which PRO assessment is either not possible or relevant. For example, approximately 89.2% of studies using a PRO and registered with ClinicalTrials.gov were in Phase II or higher [6]. Similarly, of the interventional trials which specified research phase with confirmed PROs in our review, 86.8% were Phase II–IV trials.

In ANZCTR, 55.7% of university-sponsored studies included a PRO as compared to only 29.0% of university-sponsored trials registered on ClinicalTrials.gov 2007–2013 [6]. However, PRO use in government-sponsored research was more similar: 33% of National Institute of Health (NIH)/US Federal Government-sponsored trials included a PRO in ClinicalTrials.gov database, while 39.2% of ANZCTR studies sponsored by a “Government body” included a PRO. Due to differences in trial registration forms across registries, it is difficult to compare other sponsor categories.

Strengths and limitations

This is the first study to estimate the inclusion of PRO endpoints in ANZ and the ANZCTR, and contributes valuable information about the use of PROs internationally. The strengths of this study include a robust and validated search strategy, comprehensive checking and coding of all 51,443 trial endpoints retrieved by the search. Our approach will enable comparisons with past [6, 14] and future studies of PRO use among registered clinical trials. Limitations of this study include the potential for coding errors by reviewers in classifying endpoint types. We also were unable to classify and code a minor proportion of endpoints and trials due to limitations in how data were entered by clinical trial investigators. Furthermore, we had no way of verifying the accuracy of the information recorded by investigators in ANZCTR.

We were also unable to make direct quantitative comparisons between some aspects of the results of our review and the review of ClinicalTrials.gov due to differences in how data are entered between registries. Importantly, it is unclear how many trials remain unregistered, or how many ANZ trials are included in registries other than ANZCTR; hence, the number of trials recruiting or sponsored in ANZ reported here may be underestimated. It is likely that many trials sponsored by or led by ANZ researchers are registered on ClinicalTrials.gov or other trial registries.

Next steps

Our findings indicate that a minimum of 1987 ANZCTR-registered studies with PRO endpoints were completed as of 31 March 2017. Given the evidence of PRO research waste in European studies [2], the current study provides the basis for a future study to determine how many of the 1987 trials have published their PRO results. This follow-up research question, which would require a dedicated literature search, will indicate both the extent of PRO research waste and the extent to which PRO evidence has the potential to impact clinical practice and policy through dissemination to relevant stakeholders through publication.

Future work may also examine the impact of having professional societies promote the inclusion of PROs and the impact of newly released PRO guidance documents, such as the SPIRIT-PRO guidance for protocol development [15] and CONSORT-PRO guidance [17] for reporting PRO data. Given that clinical trials typically take many years to plan and implement, it may be years before the true impact of these initiatives can be reflected in the proportion of clinical trials and observational studies including PROs. Further, our review highlights the need for clearer specification of PRO endpoints by trial investigators. It is hoped that the SPIRIT-PRO guidance will help to address this short coming. Trial registries may benefit from providing links to such guidance documents to improve endpoint definitions and other aspects of study design and methodology.

Our work also identifies research questions regarding PRO use in specific clinical areas. For example, it would be interesting to examine in detail what specific outcomes are being assessed in each clinical area; what PRO measures are being used in these areas; which PRO measures are most sensitive, reliable, and responsive in each respective clinical context; and how PRO research has contributed to each clinical area in terms of new knowledge or prompting changes to clinical practice. Such research may also enable us to assess whether there is any duplication of PRO research efforts, any gaps in current research, and how best to address both the duplication and gaps.

Within ANZCTR, certain categories had a relatively low proportion of trials which included PROs. For example, only 39.6% of surgical trials included PROs, despite potential benefit from obtaining a patient’s valuable perspective about the benefits and harms of different surgical procedures. There may be scope to add value to future surgical trials by thoughtful inclusion of PRO endpoints.

Conclusions

Approximately 45% of trials registered on the ANZCTR include PRO endpoints, about a third of which were mental health trials. The proportion of studies that included PRO endpoints was higher in the ANZCTR than in ClinicalTrials.gov, suggesting potential international differences in the types of studies registered and that use PROs. The inclusion of PRO endpoints has increased since 2006, which suggests that researchers, clinicians, policy makers, and patients are increasingly recognising the importance of PROs in the comprehensive assessment of health and health care interventions.