FormalPara Key Points for Decision Makers

This is the first systematic review of model-based economic analyses that covers the entire schizophrenia care pathway, by including any intervention for the prevention, detection, diagnosis, treatment and follow-up of schizophrenia.

This review highlights a lack of models for non-pharmacological interventions, and low quality of existing models. Common reasons for low quality include use of a time horizon that is not sufficiently long, failure to capture the health and cost impact of adverse events of the interventions under assessment and potential conflicts of interest.

Due to inconsistent conclusions reported by different studies, no antipsychotic can be considered clearly cost effective compared with the others. A very limited number of models suggest that the following non-pharmacological interventions might be cost effective: psychosocial interventions, stratified tests, employment intervention and intensive intervention to improve liaison between primary and secondary care.

A consistent basis for the model structure, use of evidence and assumptions in health economic models is required in order to improve the consistency and quality of future health economic models in schizophrenia. This consistent basis could be applied using generic agreed models, which might include a de novo whole-disease model.

1 Introduction

Schizophrenia is a chronic, severe and disabling psychiatric disorder, or cluster of disorders, characterised by psychotic symptoms that alter a person’s perceptions, thoughts, affect and behaviour. The schizophrenia clinical guideline developed by the National Institute for Health and Care Excellence (NICE) recommends a wide range of interventions for people who are at risk of, or who have a diagnosis of, schizophrenia, including antipsychotics, cognitive behaviour therapy (CBT), family intervention, peer support, physical health checks and interventions, and education and employment support [1]. However, the rates of implementation are low for some recommended interventions including physical health interventions (13%), family interventions (31%), CBT (41%) and supported employment programmes (63%) [2]. It has been reported that the allocations for mental health care in national health budgets are commonly disproportionate to the burden of mental health conditions in many countries [3]. For example, in the UK, although mental disorders are responsible for 28% of the total burden of disease, mental health care only receives 13% of total NHS funding [4]. As a result, mental health commissioners may not be in a position to fund all recommended interventions and must decide how to allocate limited budgets across the entire care pathway in a way that maximises benefits to patients.

Since clinical trials rarely collect all of the information required to estimate the full profiles of health outcomes and costs for all interventions relevant to a decision problem, health economic modelling is routinely used to simulate the current and proposed systems of care, with input data obtained from multiple sources [5]. The purpose of this review is to conduct a systematic review of existing health economic models of any type for schizophrenia and provide recommendations for future research. Specific objectives were as follows:

  1. (1)

    To assess the availability of economic models of interventions for patients who are at risk of, or who have a diagnosis of, schizophrenia.

  2. (2)

    To critically examine the quality of existing health economic models.

  3. (3)

    To summarise the conclusions reported by existing health economic models and to assess the consistency of conclusions.

2 Methods

This systematic review was conducted according to the PRISMA recommendations for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions [6].

2.1 Inclusion/Exclusion Criteria

Inclusion and exclusion criteria were defined a priori. Studies were included if they met all of the following criteria: (i) studies reporting model-based economic evaluations adopting either a cost-effectiveness analysis (CEA) or cost-utility analysis (CUA) approach; (ii) focus on young people (under 18 years of age) and/or adults (18 years and older) who are at clinical high risk of psychosis (CHR), with a non-specific diagnosis of psychosis, or with a diagnosis of schizophrenia (including schizoaffective disorder and delusional disorder), and (iii) interventions targeted at the prevention, detection, diagnosis, treatment or follow-up of schizophrenia. No restrictions by country, healthcare setting or monetary currency were applied. Studies were excluded if they met any of the following criteria: (i) reviews, commentaries, letters, editorials or abstracts; (ii) published before 2005, or (iii) not reported in English.

2.2 Search Strategy

Electronic biomedical and psychological databases searched included MEDLINE (including in-Process and other non-indexed), EMBASE and PsycINFO, accessed through the Ovid interface (https://ovidsp.ovid.com/). In addition, the NHS Economic Evaluation Database (NHSEED) and the Health Technology Assessment Database (HTA) were searched, accessed through the Cochrane Library interface (http://onlinelibrary.wiley.com/cochranelibrary/search8). The search strategies included Medical Subject Heading (MeSH) terms and text words. Each follows a similar structure: population terms AND economic evaluation terms AND modelling terms AND limitation terms. The original search, first update search and second update search were conducted on 22 June 2015, 4 March 2018, and 21 January 2020, respectively. The detailed search strategy is reported in Online Resource 1, Section 1 (see electronic supplementary material [ESM]). Retrieved search results were downloaded into Endnote X8.0.2.

2.3 Assessment of Abstracts for Inclusion

Screening of abstracts and papers against the inclusion criteria was carried out by two reviewers (HJ and EA for the original and first update search; HJ and DA for the second update search). Final inclusion of studies in the review was determined by agreement of both reviewers, with disagreements resolved by discussion. A number of additional strategies were devised to help ensure that relevant studies were not missed. Firstly, key papers and the publications of key health economists were checked for inclusion and for additional relevant papers. Secondly, published systematic reviews relevant to the target population were located through a separate search of NICE clinical guidelines, NICE technology appraisals and National Institute for Health Research (NIHR) HTA reports. The search terms used by the located systematic reviews were used to inform the development of search strategies for the current systematic review, and the studies included within those reviews were checked for relevance with respect to the inclusion criteria of the current systematic review. Finally, the reference lists of all included studies identified via the electronic search were checked for any additional studies that may have been missed by the electronic search strategies.

2.4 Data Extraction and Analysis

Data were extracted by one reviewer (HJ) and checked by a second reviewer (EA for the original and first update search, DA for the second update search), with disagreements resolved by discussion. The following information was extracted from all included studies: author; year; country; study objective; type of economic evaluation; intervention and comparator; modelling method; willingness-to-pay threshold (e.g. per quality-adjusted life year [QALY] gained), conclusions, potential conflicts of interest and information on quality criteria set out by the NICE checklist and Cooper hierarchy. Study characteristics and conclusions were summarised descriptively.

2.5 Quality Assessment

Seven commonly used checklists for economic evaluations [7,8,9,10,11,12,13] were considered for the current review; they differ from each other in terms of the aim of the quality assessment (e.g. to assess reporting quality, or methodological quality of economic evaluations, or both) and the types of studies covered (e.g. trial-based economic evaluations, model-based economic evaluations, or both). To be of value to the current review, checklists needed to (i) focus on methodological quality of studies; (2) be appropriate for modelling studies; and (3) provide an overall judgement regarding the methodological quality of the studies assessed, so as to help the reviewers to summarise and compare the methodological quality of a large number of included studies (e.g. ≥ 50 studies). Based on these three criteria, two checklists were deemed to be most appropriate for the current review: Section 2 of the NICE checklist [11] and the Cooper hierarchy [10]. The NICE checklist consists of two sections. Section 1 aims to assess the applicability of a study to the decision problems that need to be addressed by the NICE guidance; for example, whether the study population is appropriate to the review question of interest or whether the system in which the study was conducted is sufficiently similar to the current UK context. As the aim of this systematic review was to provide an overview of the availability and quality of all economic models focusing on the schizophrenia care pathway, Section 1 was not considered relevant. Section 2 of the NICE checklist aims to assess the methodological quality of the study and thus was included. Section 2 consists of 12 quality criteria and an overall assessment. Based on the number and importance of quality criteria that a study fails, an assessment regarding the overall methodological quality of the study can be classified into one of the following categories: (i) very serious limitations—the study fails to meet one or more quality criteria, and this is highly likely to change the conclusions about cost effectiveness; (ii) potentially serious limitations—the study fails to meet one or more quality criteria, and this could change the conclusions about cost effectiveness; and (iii) minor limitations—the study meets all quality criteria, or fails to meet one or more quality criteria but this is unlikely to change the conclusions about cost effectiveness, potentially serious limitations and minor limitations. The Cooper hierarchy focuses on the quality of the data sources used to inform the parameters in a model [10]. The hierarchy provides a list of potential sources for each data component of interest, including main clinical effect size, baseline clinical data, adverse events and complications, resource use, costs and utilities. Sources are ranked on a scale from 1 to 6, with the most appropriate source assigned a rank of 1. Where multiple data inputs were included within a category (i.e. adverse events and complications, resource use and cost), the score of the worst sources of evidence were recorded. Based on the value of the score, the quality of input data was then categorised as high-ranked evidence (score 1–2), medium-ranked evidence (score 3–4) or low-ranked evidence (score 5–6). The Cochrane Handbook for Systematic Reviews [14] recommends the Cooper hierarchy as a useful supplement to more comprehensive checklists such as the NICE checklist.

3 Results

3.1 Study Identification and Selection

A total of 1557 citations were retrieved from electronic searches carried out on three separate occasions (original search 22 June 2015; first updated search 4 March 2018, second update search 21 January 2020). The detailed results of the literature search are reported in Online Resource 1, Section 1 (see ESM). Four modelling studies known to one of the authors (HJ), but that were not identified by the electronic searches, were added to the database. These four studies were reported in the adult NICE schizophrenia guideline [1], and were missed by the electronic searches because NICE clinical guidelines are not currently indexed by mainstream electronic databases. After removing duplicates, 1250 citations remained: 908 citations identified from the original electronic searches, 204 identified from the first updated electronic searches, 134 identified from the second updated electronic searches, plus the four models identified from the NICE schizophrenia guideline for adults [1]. Of the 1250 abstracts reviewed, 981 were excluded for clearly failing to meet at least one inclusion criterion or meeting at least one exclusion criterion, leaving 269 for full-text review. Of these, 97 were abstracts only and for the remaining 172, full articles were retrieved. Of these, 77 papers reporting 73 studies (four papers are corrections of other included studies) satisfied the predefined inclusion criteria and were included in the review. The inter-reviewer agreement, measured by Cohen’s kappa, was 0.84, which indicates good agreement. A modified Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram [6] for the literature selection process is provided in Fig. 1. The key data extracted from included studies are reported in the Online Resource 1, Section 2 (see ESM).

Fig. 1
figure 1

Modified PRISMA flow diagram for the systematic review of economic models. Superscript a: four papers are corrections of other included studies

Full size image

3.2 Study Descriptions

Table 1 summarises the characteristics of included studies. Of the included studies, 89.0% (65/73) were from high-income countries, such as the US (11/73, 15.1%), the UK (11/73, 15.1%) and Sweden (6/73, 8.2%). Fifty-eight included studies were CUAs (79.5%), while 15 were CEAs (20.5%). The perspectives of cost adopted by included studies are healthcare system (36/73, 49.3%), third-party payer (22/73, 30.1%), healthcare system and social care (8/73, 11.0%) and society (7/73, 9.6%). The majority of studies adopted a time horizon of 1–5 years (52/73, 71.2%). The most commonly used modelling techniques were Markov model (34/73, 46.6%), decision tree (24/73, 32.9%) and discrete event simulation (DES) (9/73, 12.3%). In terms of population, the majority of included studies related to people with a diagnosis of schizophrenia (68/73, 93.2%). The remaining studies evaluated interventions for people with a non-specific diagnosis of psychosis (5/73, 6.8%) [1, 15,16,17,18] and those at CHR (2/73, 2.7%) [15, 17]. In terms of interventions assessed, most included studies compared the cost effectiveness of different antipsychotics versus each other, placebo or nothing (57/73, 78.1%). The remaining studies assessed the cost effectiveness of different coverage of Medicare drug plans (1/53, 1.6%) [19], electroconvulsive therapy (ECT) versus antipsychotic (1/53, 1.6%) [20], precision medicine test versus no test (4/73, 5.5%) [21,22,23,24], different monitoring schedules for patients receiving clozapine (1/73, 1.4%) [25], antipsychotics versus antipsychotics plus psychosocial interventions (5/73, 6.8%) [26,27,28,29,30], CBT versus no CBT (1/73, 1.4%) [17], improving patients’ access to psychological therapies versus no intervention (1/73, 1.4%) [18], supported employment programme versus no intervention (1/73, 1.4%) [1], and different modes of liaison between primary and secondary care services (1/53, 1.6%) [15]. The availability of economic evidence across the schizophrenia care pathway is presented in Fig. 2. As shown in Fig. 2, there is high availability of economic evidence for antipsychotic with or without psychosocial interventions and moderate availability of economic evidence for precision medicine test. On the other hand, there is very limited or even no economic evidence concerning the prevention, case identification, assessment and diagnosis of psychosis and schizophrenia, as well as non-pharmacological interventions for people with a diagnosis of psychosis or schizophrenia.

Table 1 Characteristics of included studies
Full size table
Fig. 2
figure 2

Availability of economic evidence across the schizophrenia care pathway

Full size image

3.3 Quality Assessment

The results of the quality assessment are reported below; further detail is provided in Online Resource 1, Section 3 (see ESM).

3.3.1 NICE Checklist

According to the quality assessment results of the NICE checklist, 62 studies were deemed to have very serious limitations (84.9%), eight were deemed to have potentially serious limitations (11.0%), and three were deemed to have minor limitations (4.1%) [1, 17, 31]. The performance of included studies on all items of the NICE checklist is shown in Fig. 3. Common problems identified for all included studies are: (i) potential conflict of interest (58/73, 79.5%); (ii) use of time horizon not sufficiently long to reflect all important outcomes (54/73, 74.0%); and (iii) baseline outcome data not obtained from the best available source: (49/73, 67.1%). Of the 62 studies deemed to have very serious limitations, the most common reasons for them to be assessed as very serious limitations are as follows (some studies can be assessed as having very serious limitations for more than one reason): (i) did not include all important and relevant costs, for example, the cost of treating adverse events of antipsychotics (42/62, 67.7%); (ii) failure to include all important and relevant outcomes, for example, disutility caused by adverse events of antipsychotics (40/62, 64.5%); and (iii) the model structure did not adequately reflect the nature of the topic under evaluation (26/62, 41.9%), for example, did not model discontinuation of antipsychotics due to intolerability or non-adherence.

Fig. 3
figure 3

Performance of included studies assessed by Section 2 of the NICE checklist. ICER incremental cost-effectiveness ratio, NICE National Institute for Health and Care Excellence

Full size image

3.3.2 Cooper Hierarchy

Figure 4 presents the results of applying the Cooper hierarchy to the included studies. Of the six categories included in the Cooper hierarchy, three of them (adverse events, resources use and costs) may include multiple data inputs (i.e. more than one data source can be used for that category). For these three categories, the score of the lowest quality evidence was reported. As shown in Fig. 4, most studies used high-ranked evidence for unit costs (49/73, 67.1%) and clinical treatment effects (47/73, 64.4%), and low-ranked evidence for baseline clinical events (49/73, 67.1%), resource use (47/73, 64.4%) and adverse events (30/73, 41.1%). Of the 58 CUA studies that modelled patients’ utilities, most used medium-ranked evidence to inform utility estimates (54/58, 93.1%).

Fig. 4
figure 4

Performance of included studies assessed by the Cooper hierarchy

Full size image

3.4 Results of Existing Models

The cost-effectiveness conclusions of exiting models are summarised in Table 2.

Table 2 Summary of included studies by decision questions assessed
Full size table

3.4.1 Conclusions for Antipsychotics

Owing to considerable variability in the number and type of antipsychotics assessed, as well as inconsistent conclusions reported by different studies, it was not possible to identify the most cost-effective antipsychotic for the following patient groups: schizophrenia patients in an acute episode, in remission, or with unspecified psychotic status; schizophrenia patients who have a history of non-adherence; and patients with treatment-resistant schizophrenia (TRS). For schizophrenia patients who are experiencing adverse events of typical antipsychotics, one study found oral risperidone to be cost effective compared with oral olanzapine or oral typical antipsychotics [53]. For patients with negative symptoms of schizophrenia, one study found oral cariprazine is more cost effective than oral risperidone [81].

Of the 57 identified antipsychotic models, 45 reported potential conflicts of interest (the study was funded by, or affiliated with, commercial companies). All 45 studies reported positive findings for the antipsychotic manufactured by the sponsoring commercial company, which indicates that the conclusions of these 45 models might have been influenced by conflicts of interest. Focusing on the 12 studies that did not report potential conflicts of interest, the relative cost-effectiveness of the two most frequently assessed antipsychotics—oral olanzapine and oral risperidone—was explored in order to assess the consistency of conclusions across studies. The results, reported in Table 3, show that for all three patient groups for whom data were available, the studies with no conflicts of interest reported inconsistent conclusions. For example, for studies that focused on schizophrenia patients in remission, two studies found oral risperidone was cost effective compared with oral olanzapine [1, 31], while three studies found oral olanzapine was cost effective compared with oral risperidone [42, 47, 50].

Table 3 Consistency of cost-effectiveness conclusions reported by studies of antipsychotics with no conflicts of interest
Full size table

3.4.2 Conclusions for Non-Pharmacological Interventions

Five models compared the cost effectiveness of antipsychotic medication alone with antipsychotic medication plus psychosocial interventions [26,27,28,29,30]. All of these studies concluded that antipsychotic medication plus psychosocial interventions was cost effective compared with antipsychotic medication alone. For the remaining non-pharmacological interventions, each was only assessed by one model. The interventions found to be cost effective by these models, and the comparators, are as follows:

  • a Medicare scheme that covers the cost of generic antipsychotics, compared with no coverage [19];

  • clozapine for patients with TRS who respond to, and who can tolerate clozapine, compared with typical antipsychotics and ECT; and ECT for patients with TRS who have not responded to, or who cannot tolerate clozapine, compared with typical antipsychotics [20];

  • a stratified test with 60% sensitivity and 60% specificity for identifying patients who would respond to a second-line non-clozapine antipsychotic after failing a first-line non-clozapine antipsychotic, compared with no stratified test [23];

  • a stratified test with 100% accuracy to inform the starting dose of risperidone for patients with first episode psychosis (FEP), compared with no stratified test [22];

  • human leukocyte antigen genotyping for identifying patients with TRS who are likely to develop clozapine-induced agranulocytosis, compared with no test [24];

  • no monitoring for patients with TRS on clozapine, compared with monitoring [25];

  • antipsychotic plus psychosocial interventions for schizophrenia patients, compared with antipsychotic alone [26,27,28,29,30];

  • CBT for patients with ultra-high risk of developing psychosis or with FEP, compared with no CBT [17];

  • a programme to improve patients’ access to psychological therapies, compared with current practice [18];

  • a supported employment programme for patients with psychosis or schizophrenia actively seeking employment, compared with current practice [1];

  • an intensive intervention to improve liaison between primary and secondary care for people with early signs of psychosis, compared with a less intensive intervention or no intervention [15].

4 Discussion

4.1 Summary of Findings

This review of economic models of interventions for schizophrenia found the quality of existing models to be generally low. Common reasons for low quality included use of a time horizon that was not sufficiently long, failure to capture the health and cost impact of adverse events of the interventions under assessment, and potential conflicts of interest that may have biased the results of the analyses.

Seventy-eight percent of existing models assessed the cost effectiveness of antipsychotics. However, it was not possible to identify the most cost-effective antipsychotic for the majority of schizophrenia patients due to considerable variation in terms of the number and type of antipsychotics assessed and inconsistent conclusions reported by different studies. Inconsistent findings were a problem for models with conflicts of interest and those where no conflict of interest was identified, which suggests that the variation in results cannot be explained solely by conflicts of interest, but are also likely to be related to differences in choice of treatment options and variances in methods, such as model structure, type of adverse events considered, source of input data and methods of evidence synthesis. The review found very limited or even no economic evidence concerning the prevention, case identification, assessment and diagnosis of psychosis and schizophrenia, as well as non-pharmacological interventions for people with a diagnosis of psychosis or schizophrenia.

4.2 Recommendations for Future Research

4.2.1 Interventions Prioritised for Future Modelling

A number of interventions for schizophrenia have been recommended in the NICE schizophrenia guideline [1], but have not been formally assessed for cost effectiveness within a model-based economic evaluation framework. These include (i) assessment and diagnosis for people with possible psychosis; (ii) interventions to manage challenging behaviour in people with psychosis/schizophrenia; (iii) interventions to promote physical health in people with psychosis/schizophrenia; (iv) peer support or self-management interventions to improve symptoms and functioning for people with psychosis/schizophrenia; and (v) teams and service-level interventions. It is recommended that the above interventions should be prioritised for future economic models.

4.2.2 Improvements to the Consistency and Quality of Economic Analyses in Schizophrenia

One option for improving the consistency and quality of economic analyses in schizophrenia would involve the development of an agreed ‘generic’ model structure [82], populated using input data obtained from high-quality evidence, which would allow for the consistent economic evaluation of new and existing treatment options as and when such analyses are required (e.g. when a new drug comes to market). Provided the basis of the model (e.g. its structure and the evidence used to inform it) can be agreed, the development of a generic schizophrenia model would remove the possibility of producing inconsistent results and improve model quality. Development of a registry of economic models by disease areas is a potential method for promoting use of the generic modelling approach [83].

As an extension of generic models, Tappenden et al. have proposed the development of Whole Disease Models (WDMs)—these are generic models which, in principle, allow for the consistent economic analysis of any individual or combination of options at any point in the disease and treatment pathway [84]. This ‘whole system’ approach would provide a single platform for the economic evaluation of all key interventions for schizophrenia based on a common set of assumptions and input data across the whole care pathway. Whilst this type of modelling approach represents a significant undertaking in terms of model development time and resource, it would provide a means of addressing the significant gaps identified within this review relating to the inconsistent and/or absent economic evidence for current treatments for schizophrenia. In addition, it may be particularly valuable in capturing interactions between interventions given at different points on the pathway; for example, interventions that reduce a patient’s duration of untreated psychosis earlier on the pathway are likely to impact upon the cost effectiveness of other treatments later on the pathway.

4.3 Strengths and Limitations

4.3.1 Strengths

Whilst a number of systematic reviews have been identified that assess economic studies for schizophrenia, most of them focused on the cost effectiveness of antipsychotics and ignored other non-pharmacological interventions [1, 85,86,87]. Before our study, there was only one review (Németh et al. [88]) that included all model-based economic evaluations for schizophrenia regardless of which intervention was assessed. However, Németh et al. only searched one electronic database (MEDLINE); in addition, it focused on the methods used by published models such as utility mapping algorithms, without reporting conclusions of the identified models. To our knowledge, our study presents the first systematic review that summarises the cost-effectiveness evidence reported by existing model-based economic analyses that covers the entire schizophrenia care pathway, including any intervention for the prevention, detection, diagnosis, treatment and follow-up of schizophrenia. The information reported by this systematic review can be used to help researchers, commissioners or other stakeholders to rapidly locate relevant economic evidence that they are interested in, critically appraise existing model-based economic analyses, and make resource allocation decisions based on current model-based economic analyses. Recommendations for future research can be used to fill the evidence gap and improve the applicability and quality of future models for schizophrenia.

4.3.2 Limitations

This review is subject to two main limitations. Firstly, this review only included model-based economic evaluations. Economic evaluations based on other analytic frameworks, such as clinical trials, cohort studies and database studies, which represent a significant proportion of economic evidence, were excluded from this review. Economic analyses undertaken alongside clinical trials without extrapolation or the use of external evidence can also be a useful source of economic evidence; however, they do not always provide a sufficient basis for decision making. For example, a single trial might not compare all the available options, provide evidence on all relevant inputs, or be conducted over a long enough period of time to capture differences in important economic or clinical outcomes. Therefore, a review of model-based economic evaluations was considered to be most relevant for decision makers who are interested in resource allocation decisions across the entire schizophrenia pathway. Secondly, this review only included models published after 2005. This is because studies published before that time were deemed to have limited relevance to current practice due to the rapidly changing nature of treatments, health services and methods of economic evaluation.

5 Conclusion

This review highlights a lack of models for non-pharmacological interventions for schizophrenia, and limitations of existing models, including low quality and inconsistency in conclusions. A consistent basis for the model structure, use of evidence and assumptions in health economic models is required in order to improve the consistency and quality of future health economic models for the economic evaluation of interventions for schizophrenia. This consistency could be applied using ‘generic’ models, which might include a de novo WDM.