Background

The results of health economic evaluations are of growing interest to political decision makers and insurance providers, not least because of increasingly tight budgets in the health care sector. To meet this new interest, clinical trials are increasingly taking into account resource and cost parameters (Drummond et al. 1997). In Germany economic aspects are becoming more important in the assessment of new therapies. Moreover, numerous methodological standards have been developed in recent years, which aim to ensure that the results of clinical studies are less open to bias. In addition, regulatory authorities tend to favour designs in which at least a minority of subjects in trials have been recruited in the host country, although they remain generally content with the fact that only a minority of subjects may actually be drawn from their country. Thus, international multicentre study designs have become a key component of clinical trial programmes (Demol and Weihrauch 1997).

Multinational studies have numerous advantages over single-nation studies. In particular, the large geographical areas involved allow investigators to recruit sizable numbers of patients in a relatively short period of time, and the quality of evidence may be improved by virtue of regional variations in the patient mix (Manca and Willan 2006; Sculpher et al. 2004; Drummond and Pang 2001). Multinational studies also have important caveats. Chief among these is the difficulty of transferring study findings to the level of individual national health care systems. After all, national decision makers are primarily interested in the results a study would have yielded if it had been conducted in their own jurisdiction.

Clinical trials are normally designed on the assumption of a common treatment effect across different strata, for example by centre or by country. It is well understood that patient characteristics and thus the absolute effects of treatment may vary between centres, and it is common to use some form of stratified randomisation to ensure an approximate balance of treatment and control subjects within centres or regions. The analysis of clinical trials usually involves some degree of examination of subgroups, although the aim is normally to support the assumption that there is a single treatment effect, and there is considerable controversy when it becomes apparent that this assumption may not be supported (Reed et al. 2005). The question of transferability becomes very apparent, however, when we consider economic parameters, for these—in contrast to clinical outcomes—are subject to considerable country-specific variability (Asplund et al. 2003; Jönsson and Weinstein 1997; Drummond and Pang 2001). Despite recent efforts to develop methodological standards for dealing with the resulting uncertainties, it is surprising to see that economic analyses are still being conducted without paying consideration to the issues raised when transferring the findings of multinational studies to national settings (Halliday and Darba 2003).

According to the most recent German health care reform act from April 2007, the Institute for Quality and Efficiency in Health Care (IQWiG) can be assigned to evaluate not only the effects of a medical intervention according to international standards of evidence-based medicine, but also the balance between benefits in the national health care setting according to international standards of health economics. In this context the question of transferring or adopting the findings of multinational trials to an actual national context will be a question of the methodological framework the IQWiG is currently about to implement.

Objectives

The present analysis summarises several of the most common international methods for generating health economic assessments based on clinical studies from different settings; it also provides an overview of potential new approaches to this subject from the perspective of Germany. In recent years, the problem of transferability has become increasingly important in this country. These developments raise the question of the extent to which the advantages of multinational studies can be reconciled with the need for unlimited transferability when adapting the findings of these studies to a specific national context.

Methods

A narrative literature review was performed to identify potential reasons for limited transferability of results of health economic evaluation from one country to another. Although we used some relevant single search terms (e.g. “transferability”) and search term combinations [e.g. “transferability” AND “cost-benefit analysis (MeSH term)”], the search procedure was not based on a systematic approach. Based on these results, we further searched the methodological literature for analytic approaches to handle the restrictions. Additionally we described the possibility of transferring foreign economic study results to the country of interest by matching trial data with routine data from national databases.

Results

Reasons for limited transferability

The transferability of cost and effectiveness results obtained in multinational studies to national contexts can be limited by a large number of factors (Bryan and Brown 1998; Koopmanschap et al. 2001; Manca and Willan 2006). Here, it is important to keep in mind that problems related to transferability affect primarily economic findings (e.g. resources or costs). A number of authors have described the sources of limited transferability (Drummond and Pang 2001; Koopmanschap et al. 2001; O’Brien 1997; Pang 2002); some of these are briefly presented in the following section:

Differences in demography, epidemiology and individual patient characteristics

It is well known that individual patient characteristics have at least an indirect influence on the effectiveness and cost-effectiveness of any given intervention (Kaplan and Keil 1993). Examples include socioeconomic or demographic factors, both of which may exhibit systematic country-specific differences in terms of their extent or degree. For instance, it is conceivable that patients in one country may show significant differences in educational attainment or wealth compared to patients in the other countries taking part in the same multinational study. This problem can be addressed, where applicable, by defining strict inclusion criteria (Koopmanschap et al. 2001). Patient-reported outcomes are also subject to regional variations and thus may also influence the evaluation of a treatment’s cost-effectiveness (O’Brien 1997). The same can be said for regional differences in the incidence and prevalence of a particular disease. For instance, high prevalence of a disease in a certain country or region may lead to downstream experience curve effects on the part of local health care providers. In turn, high incidence of a disease can increase the cost-effectiveness of a given population-wide preventive measure (Koopmanschap et al. 2001). The disease incidence and the level of patients’ comorbidities will also be influenced by the age structure of a specific country’s population (Drummond and Pang 2001).

Differences in country-specific health care and analysis structures

Additional factors that can limit the transferability of study results from one country to another can be traced to differences in the design and organisation of health care systems. Examples include differing systems of physician reimbursement and related incentive schemes (Koopmanschap et al. 2001; Drummond and Pang 2001) as well as differences in pricing (Drummond and Pang 2001). In Germany, the method and amount of physician reimbursement are based on consensus agreements reached by the National Associations of Health Insurance Funds and the National Associations of Social Health Insurance-accredited Physicians. In contrast, the remuneration of medical services in the USA is based on negotiated contracts with health maintenance organisations (HMOs) or other payers. For example, due to these differences, the US hospital care section is associated with comparably higher costs, which may lead to a higher utilisation in other health care areas (e.g. community services) in order to avoid the high hospitalisation expenditures (Pang 2002).

A striking example of the relevance of varying unit costs may be the choice of a cost-effective clinical pathway for patients with peptic ulcer disease and dyspepsia. These conditions are known to be associated with Helicobacter pylori infection, and eradication of this bacterium has been shown to be an effective treatment strategy. H. pylori infection can be diagnosed by means of endoscopy or, less reliably, by non-invasive laboratory tests. The cost-effectiveness of initial diagnosis of H. pylori infection by means of gastroscopy appears to be highly sensitive to the unit cost of the endoscopic procedure, which is much higher in the USA compared to European conventions (Bytzer 1999; Moayyedi 2007). As a consequence of this difference in cost, a clinical management strategy starting with initial endoscopy has been described as a cost-effective option in Europe, whereas in the USA empirical antisecretory treatment or non-invasive H. pylori testing were identified as preferable options based on their cost-effectiveness (Bytzer 1999).

Apart from unit costs, differences in country-specific methods (upon which the economic evaluation is based) could potentially influence the results of an economic assessment. For example the cost-effectiveness perspective could lead to different findings regarding the cost-effectiveness of a treatment. Was an analysis performed out of a societal perspective including all indirect costs or from the point of view of a single health care participant (e.g. health insurance company)? Even if an analysis was performed from the well-chosen perspective of interest, the possibility for a single-country adaptation could be limited. For example the economic evaluation results of a therapy from the perspective of a social insurance system will be deeply influenced by the structure of this system. Such a social insurance system could include a health care insurance, pension insurance, nursing insurance etc. or just a part of these services.

There are also country-specific differences in patient care pathways, which are partly shaped by guidelines established by national medical associations or differently organised educational systems for health care professionals. Further limits on transferability can result from differences in the fundamental organisational principles of health care systems (e.g. central vs decentralised planning). Other potential problems when transferring the findings of international studies to the national context can be identified in a qualitative manner using checklists, such as that provided in Table 1 (based on Heyland et al. 1996).

Table 1 Checklist for evaluating the transferability of findings from (multi)national studies to other countries (adapted from Heyland et al. 1996)
Full size table

Approaches to the economic analysis of multinational studies

In principle, there are various approaches to address the question of heterogeneity of economic variables across countries in multinational studies. All methodologies aim to translate real study data to an individual country. Under the reasonable assumption that internationally generated results for the effectiveness of a treatment are less subject to differences than cost results (Drummond and Pang 2001), the focus of the following observations is on the adaptation of economic cost results.

Provided that data are both available and accessible, in principal a bottom-up approach should be pursued where individual data on resource consumption are the basis for both individual cost calculation and a subsequent total cost evaluation (Hay and Jackson 1999). This methodology has the advantage that analysis of resource consumption can be made directly on the basis of the direct cost sets of a specific country. This approach, however, requires detailed individual data.

As shown in Fig. 1, the calculation of the cost-effectiveness of an intervention will be performed via the combination of adapted cost results and (multi)national effectiveness results.

Fig. 1
figure 1

Determination of cost-effectiveness

Full size image

Pooled/split analysis approach

Based on this pattern, different sources can be identified to analyse effectiveness and resource data within a multinational study. In principle, different combinations are conceivable which are shown in Fig. 2.

Fig. 2
figure 2

Conceivable analysis methodologies/approaches for the determination of resources and effectiveness data (our own representation)

Full size image

A completely pooled analysis assumes that there are no restrictions with regard to the adaptation of multinational study results at the level of a specific country. Since this assumption is not very likely as previously noted, the partially pooled, respectively split analysis gains more relevance.

With this approach only data of a certain country selection (e.g. with comparable patient clientele and/or health care systems) or a single country concerned are selected for further evaluation. These methodologies are easily comprehensible and thus a high transparency can be achieved for the methods and presentation of results.

However, a disadvantage of this approach is the reduced sample size, which, in a fully split analysis, can reduce the significance of the results (Pinto et al. 2005; Reed et al. 2005). In any case, these analytic approaches are only feasible when the country of interest was involved in the study. Figure 3 summarises the fundamental problems of pooled and split analyses.

Fig. 3
figure 3

Fundamental limitations of pooled/split analyses (our own representation)

Full size image

In practice, there are often pooled/split analyses in which, on the one hand, pooled effectiveness data of all participating countries are taken into consideration, whereas, on the other hand, only resource data of the country concerned are considered. This approach is a trade-off between a country-based assignment of resources and a high statistical power concerning the effectiveness data (Reed et al. 2005).

Statistical approaches

Since pooled/split analyses provide a compromise between restriction-free transferability of the study results for a certain country and a high statistical power, during the last few years, statistical analyses have been developed to approximate the optimal compromise. Among these, regression-based approaches belong to the most frequently common statistical methodologies. A potential disadvantage of the statistical methods presented is, in part, the complexity of the approach which involves again a reduced transparency, and thus reduces the potential use as a decision making aid (Reed et al. 2005).

Hence, a study published by Koopmanschap et al. recommends the use of multivariate regression analyses. This is proposed in order to quantify and adjust differences in resource consumption and costs in multinational studies in a way such that the studies would have been carried out in only one country (Koopmanschap et al. 2001). Herewith two options were pursued. A first approach uses the identification of the differences in the treatment samples and the associated consumption of resources of the countries involved in order to form homogeneous patient groups afterwards as precisely as possible. For all groups of resources, statistically significant and relevant country differences are determined, which are then subject to further correction. For example if in a country ×% less laboratory tests are carried out (with consideration of possible differences in the patient characteristics), the number of laboratory tests of the patients of other countries should be corrected downward by this factor ×. The corrected resource utilisations are then multiplied by a country-specific cost set.

A second approach tries to identify a possible country-specific influence directly at a cost expense level. In a multivariate regression analysis, the influence of different covariates (beside the country of the study, e.g. the age of the study participants and the disease status) on certain expenses is examined. A limitation of the proceedings represented by Koopmanschap et al. arises because of the isolated focus on resources and/or expenses, with which possible country-specific effectiveness differences remain unconsidered.

Other authors try to get around this limitation. In a publication of Willke et al. (1998) the attempt is undertaken to identify both country-specific differences in effectiveness and the quantification of differences in resources and cost. The statistical methodology assumes that a number of exogenous variables (e.g. disease stage), therapy outcomes as well as the interaction of country-specific treatment and country-specific outcomes per se influence costs of a treatment of an individual patient by treatment. The proposal described by these authors considered also country-specific outcome differences and thus partially goes beyond existing recommendations [e.g. Ontario Guidelines for Economic Analysis of Pharmaceutical Products (Ontario Ministry of Health and Long-Term Care 1994)] for the adaptation of multinational studies, because frequently only the consideration of possible country-specific cost and/or resource differences is required.

Besides these statistical proposals, further statistical approaches have been generated in recent years (Cook et al. 2003; Grieve et al. 2005; Pinto et al. 2005; Thompson et al. 2006; Willan et al. 2005).

Modelling of long-term costs and outcomes

In many situations the use of decision analytic models is preferred because it frequently occurs that there are no patient-based data available from the participating centres in the study or that the country of interest did not participate in the study (Drummond et al. 1997). So-called modelling studies enable the adaptation of treatment effects identified at the study level to different populations and health care systems. Often clinical studies are carried out without accompanying collection of economic parameters. An advantage, which the health economic modelling can offer, is the possibility of using and combining data from different sources.

For example decision analytic models are nowadays used to adjust study data to the real routine supply or to transfer study results to a country which did not participate in the study. An example of this approach is described in the work of Menzin et al. (1996). The authors use a decision analytic model alongside the data of a phase III study in the USA for adaptation to the European context (France, Italy and Germany).

A further type of application of decision analytic models exists in the health economic analysis of clinical studies, during which economic parameters were not collected or were not available. Based on available study information (e.g. characteristics of patient clientele, clinical processes, event rates etc.) resulting resource consumption is identified, on which basis a cost evaluation of the study process could be performed. Via extrapolation, an analysis of long-term costs of interventions or savings due to prevention of events is feasible (Siebert 2003).

According to the IQWiG guidelines, the use of a model for the economic evaluation of interventions must be sufficiently justified (Bastian et al. 2006). In addition, a model must meet numerous quality standards. Thus, decision models must be described transparently and comprehensibly and all assumptions must be explained. Likewise, there is a demand for the implementation of sensitivity analyses along with a fully probabilistic approach.

Furthermore, decision models should be subject to validation. The mathematical computations must be validated for their consistency with the model specifications, and it must be guaranteed that model input data and the outcomes are consistent with the available data. If different models come to different statements on the same questions, this identifies the necessity for a cross-validation of results (Bastian et al. 2006).

Provided that the appropriate quality criteria have been considered, the methodology of current health economic modelling is well established as a subsequent analysis of study data (Sculpher et al. 2004). Most international guidelines accept modelling in health economic evaluation (Hjelmgren et al. 2001; Schöffski and Graf von der Schulenburg 2007), although there remains some scepticism regarding the appropriateness of different modelling approaches, especially where these appear to translate unimpressive clinical effects into highly cost-effective outcomes.

Transfer of data to a national setting

One approach which uses some elements of modelling but adopts the efficacy of one single study of interest—possibly the pivotal key study on which registration was based—is to transfer data of a multinational trial to an existing national data set. A key focus of this approach is the linkage of patient data of the multinational study with patient data from national data based on specific patient characteristics (e.g. age, sex, insurance status, ICD diagnoses etc.). An important prerequisite for the practical implementation of the matching process is therefore the existence of matching variables on the patient level. If an international study delivers patient-level information only on sociodemography and therapy effectiveness, economic data of comparable patients from national databases can be assigned by means of patient matching. Important, however, is the selection of relevant matching variables such as age, sex, disease stages etc. In a following step, differences of effectiveness between the treatment arms are extracted and assessed according to the national cost schedule (e.g. the group difference concerning cardiac events in studies with cardiovascular background).

As a result, assessment of national cost-effectiveness can be obtained by matching available patient characteristics from the international trial with the national cost components. A problem is that the fundamental goal of clinical studies is the proof of the effectiveness of an intervention under controlled conditions (efficacy). Therefore, the collection of economic parameters in such studies is often of less relevance. The matching process illustrated so far would therefore only reflect the cost-effectiveness at the time of the execution of the clinical study in the country concerned (dependence on the related matching database).

A transfer of the cost-effectiveness results to the current real life setting would be achieved by incorporating data from national data sources (e.g. health insurance data). The integration of data from automated databases or health service research offers the potential to illustrate current consumption of resources and/or the costs of treatment in everyday life. Owing to specific characteristics of the German health system, this approach could realise both the adaptation to the national context and the consideration of real health care conditions.

Recommended analytic methods

Based on the analytic methods described above, a number of recommendations can be made as to which method applies best to which situation. These recommendations are summarised in a flow chart shown in Fig. 4. However, these recommendations cover a restricted range of scenarios only. Indeed, it is very plausible that there are situations in which a method different from those described would be more appropriate.

Fig. 4
figure 4

Recommendations for selected methods to analyse multinational studies

Full size image

Discussion

As the importance of health economic evaluations has grown, so has the number of methodological tools available. Today, broad ranges of approaches are used for the economic analysis of data from multinational studies, including study-based techniques, statistical methods and modelling. Despite this, none of the approaches described above is free of limitations. Finally, choosing a particular method always involves a balancing act between methodological transparency, statistical power and the transferability of results from multinational studies to different settings.

Inevitably, making decisions about the value of pharmaceuticals in any context requires the combination of data from different sources—this either can happen in a qualitative way through expert judgement or can be addressed by means of some form of modelling scenario. Such economic models have potential strengths and challenges. The greatest potential strength is in fully probabilistic modelling, which may provide appropriate estimates of cost-effectiveness in different circumstances, which may be specified by the modeller. The updated British NICE guide on the methods of technology appraisal also recommended this approach (Claxton et al. 2005). The greatest challenge is to avoid the situation where the design and population of economic models is undertaken with a particular aim and without objective use of available data.

The importance of country-specific evaluations of effectiveness and, in particular, of economic data (e.g. costs and cost-effectiveness) is something that needs to be considered even in the earliest phases of study development. Indeed, the timely documentation of potential country-specific differences can simplify the later analysis and interpretation of study findings. Nevertheless, it would seem highly desirable to place the available methods within a firm and binding framework—for example by establishing binding national and international guidelines.

Conclusion

The technical problems surrounding the health economic assessments and adaptations based on multinational studies are well known and generally accepted. From the perspective of national decision makers, there is nevertheless a great need to adapt the findings of these studies to the local setting, especially in light of their economic impact. There are a variety of methods for identifying and adjusting country-specific differences in costs, effects and cost-effectiveness. As a result, multinational studies will continue to play a crucial role in the evaluation of cost-effectiveness at the national level. It seems likely that the growing interest in multinational studies will lead to continued developments in adaptation methods, thus helping to ensure the consistency and transferability of study findings between various settings.