Case Study of Public Health

Abstract

Umbrella reviews are an established method of locating, appraising, and synthesising systematic review-level evidence. Umbrella review methodology is though only just beginning to emerge as a well-used technique in public health research. This chapter therefore summarises some of the first umbrella reviews conducted in the field of public health with a thematic focus on the social determinants of health and how interventions might affect health inequalities. The chapter discusses some of the cross-cutting methodological and thematic lessons learned from this body of work and concludes by suggesting new directions for umbrella reviews within the field.

1 Introduction

Umbrella reviews are an established method of locating, appraising, and synthesising systematic review-level evidence [1]. They use systematic review methodology to locate and evaluate published systematic reviews – most usually of interventions. Umbrella reviews are therefore able to present the overarching findings of such systematic reviews (usually considered to be the highest level of evidence) and can also extract data from the best quality studies within them if desired [2]. They therefore represent an effective way of rapidly reviewing a broad evidence base. This can be particularly useful for policymakers or practitioners in public health who may require a quick answer to a question, or a quick overview of a field (and the gaps in it).

Umbrella review methodology is though only just beginning to emerge as a well-used technique in public health research. For example, the first protocol for a Cochrane Public Health Group “overview of reviews” was registered in January 2015 [3], despite such overviews being advocated by the wider Cochrane Collaboration since at least 2008 [1]. This partly reflects the smaller number of public health systematic reviews within the Cochrane database but also the fact that systematic reviews more generally are still relatively new within public health (at least compared to health care or clinical research). This chapter therefore summarises some of the first umbrella reviews in public health (in which the authors of this chapter have been involved). They are methodologically innovative as they pioneered and expanded the use of umbrella systematic review methodology into public health research.

Methodologically, systematic reviews within public health research can differ quite significantly from those in other areas of health care or psychology. For example, a higher proportion of public health research is conducted using observational designs (given the population scale) and experimental studies are consequently less common; the traditional evidence hierarchy has been challenged by some public health reviewers with alternative models suggested instead [4]; overarching meta-analysis is unusual in public health reviews due to the heterogeneity of included study designs or outcomes [5]; qualitative research has also begun to be synthesised within public health reviews [6]; the nature of public health interventions can be very broad – potentially challenging traditional ways of framing the review question [7]; database searches for public health systematic reviews may also need to be broader than in other areas given the multiple areas of research (e.g. education, geography, psychology, social policy, health care, etc.) that might be relevant to a public health topic [8]. Consequently these differences are also reflected in the breadth and inclusion criteria of umbrella reviews conducted in public health to date – something that is apparent in our case studies.

The six case studies of umbrella reviews of public health interventions that we summarise here focus on:

• 1. Transport and health [9]

• 2. Housing and health [10]

• 3. Health-care service quality [11]

• 4. Equity in health-care services [12]

• 5. Workplace health [13]

• 6. The wider determinants of health [2]

They all have in common a concern with how interventions might affect health inequalities by addressing the social determinants of health. It is therefore worth providing a little context on these two issues so that the reviews can be understood in their appropriate conceptual context.

2 Health Inequalities and the Social Determinants of Health

The term “health inequality” is usually used to refer to the systematic differences in health which exist between socio-economic groups (in terms of income, education, or occupational class) or socio-economic areas (e.g. low-income areas). Socio-economic and spatial inequalities in health are not restricted to differences between the most privileged groups/areas and the most disadvantaged; health inequalities exist across the entire social gradient [14]. There are four levels of interventions to tackle inequalities:

• 1. Strengthening individuals (person-based strategies to improve the health of disadvantaged individuals)

• 2. Strengthening communities (improving the health of disadvantaged communities and local areas by building social cohesion and mutual support)

• 3. Improving living and school environments (reducing exposure to health-damaging material and psychosocial environments across the whole population)

• 4. Promoting healthy macro policy (improving the macroeconomic, cultural, and environmental context which influences the standard of living achieved by the whole population) [15]

These interventions are further underpinned by one of three different approaches to health inequality [16]:

• 1. Disadvantage (improving the absolute position of the most disadvantaged individuals and groups)

• 2. Gap (reducing the relative gap between the best and worst-off groups)

• 3. Gradient (reducing the entire social gradient)

Interventions are thus either targeted (such as individual level interventions which are underpinned by health as disadvantage) or universal (such as living and school conditions interventions which potentially influence the entire social gradient in health) [17].

The social determinants of health are the conditions in which people work and live – what have been referred to as the fundamental causes of health inequalities [14]. The main social determinants of health are widely considered to be:

• 1. Access to essential goods and services (specifically water, sanitation, and food)

• 2. Housing and the living environment

• 3. Access to health care

• 4. Working conditions

• 5. Unemployment [18]

Access to clean water and hygienic sanitation systems are the most basic prerequisites for good public health. Agricultural policies affect the quality, quantity, price, and availability of food, all of which are important for public health. Physical housing conditions and the cost of housing are both linked with public health. The wider living environment – such as pollution levels, transport infrastructure, access to green space, crime and safety, or place-based stigma – is also recognised as potentially important for individual-, household-, and area-level health. Access to good quality, affordable, and timely health care is a fundamental determinant of health, particularly in terms of the treatment of pre-existing conditions. Physical and psychosocial working conditions are a major cause of ill-health in the working age population and, because of the steep social gradient in conditions, are an important factor behind social inequalities in health. Unemployment is associated with an increased likelihood of morbidity and mortality as a result of the material (e.g. wage loss and resulting changes in access to essential goods and services) and/or psychosocial effects of unemployment (e.g. stigma, isolation, and loss of self-worth). Lower socio-economic classes are disproportionately at risk of unemployment.

3 Case Studies in Health Inequalities and the Social Determinants of Health

3.1 Case Study 1: Transport and Health [9]

Background to the review

Transport is an important determinant of health and there is a well-established association between socio-economic status (SES) and risk of road accidents. Effective traffic calming interventions such as 20 mph zones and limits may therefore improve health and reduce health inequalities.

Review objective

To identify systematic reviews of the effects of 20 mph zones (including speed limits and road humps) and 20 mph limits on health and SES inequalities in health amongst adults and children.

Study inclusion criteria

Population: Children and adults, all ages.

Intervention: 20 mph zones and limits. 20 mph limits consist of simply changing the speed limit to 20 mph using signage, whereas zones include additional traffic calming measures in whole areas in addition to changing the speed limit. Such traffic calming measures may, for example, include installation of road humps or mini-roundabouts.

Context: Any country, any location, English language only, and publications from 1990.

Outcomes: Health and SES inequality outcomes. Health inequalities were defined as differences by income, education, or occupational class, including area measures, e.g. area-level deprivation. Primary outcome measures included morbidity, health behaviours (especially physical activity such as walking and cycling), mortality, accidents, and injuries. Where additional data was provided, secondary outcomes included cost-effectiveness, public acceptance of schemes, and perceptions of safety.

Study design(s): Systematic reviews of quantitative evaluation studies. Publications had to meet the two mandatory criteria of Database of Abstracts of Reviews of Effects (DARE): (a) that there is a defined review question (with definition of at least two of the participants, interventions, outcomes, or study designs) and (b) that the search strategy included at least one named database, in conjunction with either reference checking, hand-searching, citation searching, or contact with authors in the field.

Search strategy

12 databases were searched from 1990 to September 2013, Campbell Collaboration, Cochrane Library (includes Cochrane Database of Systematic Reviews [CDSR], Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, DARE, Health Technology Assessment Database, NHS Economic Evaluation Database, and About the Cochrane Collaboration), EMBASE, PsycINFO, Centre for Review and Dissemination, Database of Promoting Health Effectiveness Reviews (DoPHER), Evidence for Policy and Practice Information and Coordinating Centre (EPPI-Centre), SafetyLit, Transport Research Information Service (TRIS), PROSPERO, MEDLINE, and Applied Social Sciences Index and Abstracts (ASSIA). Grey literature was also searched as well as the following websites: ROSPA, NICE, and Department for Transport.

Data extraction and quality appraisal

Screening, data extraction, and quality appraisal of included studies were carried out by one reviewer and checked by a second. The methodological quality of each systematic review was appraised using adapted DARE criteria (http://www.crd.york.ac.uk/CRDWeb/AboutDare.asp). The criteria were as follows: (1) is there a well-defined question; (2) is there a defined search strategy; (3) are inclusion/exclusion criteria stated; (4) are study designs and number of studies clearly stated; (5) have the primary studies been quality assessed; (6) have the studies been appropriately synthesised; (7) has more than one author been involved in each stage of the review process. Based on these criteria, included reviews were categorised as low (met 0–3 criteria), medium (4–5), or high (6–7) quality.

Synthesis and analysis

Narrative synthesis.

Results

Five systematic reviews were included. There were no reviews that focused exclusively on 20 mph zones or limits, but within these five reviews, there were a total of ten unique studies on 20 mph zones (n = 8) or limits (n = 2). Four of the systematic reviews were high quality and one was rated medium quality. The studies focused on accidents and injuries, traffic speed and volume, perceptions of safety, and physical activity. None of the studies, however, examined SES inequalities in these outcomes. Overall, they provide convincing evidence that these measures are effective in reducing accidents and injuries, traffic speed, and volume, as well as improving perceptions of safety in two of the studies. There was also evidence that such interventions are potentially cost-effective. There was no evidence of the effects on SES inequalities in these outcomes.

Conclusion

20 mph zones and limits are effective means of improving public health via reduced accidents and injuries. Whilst there was no direct evidence on the effects of interventions on health inequalities, targeting such interventions in deprived areas may be beneficial.

3.2 Case Study 2: Housing and Health [10]

Background to the review

Housing and neighbourhood conditions are widely acknowledged to be important social determinants of health, through three main pathways: (1) internal housing conditions, (2) area characteristics, and (3) housing tenure. Poor housing conditions disproportionately affect lower socio-economic groups. Housing or neighbourhood interventions which target these pathways may improve health and health inequalities.

Review objective

To identify systematic reviews of housing and neighbourhood interventions which target internal housing conditions, area characteristics, or housing tenure and measure impacts on health and health inequalities.

Study inclusion criteria

Population: Adult participants or the general population.

Interventions: Interventions aimed at altering housing or neighbourhood conditions which collected data on health or well-being outcomes.

Context: OECD countries (North America, Europe, Australasia, Japan). English language from 2000 to 2007.

Outcomes: Health and health inequality outcomes. Physical and mental health outcomes, including youth behavioural problems, morbidity, mortality, violence, injuries, and health and safety risks. Secondary outcomes included crime and social disorder, community cohesion, and economic outcomes.

Study design(s): Systematic reviews of quantitative evaluation studies. Systematic reviews had to meet the two mandatory criteria of DARE.

Search strategy

We searched 6 electronic databases including the CRD Wider Public Health database (2000–2002), the Cochrane Database of Systematic Reviews (2000–2007), the Criminal Justice Abstracts database (2000–2007), DARE (2002–2007), the Campbell Collaboration Database (2002–2007), and EPPI-Centre (2002–2007). Bibliographies and relevant websites were searched, and experts were contacted. Through expert contacts, we identified one review conducted outwith the search time frame, which was included because it represented a major contribution to the evidence base.

Data extraction and quality appraisal

All titles and abstracts were independently screened by two reviewers, and relevant reviews were retrieved and assessed for inclusion. Data relating to the review methods (search strategy, inclusion criteria, synthesis) were extracted along with information about the intervention, participants, outcomes, results (including number of studies and study design) authors’ conclusions and research recommendations. Each systematic review was critically appraised by one reviewer and checked by another using a checklist list adapted from DARE criteria (see prior section).

Synthesis and analysis

Narrative synthesis by pathway of effect.

Results

Five reviews met the criteria for inclusion. Four of the reviews were judged to be high quality, and one was medium quality. Impacts on health inequalities were not measured directly. However, all of the included interventions were aimed at people of lower SES.

Area characteristics: Two reviews found that relocating families living in high poverty areas to more affluent areas has the potential to improve health but evidence is inconclusive due to methodological issues. One review of area-based regeneration was inconclusive, with positive and negative health impacts reported.

Internal housing conditions: There is compelling evidence for positive effects on warmth and energy efficiency interventions targeted at vulnerable individuals. However, the health impacts of area-level internal housing improvement interventions are as yet unclear.

Housing tenure: No reviews of these interventions were identified. This remains an important area for further research and potentially new evidence syntheses.

One further review included interventions aimed at several of the pathways linking housing to health, reporting that many of the studies found positive impacts on health. However, neither interventions nor outcomes were specified, hampering interpretation of the findings.

Conclusion

Targeted warmth and energy efficiency interventions show positive impacts on a range of health measures. There was less robust evidence for positive effects of residential mobility programmes and a gap in the evidence base around housing tenure.

3.3 Case Study 3: Health-Care Service Quality [11]

Background to the review

Health systems in high-income countries are coming under unprecedented pressure from several directions: pressure on costs, expenditure, and ideological pressure. In some countries these pressures are being used to justify renewed calls to undertake major reforms to the financing and delivery of health care. This is part of a longer trend in high-income countries of the marketisation and privatisation of health-care provision since the mid-1980s. The implications of these changes for the effectiveness of health-care systems need to be examined, particularly in relation to their effects on quality of care.

Review objective

To review the systematic review-level evidence base on the effects of organisational and financial health system interventions on quality of health care.

Study inclusion criteria

Population: Adults and children of all ages.

Interventions: Organisational and financial health-care system interventions were defined as: (1) system financing, (2) funding allocations, (3) direct purchasing arrangements, (4) organisation of service provision, and (5) health and social care system integration.

Context: Limited to the health systems of 15 high-income countries used by the Commonwealth Fund: Australia, Canada, Denmark, France, Germany, Iceland, Italy, Japan, the Netherlands, New Zealand, Norway, Sweden, Switzerland, the United Kingdom, and the United States.

Outcomes: Quality of care was defined in terms of (1) professional performance, (2) efficient treatment and care, (3) clinical outcomes, (4) person-centred care, (5) holistic care, and (6) patient satisfaction.

Study design(s): Only systematic reviews of intervention studies with quantitative outcomes (experimental and observational) were included. Reviews were defined as “systematic” if they met the two mandatory criteria of DARE.

Search strategy

Seven electronic databases were searched for English language studies from start to January 2013 – ASSIA, Campbell Collaboration Database (CDSR), DARE, EPPI-Centre; Medline; and PROSPERO. Citation follow-up was conducted on the bibliographies of included studies.

Data extraction and quality appraisal

Screening, data extraction, and quality appraisal of included studies were carried out by two independent reviewers. The methodological quality of each systematic review was appraised using adapted DARE criteria (see prior sections).

Synthesis and analysis

Narrative synthesis by intervention type.

Results

Nineteen reviews met all criteria and were included in the synthesis. Nine of the nineteen reviews were of a high quality (mostly Cochrane reviews), three were of moderate quality, and seven were low quality. This umbrella review has identified only a small systematic review-level evidence base and substantial evidence gaps around certain interventions, most notably on changes to resource allocation systems (something also noted in our companion review of equity).

Paying providers

The eight reviews of paying providers to promote quality were largely inconclusive.

Purchasing and provision

The five reviews provided that no conclusive evidence on the outcomes of various forms of purchaser-provider split is particularly striking. The findings suggest that structural changes, such as the creation of new purchasing organisations, have very little impact on patients or frontline providers, and any changes that do occur are short lived. Furthermore, such arrangements seem to give rise to increased transaction costs that are not compensated for by cost savings.

Integration of services

In contrast, there was some evidence from six reviews that greater integration of services can benefit patients, although much seems to depend upon the approach taken.

Funding allocation

No systematic reviews examined the effects of funding allocation reforms.

Direct purchasing arrangements

No systematic reviews examined the effects of funding allocation reforms on quality of care.

Conclusion

The evidence base suggests that the privatisation and marketisation of health-care systems does not improve quality and that most financial and organisational system-level reforms have either inconclusive or negative effects.

3.4 Case Study 4: Equity in Health-Care Services [12]

Background to the review

Over the last 25 years, the health-care systems of most high-income countries have experienced extensive – usually market-based – organisational and financial reforms. The impact of these system changes on health equity has been hotly debated. Examining evidence from systematic reviews of the effects of health-care system organisational and financial reforms will add empirical information to this debate, identify any evidence gaps, and help policy development.

Review objective

To conduct an umbrella review of the evidence of the effects of organisational and financial health system interventions on equity of health care.

Study inclusion criteria

Population: Adults and children of all ages.

Interventions: Organisational and financial health-care system interventions were defined as: (1) system financing, (2) funding allocations, (3) direct purchasing arrangements, (4) organisation of service provision, and (5) health and social care system integration.

Context: Limited to the health systems of 15 high-income countries used by the Commonwealth Fund: Australia, Canada, Denmark, France, Germany, Iceland, Italy, Japan, the Netherlands, New Zealand, Norway, Sweden, Switzerland, the United Kingdom, and the United States.

Outcomes: Health equity was defined in terms of socio-economic inequalities SES in health-care access and utilisation, health outcomes (e.g. self-rated health, mortality rates, disease prevalence, etc.), or income. SES inequalities were defined in terms of differences in outcomes by SES (income, education, occupational class) or outcomes for the most vulnerable or deprived groups (e.g. unemployed, lone parents, deprived areas, etc.).

Study design(s): Only systematic reviews of intervention studies with quantitative outcomes (experimental and observational) were included. Reviews were defined as “systematic” if they met the two mandatory criteria of DARE. Reviews were defined as “partially systematic” if two or more of these components of the review question could be inferred from the title or text and the search criteria were fulfilled.

Search strategy

7 electronic databases were searched for English language studies from start to January 2013 – ASSIA, Campbell Collaboration Database, CDSR, DARE, EPPI-Centre database of health promotion and public health studies; Medline; and PROSPERO. Citation follow-up was conducted on the bibliographies of included studies.

Data extraction and quality appraisal

Screening, data extraction, and quality appraisal of included studies were carried out by two independent reviewers. The methodological quality of each systematic review was appraised using adapted DARE criteria (see prior sections).

Synthesis and analysis

Narrative synthesis by intervention type.

Results

Nine systematic reviews met all aspects of the inclusion criteria and were included in the synthesis. Only three of the nine reviews were of a high quality and only four were considered to be fully systematic.

General system financing

The four systematic reviews identified suggest that increased use of private insurance has negative health equity impacts. In contrast, there is evidence from the United States that increased use of free-care programmes has positive health equity outcomes. The effects of US-managed care programmes are inconclusive.

Direct purchasing

The single review of increased user fees and out of pocket payments found a negative impact on health equity.

Organisation of services

In terms of the marketisation and privatisation of health-care services, two of the three relevant reviews (including the better quality one) found that such reforms were negative for health equity, whilst the other review was inconclusive.

Health and social care integration

The evidence on the equity effects of integrated partnerships between health and social services is inconclusive.

Resource allocation

There were no relevant studies located that related to resource allocation reforms.

Conclusion

The systematic review-level evidence base suggests that financial and organisational health care system reforms have had either inconclusive or negative impacts on health equity both in terms of access relative to need and in terms of health outcomes.

3.5 Case Study 5: Workplace Health [13]

Background to the review

Although the work environment has long been acknowledged as an important determinant of health and health inequalities, physical working conditions have improved a great deal. However, inequalities remain in the psychosocial work environment and interventions to improve these may improve health and reduce health inequalities.

Review objective

To systematically review studies reporting the impacts on health and health inequalities of workplace interventions aimed at psychosocial working conditions delivered at an organisational level.

Study inclusion criteria

Population: Adult participants (16+) or the general population.

Interventions: Any change to the psychosocial work environment which focused on the organisational (rather than individual) level.

Context: Developed countries (North America, Europe, Australasia, Japan), reviews from 2000 to 2007.

Outcomes:

Health: Disease prevalence, general physical and psychological health measures, sickness absence, accident-related injury, and health behaviours.

Well-being: Physical and mental well-being, work/life balance, and quality of life. Health inequalities: differences in health or well-being by socio-economic status or demographic characteristics.

Study design(s): Systematic reviews of quantitative evaluation studies. Systematic reviews had to meet the two mandatory criteria of DARE.

Search strategy

The Centre for Reviews and Dissemination (CRD) Wider Public Health (WPH) database (a web-based database of systematic reviews of public health and related interventions) was searched from 2000 to 2002. In addition, CDSR, DARE, the Campbell Collaboration Database, and EPPI-Centre were searched from 2002 to 2007. The Criminal Justice Abstracts database was searched from 2000 to 2007, and hand-searching of relevant journals, bibliographies, and websites was conducted.

Data extraction and quality appraisal

Two reviewers independently screened all titles and abstracts identified. Data were extracted by one reviewer and checked by a second. Each systematic review was critically appraised using a checklist list adapted from DARE (see prior sections).

Synthesis and analysis

Narrative synthesis by intervention subtype.

Results

Seven reviews addressing the health effects of changes to the psychosocial work environment were located: three examined increased employee control and four evaluated the effects of changes to the organisation of work (shift work, privatisation, health and safety legislation). Five of the reviews specifically examined effects on health inequalities. Five of the reviews met all seven of the critical appraisal.

Employee control

One review of employee discussion groups found no conclusive effects on health. In another review, participatory employee committees were found to have positive impacts on self-rated health, and there was evidence of some effects on health inequalities. A further review of interventions which increased employees’ control over work tasks found that mental health worsened when job control decreased. There was some evidence of differential effects on depression.

Changes to work organisation

Four reviews, examining changes to shift work schedules (2), privatisation (1), and implementation of health and safety legislation (1), were located. Shift work interventions reported improved work/life balance and evidence of improved health outcomes, but little evidence on health inequalities. The review of privatisation reported that decreased job security led to adverse effects on mental health and on some physical health outcomes. Increased enforcement of health and safety regulation was associated with improved rates of fall injuries. There was limited evidence in the latter two reviews of differential effects by gender or occupational class.

Conclusion

Organisational-level changes to the psychosocial work environment can have important and generally beneficial effects on health. Five reviews which examined differences by socio-economic or demographic group tentatively suggest that organisational workplace interventions may also have stronger effects on men, lower SES groups and ethnic minorities.

3.6 Case Study 6: Wider Determinants of Health [2]

Background to the review

It is increasingly recognised that interventions aimed at the wider social determinants of health are necessary to tackle health inequalities. Developing the evidence base about interventions aimed at the social determinants of health requires that we identify existing evidence and highlight gaps in research.

Review objective

To identify and synthesise existing systematic reviews which report the health impacts of interventions aimed at the wider social determinants of health.

Study inclusion criteria

Population: Adult participants (16+) or the general population.

Interventions: Interventions aimed at the outermost layers of Dahlgren and Whitehead’s “rainbow” model of social determinants: macroeconomic, cultural, and environmental conditions and living and working conditions (including water and sanitation, agriculture and food, access to health services, unemployment, work conditions, housing, education, and transport).

Context: Developed countries (North America, Europe, Australasia, Japan).

Outcomes: SES inequalities in health or well-being, overall population health impacts. Also impacts on social determinants of health amongst disadvantaged groups with an existing health condition.

Study design(s): Systematic reviews of quantitative evaluation studies were included if they met the two mandatory DARE criteria.

Search strategy

The Centre for Reviews and Dissemination Wider Public Health was searched from 2000 to 2002. CDSR, DARE, the Campbell Collaboration Database, and the EPPI-Centre database were searched from 2002 to 2007. The Criminal Justice Abstracts database was searched from 2000 to 2007. A wide range of relevant websites was also searched, as well as bibliographies and four leading journals (American Journal of Public Health, American Journal of Preventive Medicine, Journal of Epidemiology and Community Health, Social Science and Medicine), from January 2002 to April 2007.

Data extraction and quality appraisal

Screening of titles and abstracts was conducted by two reviewers independently. Data from included reviews was extracted by two reviewers and cross-validated by another. Data from included reviews were only extracted if the primary studies and/or outcomes were relevant to the umbrella review. Quality was assessed by one reviewer and independently checked by a second, using criteria adapted from DARE (see prior sections).

Synthesis and analysis

Narrative synthesis by social determinant domain and intervention type.

Results

Thirty systematic reviews were identified, corresponding to the following domains within the “rainbow” model: housing and living environment (9), work environment (7), transport (5), access to health services (4), unemployment and welfare (3), agriculture and food (1), and water and sanitation (1). Twenty-six reviews were high quality and 4 were appraised as medium quality.

Housing and living environment

There is some evidence for positive effects on health and social outcomes following relocation to less disadvantaged areas. Improvements to internal housing conditions are also associated with small improvements in health. Finding from reviews of fall reduction interventions was inconclusive. Reviews of area-based interventions also reported mixed results. There was little evidence on the effects on health inequalities.

Work environment

Employee control interventions reported improved health when job control actually increased, and vice versa. Interventions which increased control over shift times had positive impacts on self-reported (particularly mental) health. Privatisation had negative effects on mental health associated with increased job insecurity. Increased health and safety enforcement in the construction industry was associated with a decrease in fall-related injuries. There was some evidence of differential effects.

Transport

There was strong evidence from three reviews that driver alcohol restrictions, traffic calming, and speed cameras led to reductions in fatal and non-fatal crashes. Impacts of new road building varied according to road type (bypasses reduced injuries while major new roads did not). The evidence base on interventions promoting walking and cycling was limited. Effects on health inequalities were not reported.

Unemployment and welfare

Two reviews of interventions to promote employment found little evidence of health impacts and inconclusive evidence of employment impacts. A review of interventions to increase uptake of welfare benefits indicated that there were clear financial benefits. However, there were only short-term improvements in mental health. None of the reviews reported differential impacts, but all were aimed at disadvantaged groups.

Access to health services

Three reviews of interventions to overcome cultural barriers to health-care access were inconclusive, although the use of lay health workers in low-income countries was associated with an increase in immunisation uptake. A review of rural outreach interventions reported improved health-care access and better self-reported health. All health-care reviews showed some promise in increasing access for disadvantaged groups, but none reported effects by SES or demographic characteristics.

Agriculture and food

One review of financial incentives to improve diet found positive effects on weight loss and fruit and vegetable consumption. No evidence on differential effects was included in the review.

Water and sanitation

One review of water fluoridation found no evidence of adverse effects on bone fracture incidence, bone mineral density, or bone strength in developed countries. The review did not report on the effects on health inequalities.

Conclusion

There is a lack of evidence about the health impacts of interventions aimed at the wider social determinants of health, which is even greater in relation to health inequalities. Those reviews which reported differential impacts found some indications of differential effects by gender, occupational class, and ethnicity. The domains of education, food, water, health service access, and unemployment show the most striking paucity of evidence. Changes to housing conditions are associated with small positive effects on physical and mental health. Workplace interventions appear to have differing effects on different levels of employee. A number of transport interventions seem to deliver reductions in crash injuries. Evidence for the health effects of interventions aimed at unemployment and welfare, and health service access is either absent or inconclusive. Financial incentives show some promise in improving health and health behaviour.

4 Discussion

Umbrella reviews of systematic reviews of interventions in the field of public health can be particularly useful for giving a broad overview of the evidence in a given field, particularly when the growth in systematic reviews outstrips the ability of the lay reader/practitioner to keep pace. They are also extremely useful for identifying gaps in the evidence base on a given topic. However, based on our experience of conducting umbrella reviews in public health, we have several observations, which may point the way towards methodological developments or improvements for the future.

As noted in the introduction to this chapter, public health is notable for the relative lack of experimental studies, in part due to the difficulty of evaluating many public health interventions using such study designs. However, as yet we are unaware of any umbrella reviews that only examined systematic reviews of observational studies in the field of public health. This may of course reflect the relative lack of systematic reviews of observational studies, but it is perhaps worth beginning to consider in what ways umbrella review methodology may need to develop in order to accommodate such study designs and, in particular, natural experiments and comparative studies, which are well suited to the evaluation of macro-level policy interventions [19].

The tendency for public health interventions to be particularly broad and to cross multiple disciplines presents particular challenges in developing search strategies for umbrella reviews. It is extremely common for different disciplines to employ different terms for the same concept, outcome, or indeed intervention, and it is unlikely that any one review team will contain expertise across all of the disciplines which might be involved. It may be that methodological progress in public health umbrella reviews will need to focus on developing new ways of developing search strategies. This is also a problem common to systematic reviews in public health too.

Many of the challenges involved in conducting public health umbrella reviews mirror those of systematic reviews, but are magnified by the increased scale on which they operate. Public health interventions tend to be particularly complex. A common issue with public health systematic reviews is that they do not report sufficient detail on intervention content or context, meaning that important information on factors which may modify the impacts of the intervention is lost [20]. This is to some extent unavoidable when reviewers attempt to include data from multiple studies in one review. However, it is magnified still further in any umbrella review. Similarly, encompassing the heterogeneity of public health intervention studies is challenging for the systematic reviewer – multiple interventions, populations, outcomes, and so forth – which again is escalated within an umbrella review. There are particular problems in terms of getting balance between being totally overwhelmed in terms of question breadth and not losing vital nuance in terms of understanding. Finally, many of the systematic reviews failed to adequately describe the results of their included primary studies or the interventions under evaluation or relied on very broad and vague descriptions.

While recommended appraisal criteria assess the quality of the included systematic reviews, they do not take account of the quality of the studies included in the original systematic review. This can have a major impact on the robustness of the reviews’ findings and can mean that a well-conducted systematic review which includes studies of low quality or at high risk of bias will score as highly as a systematic review which includes only well-conducted randomised controlled trials. Hence, conclusions or recommendations based on these quality judgements may give undue weight to studies of low quality. A means of appraising the included primary studies and their influence on the robustness of the reported findings would be a useful contribution to improving the evidence derived from umbrella reviews. Umbrella reviews also need to start incorporating the quality of included systematic reviews in their interpretation of findings.

We have focused our chapter on case studies from a specific area of public health research – health inequalities and the social determinants of health. There are some common topic themes that come out of this body of work too which will be briefly reflected upon here. Firstly, in all case studies, there is a noticeable lack of systematic reviews that examine the effects of public health interventions on health inequalities (as opposed to just public health in general). Secondly, in methodological terms there are many commonalities between the case study umbrella reviews – as shown in Table 20.1. For example, the quality appraisal tools used and the definition of what constitutes a systematic review (as opposed to just a traditional literature review or a structured review) are also shared. This is of course partly due to the fact that the case studies all involve the work of just two research teams based in the Universities of Durham and Glasgow of which we are both members. Another issue with a number of the case studies is that often only a few databases were searched. This is because umbrella review methodology is often employed to be used as a quick way of surveying the research landscape and providing quick evidence-based responses to time-sensitive public health policy or practice-driven questions. Future development of umbrella reviews in this sub-discipline will need to balance off these tensions.

Table 20.1 Methodological summary of umbrella reviews on health inequalities and the social determinants of health

Conclusion

This chapter has summarised some of the first umbrella reviews conducted in the field of public health with a thematic focus on the social determinants of health and how interventions might affect health inequalities. It has discussed some of the cross-cutting methodological and thematic lessons learned from this body of work. In terms of new directions for umbrella reviews within this field, the case studies suggest a number of areas for potential methodological development of umbrella reviews in the future including: how umbrella review methodology may need to develop in order to accommodate non-experimental designs, new ways of developing search strategies, assessing implementation of interventions within umbrella reviews, and the potential to extend the critical appraisal undertaken by umbrella reviews to include the quality of the studies included in the original systematic review. However, the future development of umbrella review methodology will need to balance off tensions between methodological refinement and maintaining the role of umbrella reviews in providing a summary of the evidence base. The use of umbrella reviews in public health is likely to grow especially since the publication of the first Cochrane Public Health Group review in 2015 [3].