Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A little more than 20 years after its birth, marked by the launching of its foundational document at an international WHO conference in Ottawa (World Health Organization, 1986), health promotion appears to be well and alive. Many western countries have now incorporated health promotion into mainstream public health practice. In Australia, for example, health promotion is defined as one of the core public health functions (National Public Health Partnership, 2000). The “Health on equal terms”, Swedish national public health program (Swedish National Committee for Public Health, 2000), is clearly a deliberate attempt to operationalize and implement values such as equity and action principles such as intersectoral action, spelled out in the Ottawa Charter (World Health Organization, 1986). In addition to this institutionalization in western states where it was an essential element of the strategy to meet the Alma Ata declaration goal of “Health for All in the year 2000” (Kickbusch, 2003), health promotion is now spreading in developing countries where it is increasingly conceived as an appropriate response to the enormous task of addressing the challenges associated with the epidemiological transition from infectious to chronic disease (Reddy, 2002) and from rural to urban life (Neiman & Hall, 2007). A decade ago, a leading scholar from the field of epidemiology characterized health promotion as the third revolution of public health (Breslow, 1999). Recently, the Bangkok Charter (World Health Organization, 2005) reiterated the relevance and appropriateness of health promotion to face the challenges of a globalized world. Clearly, what started in 1986 as a regional reform for public health has become a strong global current that shapes and orients public health practice (Kickbusch, 2007).

In the Continuity of the Work by WHO-EURO Working Group on Health Promotion Evaluation

Associated with this mainstreaming and expansion of health promotion is an increasing demand for it to prove its worth. During the past 10 years various initiatives have been launched in western countries in order to demonstrate that interventions designed along the strategies of the Ottawa Charter can impact population health. From the Centers for Disease Control in the United States (Zaza, Briss, & Harris, 2005), to the International Union for Health Promotion and Education (McQueen, & Jones, 2007a), to the Cochrane collaboration (Waters et al., 2006), various groups and organizations have come to the conclusion that much more primary evaluation research should be undertaken in order for health promotion to show its actual value as a public health approach to population health intervention (McQueen & Jones, 2007b). To answer this call, however, researchers and evaluators have to take into account many of the specificities that define health promotion. Outlining many of those challenges, the WHO-EURO Working Group on Health Promotion Evaluation concluded that failing to consider the specificities of health promotion in the design and implementation of evaluation research may lead to inaccurate results and eventually to misguided policy decisions about population health interventions (Rootman et al., 2001). Finally, the recent debate about the nature of evidence that is needed to feed evidence-informed public health decision-making (McQueen, 2001; Rychetnik, Frommer, Hawe, & Shiell, 2002) as well as the best practice and evidence-based practice movements in health promotion (Ziglio, 1997) have reactivated the need for an in-depth reflection and discussion on whether or not health promotion evaluation should be approached any differently from any other evaluation endeavor and if yes, what and how pitfalls should be avoided (Potvin, Gendron, Bilodeau, & Chabot, 2005).

Early attempts at developing a perspective on health promotion evaluation were linked to health education and were synthesized in two books (Green & Lewis; 1986; Windsor, Baranowski, Clark & Cutter, 1984). These publications adopted a post-positivist Campbellian perspective on evaluation founded in a quasi-experimental approach to evaluation, in which random assignment of subjects into different study arms corresponding to being exposed or not to the program, or versions of it, is seen as the gold standard for establishing causal relationships between an intervention and an effect (Shadish, Cook, & Leviton, 1991). In line with the quasi-experimental paradigm, early evaluators in health promotion primarily defined evaluation as a methodological tool for establishing internally valid causal links between an intervention and its measured effects, advocating for strong investigator control over the parameters of the intervention that was to be conceived as a fixed package. Context was mainly conceptualized as a source of confounding to be controlled for. These pioneer works established a strong affiliation between health promotion evaluation and a methodologically oriented tradition of evaluation that emphasizes the importance of an objective appraisal of interventions and primacy of internal validity and causal reasoning as epistemological tools to achieve this goal. What was missing in this import of a Campbellian thinking into the field of health education/health promotion, however, was Campbell’s critical discussion on the need for closure or quasi-closure for establishing causal relations in real-life situations (Cook & Campbell, 1979) and on the inherently fuzzy nature of the interventions to be evaluated, that is interventions are complex treatment packages, the components of which are acting in synergistic manner (Campbell, 1986). The failure of the first American community prevention projects to demonstrate positive effects, specifically in the domain of heart health, led several top epidemiologists to question the adequacy of the experimental/quasi-experimental paradigm for the evaluation of health promotion and prevention projects (Susser, 1995; Winkleby, 1994).

The first project clearly addressing evaluation issues specific to the field of health promotion was the work of the WHO-EURO Working Group on Health Promotion Evaluation that was active from 1995 to 1998 and which produced a policy statement (WHO European Working Group on Health Promotion Evaluation, 1998), a practical guide (Springett, 1998) and a book (Rootman et al., 2001). Formed by scholars from Europe and North America, this group developed a perspective for health promotion evaluation that (1) calls for policy makers to fund evaluation of health promotion projects, (2) is broader in concern and methods than the traditional quasi-experimental approach and (3) advocates for opening up the field to a variety of actors outside the scientific world. Despite these features that situate this work as more aligned to the principles underlying the Ottawa Charter than what had been previously published, the outcome of this working group nevertheless reiterates the primacy of methodological issues in defining the contour and articulating the content of health promotion evaluation. It also failed to develop a clear and coherent vision, as well as a critical appraisal of the role of evaluation with regard to the evolution of the field of health promotion. Further, because of a highly delayed publication schedule, much of the impact was reduced as the debate about evaluation and evidence had moved on.

In the meanwhile, more recent endeavors in health promotion evaluation were all concerned with the issue of effectiveness evaluation. Four projects, international in scope, need to be acknowledged: (1) the Cochrane collaboration group on health promotion; (2) the CDC Community preventive services synthesis; (3) the IUHPE European work on the effectiveness of health promotion; and (4) the IUHPE Global project on health promotion effectiveness.

Some of these projects are ongoing; however, most have pursued either or both of the following goals. The first was to estimate overall benefits, both in qualitative and in quantitative terms, of public investments in health promotion. This was the primary goal of the two IUHPE projects. In its first report to the European Commission, the IUHPE “provides a summary of the main evidence, and puts forward a case for ensuring that Health Promotion is properly resourced. This will enable Health Promotion to play its full part in the public health policy framework which is currently being shaped by the European Commission” (International Union for Health Promotion and Education, 1999, p. i). Later, in their first report on the IUHPE Global Program on Health Promotion Effectiveness, McQueen & Jones (2007b), taking stock of the field of health promotion evaluation, raised three issues that limit our capacity to estimate the overall value of investments in health promotion: there is a wide variety of methods used in health promotion evaluation; those evaluations are often conducted internally by people involved in program delivery; and there are not enough evaluation studies conducted as yet to produce any clear evidence. Such a goal clearly establishes evaluation as a management instrument to support decision-making about health promotion as a general item in the gigantic hypertrophied health sector budget or about specific interventions.

The second goal was to extract from the available reports of health promotion writ large (i.e., mostly including integrated prevention projects), those activities, programs and interventions that seem to be effective in order to group them in user-friendly retrieval systems. It is this kind of goal that is actively pursued both in Cochrane collaborations and in the CDC Guide to preventive services. The intention is to provide authoritative arguments in favor of interventions in order to improve specific health promotion practices by providing data on the outcomes empirically associated with a variety of practices. “The Community Guide summarizes what is known about the effectiveness, economic efficiency, and feasibility of interventions to promote community health and prevent disease. The Task Force on Community Preventive Services makes recommendations for the use of various interventions based on the evidence gathered in the rigorous and systematic scientific reviews of published studies conducted by the review teams of the Community Guide. The findings from the reviews are published in peer-reviewed journals and also made available on this Internet website” (Guide to Community Preventive Services, 2008). In order to contribute to this goal, evaluation is implicitly defined as a tool to improve health promotion through coding, standardization and dissemination of its best practices.

In most of these endeavors the role of health promotion evaluation has always been implicitly or explicitly situated as being “external” to health promotion itself. In most instances, because the role of evaluation is often conceived as instrumental for decisions made outside of health promotion practice, the latter is also objectified. The relationship between evaluation activities and health promotion practice is one in which the former objectively studied the latter. It is generally understood that, in most instances, evaluation does not have a significant role in the implementation and effectiveness of the programs being evaluated. While in some instances, mainly associated with process evaluation, it is acknowledged that local health promotion practice can be transformed by specific evaluation, the reverse is never considered, mainly because when conceived primarily as a tool kit of scientific methods and techniques, evaluation cannot be influenced by its object of investigation.

Purpose of This Book

In response to the limited capacity demonstrated to date by evaluation endeavors to contribute solid evidence to both the overall value of health promotion and to some of its specific interventions, the purpose of this book is to explore the specificity of health promotion evaluation, developing the argument that, over and above a methodological kit, evaluation is a practice that seeks to transform the social reality of interventions (Schwandt, 2005). This book answers the question whether health promotion evaluation should be approached differently from other evaluation endeavors, with a clear yes. Contributors, who are either evaluators or health promotion practitioners from the three Americas, were asked to reflect on how it can be done and the challenges associated with such an enterprise and to report on the practical solutions they implemented in real health promotion evaluation projects.

All contributors to the book are associated with health promotion evaluation in North, Central or South America. Narrowing down our discussion of health promotion evaluation to experiences from the New World was a deliberate choice. We think that it gives us the opportunity to present a diverse yet still very coherent perspective on health promotion evaluation. Diverse because it encompasses realities of countries that represent all stages of economic development and a variety of public health issues. Diverse also because contributors are from various cultural backgrounds, bringing together the richness of the Latin and Anglo-Saxon understanding of the world.

Taking up the task where the WHO-EURO book of 2001 has left it, this book aims to explore how health promotion’s unique characteristics influence the conduct of evaluation research. In line with the most recent development in the field of evaluation our definition of evaluation encompasses much more than research design and methods to collect and analyze scientific data. We conceive evaluation as a social and research practice, which, like all professional practices, represents rational attempts to transform or reproduce the world. Having programs and interventions as objects of inquiry, evaluators’ practices shape the way decision makers and public policy advisors conceive programs. This is one of the great lessons of the decisive work of Carol Weiss and later Michael Patton on the utilization of evaluation results.

We think that health promotion evaluation in the Americas is unique and challenging enough so as to require a book in which evaluators would describe, and reflect on, their way of doing evaluation: their evaluation practice (Schwandt, 2005). The main challenge of health promotion evaluation rests chiefly with the openly value-oriented nature of health promotion. In our discussion of the Ottawa Charter, the founding document of health promotion, we have a tendency to retain only that it proposes five strategies of action for health promotion: creating supportive environments; building public health policy; strengthening community actions; developing personal skills; and reorienting health systems. We rarely mention that it also proposes a set of values and principles that would characterize the way these strategies should be implemented. Indeed, health promotion has spelled out a set of humanistic values and principles to guide public health intervention.

In our reading of the WHO-EURO Working Group on Health Promotion Evaluation (1999) report on health promotion evaluation, we identify seven values and principles that seem to form the core of health promotion. Two of these values, participation and empowerment, are more largely associated with health promotion than the others. A closer examination of the Ottawa Charter would show that values of equity and sustainability are also very much present as well as principles of intersectoral action, multi-strategy and contextualism, understood as embedding interventions into local circumstances. For some people, values and principles are incompatible with the notion of scientific rigor and health promotion should therefore be evaluated with the sole preoccupation of implementing rigorous methods.

Like many evaluators preoccupied with the use of their evaluation results, the editors and contributors of this book do not believe that scientific rigor in evaluation is incompatible with humanitarian values and principles. After all, what is evaluation about if it is not about value? And the past three decades of work in the domains of sociology and anthropology of sciences (Campbell, 1984; Latour, 2001 Toulmin, 2001) have shown that any scientific project is crowded with arbitrary decisions that more or less impact on the methodological rigor and on the validity of the results. In our conception, those decisions are characteristic of the scientific practice associated with certain fields and disciplines. We further argue that the practice of health promotion evaluators should be informed by the same values as that of health promotion practitioners, and this is exactly what this book is about. We first want to demonstrate that despite, and probably in part because of, its rigor, evaluation is a practice that can be informed and shaped by the same values and principles that underlie health promotion.

This project was conceived by a group of practitioners and evaluators from South and North America who believe that there is something distinctive to be described about the region of the Americas. The manner in which health promotion is practiced, and the challenges encountered in evaluating health promotion in the context of the Americas deserves a unique space for reflection, and the editors aim to provide structure for this examination through this book. The six editors, who hail from Brazil, Colombia, the United States and Canada, began meeting in 2003 to engage in a dialogue about whether there are differences in this region compared to the rest of the world, and what these differences might imply for the practice of health promotion, the practice of evaluation, and what might be learned from this reflection. Certainly the various professional histories and practical experiences of the editors have shaped their beliefs about the uniqueness of the region, as well as the forces shaping the fields of health promotion and evaluation. Yet, all agreed that the values and principles examined are essential elements for the practice of evaluation as it relates to health promotion in the Americas. Once these values were chosen, the editorial group searched the region for practitioners in the field whose work reflects these values in practice and selected those few who appear in this book.

Contributions to the book were written in the author’s native language and were translated for the book. In some cases native Portuguese speakers paired with native French speakers, and the translation process has an inevitable impact on the process and the product. Translators and editors made monumental efforts to maintain as much of the text as close to literal translation as possible. However, some meaning has inevitably changed from the original due to the necessity to present the book in a single language. Through the process of editing, the editors have discovered that the cultural mindset going with the language is of ultimate importance, and every attempt was made to maintain this mindset.

The Book

The book is constructed in three parts. In the first part, as editors, we present and defend our conception of health promotion as a value- and knowledge-driven enterprise and of health promotion evaluation as a practice whose underlying values could be aligned with those of its object. In the second part we discuss four nexus of an evaluation practice where this alignment of values and principles with health promotion seems to be more crucial. Finally for the third part, we asked contributors who have developed a practice of evaluators or health promotion evaluators to analyze and reflect on their practice and to draw the lessons on what they do and implement to realize this alignment between their evaluation practice and the values and principles of health promotion.

The third section of the book, which provides a series of chapters that reflect on the practice of evaluation, is meant to illustrate the challenges to, and lessons learned from, evaluating according to certain health promotion values. For example, the chapter by Strickland et al. demonstrates the challenges posed by participation, as well as the necessity to acknowledge cultural values in the evaluation process in order to increase empowerment. For some of the values, namely context and equity, only a single chapter is presented exploring these values through the process of evaluation. This may perhaps reflect the difficulty that exploring these values presents in true health promotion practice. Authors in this section were given the charge of describing their evaluations in a manner not often seen, which calls for a reflection on the practice, rather than a presentation of the results. Authors were asked to reflect upon the process of evaluation as a social practice, which is value-laden, and that is intended to reinforce the values of health promotion. We believe that the collection of analyses presented here currently cannot be found elsewhere.

We think that a wide variety of audiences will be attracted by this book. First of all, graduate students will be able to develop their own thinking and critical appraisal of health promotion evaluation practices. Health promotion practitioners will find inspiration and arguments in their dealing with evaluators. Public health decision makers and policy people will be interested in examining how this alignment of scientific rigor with practical values and principles is possible and can be operated. Finally, we think that evaluators in all fields of evaluation will be interested in our work, perhaps particularly those interested in the Americas. In the late 1990 s, the New York-based Aspen Institute published two books that dealt with the practical challenges of conducting community-based project evaluations. These books were the product of the reflection conducted by the Roundtable on Comprehensive Community Initiative for Children and Families appointed by the Institute “with the goal of helping resolve the lack of fit that exists between current evaluation methods and the need to learn from and judge the effectiveness of comprehensive community initiatives” (Connell, Kubisch, Schorr, & Weiss, 1995, p. viii). The outcome of the work of this roundtable had a significant impact in the field of evaluation in that it explicitly positioned evaluation as a practice that affects the way social betterment interventions are planned and implemented. The two books from the Aspen Institute (Connell et al., 1995; Fulbright-Anderson, Kubisch, & Connell, 1998) are already more than ten years old. In the continuity of this work, the present book is an excellent example of the enormous potential for health promotion evaluation to lead the way to important innovations in the general field of evaluation.