FormalPara Chapter Overview

The wide variety of ways of evaluating social marketing efforts can be intimidating to a new social marketer but empowering to those more familiar with the variety of methods. However, research is essential for a successful social marketing program. Each type of research has its own use, and you want to use that which is most appropriate for the questions you need to answer. Time and money are typically limited. One strategy is to think about what you’d most like to know at every stage of the process. First, use formative research before beginning the program to determine what we currently know (secondary research) and what we would most like to know about what people are doing, and why. Before engaging time, effort, and money, we should find out what people think about this possible approach. Second, process research is needed to determine how things are going during the program. It is better to evaluate the effort as it is going than to wait until the effort is over to realize a mistake. As the saying goes, “A stich in time saves nine.” Finally, after the social market effort has concluded, outcome evaluation is conducted to determine what succeeded and what did not. This helps us to understand what should be repeated, what should be changed, and what should be eliminated next time. Social marketers should make use of any or several of the large variety of available research methods depending on what best suits their particular question. An apt quote from David Ogilvy, the advertiser who co-founded Ogilvy, a major international advertising agency, is “People who ignore research are as dangerous as generals who ignore…enemy signals.” That is, research should be a servant who helps us learn and improve our social marketing efforts before we start, as we go along, and at the end of the effort.

Introduction

Social marketing efforts do not only benefit from research, but research is essential to effective social marketing. This research can be done at several points in time. Before the program begins (or, as it is often labeled, intervention), you can find out what people are already doing, why they are doing, what they are doing, how they see the desired behavior, and whether they have any desire to change; during the intervention, you can ask how well your effort is working, and what they think about your social marketing intervention; afterward, you can discover how effective the intervention was, where it succeeded and where it didn’t. You can also understand which of the social marketing tools were employed in a given intervention and compare that to how successful the effort was. That is, research is critical in finding out what people do and why, and this provides an understanding of what you should do, as well as what worked, what didn’t, and why. This chapter will review the major categories and approaches to social marketing research. In the appendix following this chapter, online resource links are provided for further guidance in research and evaluation.

According to leading social marketing theorists, two distinguishing characteristics of social marketing are that we have a customer focus and that we provide something of value to people—an exchange (Grier & Bryant, 2005; Lefebvre & Flora, 1988; Rothschild, 1999). To accomplish these goals, it is important to understand your audience. Research tells us what the audience is like and what they want. Ideally, this means not just their initial knowledge and behavior, but also their habits, motivations, and barriers. Overall, social marketers should understand what people are doing, why they are doing it, what might motivate people to act differently, their reactions to social marketing strategy ideas, and ultimately, examine what people actually do after our efforts. This range of evaluation possibilities can be characterized as formative, process, and evaluation research. Additionally, after a program has been conducted, we can evaluate which social marketing strategies were actually employed, in order to aid future programs. This practice has become known as “benchmarking.”

Useful Forms of Research

There are a wide variety of research methods that are and can be employed to evaluate social marketing efforts. As a brief explanation, these approaches “diagnostic” research is used to identify problems and causes, while “evaluation” research is used to investigate the effectiveness of an effort. “Primary” data are collected for a particular problem or client but when these same data are then used for other purposes they become “secondary” data. “Qualitative” research and data are based on words, usually through talking directly with people, while “quantitative” research and data are based on numbers, which could be from counting things that occur in the environment (such as the number of television commercials showing unhealthy foods), measuring things such as a person’s weight, or through surveys which ask people to answer some structured questions and then counts the number of people who mark a particular response such as “yes” or “no” or “strongly agree.”

Formative Research

Formative research is diagnostic and undertaken before the social marketing effort is developed. Too often social marketing programs, especially many health interventions focus on education with the assumption that people behave the way they do as a result of a lack of knowledge, or sometimes because they be lacking in skills or values. Sometimes this is the case, but often it is not. For example, several countries are now requiring that children sit in approved protective seating while in automobiles. Formative research might investigate: (1) whether parents are aware of these laws, (2) whether they currently own child seats, (3) whether they currently use the child seats they own, or (4) for those who do not own child seats, if they are motivated to purchase these seats. If the results of the research show that parents are not aware of the law but would be eager to take protective measures to protect their children, such as purchasing these seats, then an educational approach can be taken. Social marketing efforts can center on providing information about these laws and the safety improvements offered by these seats. If, however, parents are aware of these laws and believe the seats would protect their children, but feel the seats are too expensive, then finding ways to reduce the costs of these seats for these parents would be an ideal strategy. If parents are aware, willing, and find the price acceptable, but find the seats hard to use, then social marketing efforts should probably focus on redesigning the seats to make them easier to use. In these cases, formative research is important in learning how to attack the problem. If the social marketing effort assumes that the issue was a lack of knowledge, then the educational efforts could succeed in the first situation, but not in the other two. This is why formative research is fundamentally important to social marketers and social marketing efforts.

The topic of obesity offers a good example of the importance of formative research. Some health efforts assume that people do not know what a healthy weight is, the advantages of maintaining a healthy weight, or why people gain weight. In these cases, interventions often focus on educating people about these factors. Yet many efforts that started with formative research have found that people know what a healthy weight is and want to maintain a healthy weight, yet they find themselves overweight or gaining weight despite their knowledge and motivation. In these cases, interventions should focus on providing easier access to healthy foods that are tasty, low cost, and convenient. An excellent example of this can be seen in Celebrity Chef Jamie Oliver’s television miniseries, “Ministry of Food.” This popular TV show focused on teaching people how to prepare healthy low-cost foods and developed programs which encouraged people to spread those recipes to others. In other cases, local food banks and gardening programs have also helped those who are knowledgeable and eager but might lack money or skills. Finally, structural changes such as providing nutrition information on packaged foods and in restaurants can also enable those who are knowledgeable and eager to make better choices. In each of these cases, formative research can help social marketers understand what the target audience needs in order to eat a healthier diet.

Typical areas of formative research examine awareness and knowledge—what do people know about the issue; motivation—how motivated and eager they are to do something about it; and opportunity—if they feel they are able to do a particular behavior or the barriers they feel stand in their way. Although knowledge is often seen as an antecedent to behavior, in many instances, knowledge is not required to change behavior. Specifically, there are “low-involvement” models which propose that when people do not care very much about the issue, it is best not to overload then with information. Instead, “dissonance models” such as the behavior–knowledge–attitude (B-A-K) approach where the marketer tries to start with behavior change (such as using free samples or coupons) and expects attitudes to change as a result, leading to new knowledge, are better approaches. Which approach may work best may hinge on people’s involvement with the issue (Ray et al., 1973).

Formative research can be done in a variety of ways, including using existing data (which is called “secondary” research), conducting face-to-face qualitative interviews and focus groups, or by creating larger-scale measures such as counts or surveys. The choice between these methods often depends on how much is already known about the topic and population and how much precision is necessary. Secondary research is usually used to see what is already known about a topic or target market. Secondary research is often available for free and is relatively easy to access on the Internet or at the library, so all efforts should begin with some secondary research. Because qualitative research usually involves words, qualitative research is often a result of speaking with people, either individually in interviews, or in small groups known as focus groups. Sometimes short questionnaires ask people what they know, feel, or do. This approach is often used to generate new ideas or test intervention ideas. Questionnaires, meanwhile, are most typically used to get a sense of the prevalence of attitudes or behaviors in the audiences and to select a target market (Weinreich, 1996). One strategy is to begin with smaller-scale qualitative research where respondents can help you understand what they already know, how they think about the issue, and the vocabulary they use to discuss the topic, before moving on to apply these insights to understand how common or widespread knowledge, beliefs and behavior are, or how they are distributed in a population.

Commercial marketing often relies on formative research not only to understand the audience but also to test creative promotional concepts. The notion of concept testing can also be applied to social marketing, not only for promoting ideas, but for broader marketing ideas as well. For example, formative research could be used to see where people would think to look for social marketing products such as condoms or tick prevention products. If research shows that people do not look for these things in the right place, then the social marketer might consider offering them in the places where people tend to look.

A critical aspect of consumer-centered social marketing is understanding what people think, feel, and do and what they want to do. It may be best to think of social marketing not as an effort to tell people something, but rather as an effort to help solve the barriers they face in trying to do what they should do (McKenzie-Mohr, 2000, pp. 546–549). For example, formative research was used by the United States National Cancer Institute in a fruit and vegetable campaign to get a baseline measure of how many fruits and vegetables Americans typically ate. This was the basis for the “five-a-day” effort which encouraged people to eat five fruits or vegetables per day. The United States Centers for Disease Control and Prevention’s Office of Nutrition and Physical Activity regularly conducts a nationwide HealthStyles survey which revealed that many people wanted to get more physical activity but did not feel they had enough time in their day. The social marketing effort that resulted was to list a variety of physical activities that they could do in the course of their day such as parking further away, taking the stairs at work, taking a walk at lunch, and encouraging employers to provide more opportunities for people to be active at work.

Another example of the value of formative research in social marketing can be seen in Mike Rothschild’s “Road Crew” drunk driving efforts in Wisconsin (Rothschild, Mastin, & Miller, 2006). Before starting the program, small focus groups were conducted in bars in small towns where the risk of drunk driving was highest. The results suggested that young men were aware of the risks of drunk driving but did not see any good alternatives to getting home from a bar besides driving, due to the high cost and difficulty of other solutions. As a result of the formative research, the program decided to buy used limousines in order to provide low-cost transportation to and from local bars.

Process Evaluations

After formative research has suggested a strategy and as it is being implemented, the next step is to evaluate the program’s progress. Formative evaluation is done before the program, whereas process evaluation is done during the program. It provides information about the ongoing efforts, or the process. Process evaluations can include examining levels of the current awareness and knowledge that arise from ongoing social marketing efforts, examining people’s interest in the topic, assessing their motivations to do what is being advocated, learning what opportunities they feel they have to perform the desired behavior, or their current behavior, or any other factor of interest to the program. Often process evaluations are the least expensive type of evaluation research. In commercial marketing, tests are often used to assess the market potential for particular products by introducing the product in trials and measuring sales in a few selected locations. Similarly, commercial advertising research makes use of a variety of methods to assess the effectiveness of advertising. “Broadcast tests” examine memory for the message, often in the form of “day-after recall” measures. Another way to assess the effectiveness of advertising messages is to run the ads in a small area and look at actual behavior rates such as the number of pizza orders after the advertisement is broadcast. In the case of social marketing, at its simplest instantiation, imagine trying to gauge the effectiveness of parenting classes. In this case, you can gain insights into how things are going in a variety of ways including enrollment levels (which may tell you about the appeal of the program or the effectiveness of your promotions), attendance rates (which may speak to people’s satisfaction with the form or content of the classes), or asking attendees what they remember (what was learned and retained because what people are learning is a simple evaluation tool in any educational environment).

A more formal evaluation of the process can make use of measures that evaluate awareness and memory for the messages over time. These are called “tracking” questions and are usually done by drawing a sample of people and asking them what they think, feel, or are currently doing. Alternatively, if you are interested in a broader population, process research may involve a survey. You may need to conduct your own survey, though sometimes you can “piggyback” onto another survey that is being conducted by your own organization or by another organization with the same target audience. Surveys can give an estimate of knowledge, attitudes, or behavior of either the target group of the overall population. Rothschild’s “Road Crew” effort kept track of the number of rides given to ensure that the program was working and having an immediate impact (Rothschild et al., 2006). Seeing how many people were booking rides provided immediate feedback on the effort. Specifically, they examined if the number of rides being booked, if the general appeal was good, and if people were aware of the effort. If rides were low, additional research should examine awareness of the effort because this would be the easiest to measure (by asking a few people in the targeted bars), and if awareness was high, this would suggest the program was not sufficiently attractive.

Other examples of process evaluations include the National Cancer Institute’s use of tracking questions during the five-a-day campaign. They assessed awareness “Do you remember seeing…”; knowledge “How many servings should people eat per day,” attitudes “Are you interested in…” and actual behavior “Are you trying…” “How many fruits and vegetables did you eat yesterday?” Another process evaluation occurred in an intervention aimed at children called the “Gimme 5” program which attempted to get children to eat more fruits and vegetables (Baranowski et al., 2000). For this effort, the process evaluations included a wide variety of methods including an examination of (1) whether teachers implemented the recommendations contained in each lesson, (2) school lunch offerings on menus and what was actually offered in the cafeteria (which were then compared to reported fruit and vegetables consumption in food diaries), (3) parent telephone interviews that asked about children’s mentioning the information or sharing materials, and (4) point-of-purchase events counting store activities, attendance for these activities, and interviewing the produce manager (Baranowski et al., 2000). Other process evaluations for five-a-day could have examined how the program was working with partners by assessing the number of supermarkets who chose to place the five-a-day logo on their produce bags, and the amount of products sold. In another example, USA’s Office of National Drug Control Policy tracked people’s concern over children’s drug use to evaluate whether their advertising efforts were working and to decide on the optimal level of exposure.

Some social marketing efforts have focused on increasing the availability of important health-related products, often in developing countries. In these cases, knowing product availability is as important as knowing target audience response. For example, several efforts have tried to get people to make use of condoms or mosquito netting. In these cases, process evaluations also involved monitoring the availability of social marketing products to consumers. This is important because efforts to increase the use of condoms and other contraceptives or mosquito netting would have failed if these programs had created interest, but the products were too expensive or unavailable. In these cases, had the social marketing efforts increased consumer interest and demand, even though the initial efforts to raise interest in the products had been successful, this would have been frustrating to consumers, and if judged only by sales or by other post-intervention measures, this would have probably been evaluated as an unsuccessful social marketing effort.

Outcome Evaluations

Finally, the form of research that social marketers are probably most familiar with is outcome evaluations—was the effort effective? Often these outcome measures are done at the behest of the funding agency. These evaluations can include any of the typical measures of what happened as a result of the social marketing effort. These can come in the form of outcome, output, and impact evaluations. Outcome evaluations might examine people’s exposure, awareness, knowledge, attitude, or behaviors. Meanwhile, an “output” evaluation might keep track of sales figures such as the number of condoms or mosquito nets that were sold, or the number of pounds of cans, bottles, or papers that were recycled. These two forms of research can be seen as evaluating the efficacy or effectiveness of the social marketing effort. Impact evaluations are more rare and might examine the number of drunk driving arrests or accidents that occurred, or the number of lives that are estimated to have been saved by these efforts, perhaps even in terms of the cost efficiency of the effort. Regardless of what is measured, the goal of an outcome evaluation is to discover how effective the social marketing program was.

One social marketing program that has undergone considerable outcome evaluations is the five-a-day efforts. For example, the evaluation of the “Gimme 5” effort with children evaluated a 7-day food record as well as psychosocial measures from students, telephone interviews with parents, and observational assessments of fruit and vegetable consumption (Baranowski et al., 2000).

One effort at an impact evaluation can be seen in Rothschild’s “Road Crew” effort. This program not only kept track of the number of rides but examined some of the efforts. The evaluation showed a significant shift in increases among rides (both output measures), but also demonstrated that the rides were most frequent among the target audience of 21–34 year olds. Perhaps most impressive, the results showed a 17% decline in alcohol-related crashes in the first year, and a projected cost savings of avoiding crashes and the costs of cleaning up after crashes. This type of outcome impact evaluation is very important in encouraging the state government to continue or even expand this program.

Often there is a belief in commercial marketing that the ultimate test of a venture is sales. Relying exclusively on a single outcome measure, however, is problematic for the reasons that can be seen in the contraceptive example in India above. Even if the promotion was wildly successful in generating widespread interest in these products, had the people not known where to buy the contraceptives, had they not been available, or had they been priced too high, this effort would not have resulted in sales of the product. This example demonstrates why relying exclusively on behavior or sales is a risky strategy. Behavior and sales outcomes usually only happen when everything works, so it does not help you diagnose where failures may be occurring. Sometimes behavior change, or sales, can happen for reasons outside of your own efforts. For example, a popular movie might model the behavior change you are advocating. It would be misleading to fully attribute the behavior change to your own social marketing efforts; similarly, sometimes behavior change or sales does not occur as a result of other factors that are outside your control. For example, a neighborhood recycling center might be closed for road construction in the area. It would also be misleading to blame that on your social marketing efforts.

Some excellent outcome evaluations have demonstrated not only behavior change such as product ownership or use, but even more important or “higher-level” outcomes including actual reductions in risk factors or demonstrable health improvements. For example, the “Road Crew” effort showed lowered mortality rates, and other international efforts which involved the low-cost distribution of mosquito netting have been shown to reduce anemia and lower infant mortality (Alden, Basil, & Deshpande, 2011, p. 169). Finally, there are audits that evaluate the cost efficiency of efforts, that is, the value for money. Audits can be done internally or externally but are especially impressive and bound to impress any funding agency which could result in increased funding not only for the program in question but also result in greater recognition of the value of social marketing.

Benchmarking

The name “benchmark” is often used in other fields to refer to baseline measures. However, in 2002, in trying to discern the essential features of social marketing, Alan Andreasen proposed six criteria, he called  them “benchmarks,” that were important in evaluating or monitoring whether the efforts really constituted a full social marketing program (or simply gave lip service to the term social marketing). These six criteria were: 1. a behavior focus, 2. audience research, 3. segmentation of target audiences, 4. attractive and motivational exchanges, 5. all four Ps of the traditional marketing mix (not just advertising or communications), and 6. consideration of competition (Andreasen, 2002). The evaluation of whether these aspects were used in a program has become a popular approach to assess which program components are critical for program success. Typically, this “benchmarking” is undertaken after the social marketing effort has occurred. It is often used in meta-analyses of published research to compare the strategies that were used with the success of the effort (e.g., Carins & Rundle-Thiele, 2014; Cugelman, Thelwall, & Dawes, 2011; Gordon, McDermott, Stead, & Angus, 2006; Stead, Gordon, Angus, & McDermott, 2007).

Research studies evaluating a social marketing program’s success have compared program outcomes to the number or variety of social marketing attributes. Overall the evidence supports the notion that making use of more social marketing strategies within a program is associated with a greater level of success. Therefore, this benchmarking research can be interpreted to say that social marketing is effective in general, and the more social marketing strategies that are employed the more effective the interventions. Perhaps later efforts at benchmarking will eventually be able to identify the most critical aspects of social marketing, or even put them in order of importance for different topic areas. The ultimate goal of these benchmarking analyses is to identify essential program components in order to guide the development of future programs and to increase their potential for success.

Which Type of Research Should I Use?

As a pragmatic field, social marketing should draw on the full range of research data and methods that are available. This should involve making use of both existing secondary data as well as collecting new primary data. This can and should involve both qualitative and quantitative data. We should also make use of any of the other approaches such as content analysis, or even more novel strategies such as visual and physiological approaches as best suits our questions. The strategy here is to have as broad a research toolbox as possible, and to use whichever tool best suits the question at hand. Few people are well-versed in all of these approaches—that is one contribution of a team approach in social marketing. By recruiting a wide variety of contributors, especially researchers, we can bring together various forms of expertise.

When does each form of research make the most sense? A flowchart can provide a helpful heuristic. This flowchart is presented in Fig. 3.1.

Fig. 3.1
figure 1

Research flowchart

Full size image

When little is known about the topic at hand, a literature review is often a good starting point. Starting with what we already know is always advised. A search of existing data or secondary research will often turn up a good deal of information on the topic that can be very valuable. In public health, it is common to use epidemiological data or studies to provide an “evidence base” for interventions. Which behaviors are costing the most in terms of years of life, quality of life, or financial costs? Is there evidence for this behavior as being alterable? Next, armed with that information, we often move on to gain an understanding of people’s thoughts, attitudes, and behaviors. If we need deeper insights into why behaviors are (or are not) occurring, but we do not need precise measures of how many people are doing this, then qualitative research such as interviews often fits the bill. Interviews with the public, as Rothschild et al. (2006) did with his “Road Crew” effort, can provide social marketers with a deeper understanding of the issue. Commercial research firms often make use of qualitative research such as focus groups to gain insights into consumer thought processes. If you need a more precise estimate of the prevalence of knowledge, attitudes, or behaviors, that is, you want to estimate what percentage of people think, feel, or act in a certain way, then a larger-scale quantitative effort is called for, often in the form of a survey. If you want to know about a large segment or the public in general, there are commercial marketing companies that specialize in large-scale surveys that can be used to conduct original research or to “piggyback” a few questions within someone else’s survey. If you are concerned about people’s ability to remember or tell you what they are doing, often because behavior can be automatic, then observational or visual research may be in order. Several years ago, a toothpaste company was interested in people’s teeth brushing. After some initial interviews, the company became skeptical that people could tell them honestly how long they brushed, so the company paid people to install cameras in their bathrooms from which they calculated how often people brushed, for how long, and if this varied across people. Sometimes, instead of asking, you have to observe. Then, armed with an understanding of the costs of people’s behavior, and an understanding of why people are doing the things they are, social marketers can develop possible programs or interventions. Before fully launching the program, it is helpful to conduct small-scale experiments or pilot tests of the intervention. Using few participants, or a smaller town gives you the chance to test out your idea and see what works and what does not.