Introduction

Abstract

This chapter introduces and previews the book with a broad overview of the problems involved in conducting Universal Basic Income (UBI) experiments and in reporting the results in ways that successfully increase public understanding of the issue. It argues that experimenters should work backward from the big “bottom-line questions” that are most important to the public discussion of UBI to the variables that tests can actually address, and then forward again, closely explaining the relationship between experimental findings and the things people discussing UBI as a potential national policy really want to know.

“The devil’s in the details” is a common saying about policy proposals. Perhaps we need a similar saying about policy research —for example, “the devil’s in the caveats.” No simple list of caveats can bridge the enormous gap in understanding between the specialists who conduct policy research and the citizens and policymakers who are responsible for policy but often have overblown expectations about what policy research can do.

Consider this headline from MIT Technology Review , December 2016, “In 2017, We Will Find Out If a Basic Income Makes Sense.”¹ At the time, several countries were preparing to conduct experiments on the Universal Basic Income (UBI)—a policy to put a floor under everyone’s income. But none of the experiments had plans to release any findings at all in 2017 (nor did they). The more important inaccuracy of this article was that it reflected the common but naïve belief that UBI experiments are capable of determining whether UBI “makes sense.” Social science experiments can produce useful information, but they cannot answer the big questions that most interest policymakers and voters, such as does UBI work or should we introduce it.

The limited contribution that social science experiments can make to big policy questions like these would not be a problem if everyone understood it, but unfortunately, the article in MIT Technology Review is no anomaly. It’s a good example of the misreporting on UBI and related experiments that has gone on for decades.² MIT Technology Review was founded at the Massachusetts Institute of Technology in 1899. Its website promises “intelligent, lucid, and authoritative … journalism … by a knowledgeable editorial staff, governed by a policy of accuracy and independence.”³ Although the Review’s expertise is in technology rather than in scientific research, it is the kind of publication nonspecialists expect can help them understand the limits and usefulness of scientific research.

Policy discussion, policy research, and policymaking involve diverse groups of people with widely differing backgrounds: citizens, journalists, academics, elected officials, and appointed public servants (call these last two “policymakers”). Although some people fit more than one group, the groups as a whole don’t have enough shared background knowledge to achieve mutual understanding of what research implies about policy. Researchers often do not understand what citizens and policymakers expect from research, while citizens and policymakers often do not understand the inherent difficulties of policy research or the difference between what research shows and what they want to know.

Specialists usually include a list of caveats covering the limitations of their research, but caveats are incapable of doing the work researchers often rely on them to do. A dense, dull, and lengthy list of caveats cannot provide nonspecialists with a firm grasp of what research does and does not imply about the policy at issue. Therefore, even the best scientific policy research can leave nonspecialists with an oversimplified, or simply wrong, impression of its implications for policy. People who do not understand the limits of experiments also cannot understand the value that experiments do have.

Better-written, longer, or clearer caveats won’t solve the problem either. The communication problem, coupled with the inherent limitations of social science experimentation, calls for a different approach to bridge the gap in understanding.

This book considers how these sorts of problems might affect future UBI experiments and suggests ways way to avoid them. As later chapters explain, UBI has many complex economic, political, social, and cultural effects that cannot be observed in any small-scale, controlled experiment. Even the best UBI experiment makes only a small contribution to the body of knowledge on the issue. It addresses questions only partially and indirectly while leaving many others unanswered.

Citizens and policymakers considering introducing UBI are understandably interested in larger issues. They want answers to the big questions, such as does UBI work as intended; is it cost-effective; should we introduce it on a national level? The gap between what an experiment can show and the answers to these big questions is enormous. Within one field, specialists can often achieve mutual understanding of this gap with no more than a simple list of caveats, many of which are self-evident and need not be mentioned. Across different fields, mutual understanding quickly gets more difficult, and it becomes extremely difficult between groups as diverse as the people involved in the discussion of UBI and those involved in the discussion of UBI experiments.

The process that brought about the experiments in most countries is not likely to produce research focused on bridging that gap in understanding. The demand for the current round of experiments seems to be driven more by the desire to have a UBI experiment than by the desire to learn anything specific about UBI from an experiment. An unfocused demand for a test puts researchers in position to learn whatever an experiment can show, whether or not it is closely connected to what citizens and policymakers most want to know.

The vast majority of research specialists who conduct experiments are not fools or fakers. They will look for evidence that makes a positive and useful contribution to the body of knowledge about UBI. But the effort to translate that contribution into a better public understanding of the body of evidence about UBI is far more difficult than often recognized. This communication problem badly affected many past experiments and is in danger of happening again.

To understand the difficulty of the task, imagine a puzzle strewn out over the floor of a large, dark, locked room. A map of the entire puzzle, assembled together, provides answers to the big questions—does it work, and should we implement it? An experiment shines a light through a window, lighting up some of the puzzle pieces, so that researchers can attempt to map how they might fit together. They can easily map the pieces near the window, but further away, their view gets dimmer, the accuracy of their map decreases, and in dark corners of the room many pieces remain unobservable.

Although scientists like to solve entire puzzles when possible, under normal circumstances, they have to settle for something less ambitious. That’s why the basic goal of scientific research is to increase the sum of knowledge available to the scientific community—even if that increase is very small. In terms of the example, a research project can achieve the basic goal by mapping even one new piece, even if the puzzle as a whole remains unsolved and the map is only readable to other scientists.

As the MIT Review article illustrates, nonspecialists tend to expect something far more definitive, as if a social science experiment had the same goal as a high school science test: to determine whether the subject passes or fails. People often expect research to produce an estimate of whether UBI works or whether the country should introduce it. In terms of the metaphor , they expect researchers to provide their best estimate of the solution to the entire puzzle.

If researchers present their findings in the normal way for social scientists, they present something fundamentally different from what citizens and policymakers are looking for and possibly expecting. The potential for misunderstanding is enormous when research reports say something to the effect of here are the parts of the puzzle we were able to map to an audience looking for something to the effect of here is our best estimate of the solution to the entire puzzle. Caveats do not and cannot draw the necessary connection, which requires something more to the effect of here is how the parts we were able to map can be used toward a larger effort to find the solution to the entire puzzle and how close or far we remain from it.

In research reports, caveats typically focus not on the connection between the two goals, but on trying to help people understand research on its own terms. In the analogy, caveats tend to focus on the areas that experiments were able to map: how did they map this area; what does it mean to map this area; how accurate is the map of this area, and so on. The relationship between the areas mapped and the solution to the whole puzzle is often covered by one big caveat so seemingly simple that it often goes unstated: the areas we mapped are far from a solution to the entire puzzle. In other words, the information gathered about UBI in an experiment is far from a definitive, overall evaluation of UBI as a policy. As obvious as that caveat might be to researchers, it is not at all obvious to many nonspecialists.

Of course, nonspecialists know there are some caveats about the reliability of the experiment, but if they overlook or misunderstand that one big caveat, they will nevertheless believe that researchers provide their best estimate of whether “Basic Income Makes Sense,”⁴ and they will tend to look for that answer in any report on the study. If they get no help doing it, they are likely to overestimate the political implications of the information that experiments find, providing a great opportunity for spin and sensationalism by people willing to seize on small findings that sound positive or negative as proof that the program has been proven to be a success or a failure. Some of my previous work has argued that earlier UBI-related experiments have been misunderstood and misused in these ways.⁵ This book focuses mostly on how to avoid those problems.

Although so far, I have only talked about difficulties related to the science involved, ethical and moral issues complicate the issue even further. In terms of the analogy, this puzzle is a very special kind: the pieces fit together in different ways depending on one’s moral values. In concrete terms, if a policy is sustainable, achieves some goal, and has some side effects, reasonable people can disagree about how good or bad those goals and side effects are and how we should evaluate tradeoffs between them. Except in the rare case where research definitively proves a policy has failed to achieve its supporters’ goals, reasonable people can disagree on whether the evidence indicates the policy works and should it be introduced or if that same evidence indicates the policy does not work and should be rejected. This problem greatly affects the UBI discussion because supporters and opponents tend to take very different moral positions.

Many people, including many specialists, are less than fully aware of the extent to which their beliefs on policy issues are driven by empirical evidence about a policy’s effects or by controversial moral evaluation of those effects. For example, mainstream economic methodology incorporates a money-based version of utilitarianism. Nonmoney-based utilitarianism was the prevailing ethical framework when basic mainstream economic techniques were developed, but it lost prominence decades ago. Many articles in economics journals read as if the author is unaware of the controversial moral judgments incorporated into that methodology.

Additionally, not everyone is honest about the extent to which their policy judgments are driven by controversial moral judgments. Some will try to spin the results by hiding the extent to which their evaluation of the evidence is driven by their moral position and portray it as the only objective reality. Specialists are not above exaggerating the definitiveness of their research.

Into this ethical morass falls the dense and difficult research report of an experiment’s findings with an often tedious and easily ignorable list of caveats about the research’s limitations and usually a complete absence of discussion about the moral judgments needed to evaluate the study’s implications for policy. Under such circumstances, social science experiments easily fall victim to misunderstanding, spin, sensationalism, and oversimplification. Perhaps we should expect these problems to happen more often than not.

After all, it is easier to understand an oversimplification than genuine complexity.

Solutions to these problems are difficult and imperfect, but we have to try to address them, if UBI experiments are going to achieve their goal.

I presume the overall goal of UBI experiments is (and should be) to enlighten the public discussion by increasing the public understanding of evidence about UBI. I don’t think that this goal is controversial or new. I believe it should be endorsed by virtually any UBI-related experiment, no matter what other goals it might have, such as the basic goal of scientific research (mentioned above), working out technical issues that are important to policymakers, or in some cases, politically promoting UBI. There is nothing inherently wrong with using a study—even a small-scale, less rigorous study—to promote a policy, as long as the evidence is presented honestly and aimed at improved understanding. Therefore, the need to keep the goal of enlightening discussion through good communication and an orientation toward the most important issues is as important to virtually all UBI studies.

Some past researchers (either conducting or writing about experiments) have failed to appreciate how difficult it is to accomplish this goal, especially when they focus primarily on the basic goal of scientific research. Increasing the amount of knowledge available to the scientific community does not necessarily or easily translate into improved public understanding of that evidence. The gap in background knowledge has to be addressed because it creates risks that less politically oriented research does not have, including vulnerability to misunderstanding, spin, misuse, sensationalism, or oversimplification.

Perhaps the main message of this book is that UBI experiments seldom, if ever, succeed in enlightening the public discussion merely by trying to get nonspecialists to understand experimental findings on their own terms. It’s not enough to explain what the experimental group is, what a control group is, and what the differences were between the two groups in the study. It’s not enough to have a new and improved list of caveats about experimental limitations.

Experimental findings should not be presented as a stand-alone piece of research but as a small part of a larger effort to use all available evidence to answer the big questions about UBI and to explain the extent to which the big questions remain unanswered. Researchers have to attempt to find the information that will be of most value to the public discussion, and someone—not necessarily the researchers conducting the study—has to attempt the difficult task of communicating those results in a way that people involved in the public discussion of the issue will understand. The difficulty of these tasks is at least half of what this book is about.

The book discusses the difficulty of conducting UBI experiments and communicating their results, given both the inherent limits of experimental techniques and the many barriers that make it difficult for researchers, journalists, policymakers, citizens, and anyone else interested in UBI or UBI experiments to understand each other. The book’s goals are to improve both the experiments and the public understanding of them.

With the experiments’ goal of enlightening the public discussion in mind, this book asks two distinct but closely related questions: (1) how do you do a good experiment given the difficulties involved? (2) How can citizens, policymakers, researchers, journalists, and others interested in UBI and UBI experiments communicate in ways that lead to a better public understanding of the experiments’ implications for the public discussion of UBI? I am less interested in the question of whether we should have experiments, taking it for granted that they are happening, but that question will come up.

This project is an applied examination of a family of problems specific to UBI experiments, with no claim that these problems are necessarily unique to UBI experiments. Many such difficulties apply to all social science experiments, and some apply to all policy-related research.⁶ To the best of my knowledge, this book is the first to focus entirely on applying this kind of analysis to UBI experiments, but it does not explore whether the kinds of problems discussed for UBI experiments are as bad or worse than the problems involved in other social science experiments.

This book is written for anyone interested in UBI experiments and UBI as a policy—that is, for researchers, journalists, policymakers, citizens, and people who are a little in one group and a little in another. Dangers of misunderstanding exist between everyone involved; everyone involved can help solve them; no single group can easily fix them on their own; and hopefully we can all benefit from thinking through the problems this book examines.

Policymakers , journalists, and citizens who understand the place of experiments in the political economy of the UBI discussion can better communicate their desire for experiments relevant to that discussion. They will learn more from whatever experiments are conducted. And they will be better prepared to counter spin and sensationalism.

Researchers who understand the place of experiments in the political economy of the UBI discussion can communicate their results more effectively. But it’s not just about communication. Researchers who understand and respect the public discussion can design better experiments.

Researchers conducting experiments cannot resolve all these communication issues on their own. Although research specialists are professionals at communicating with other specialists, the vast majority of them are amateurs at communicating with nonspecialists—and I am no exception. Scientists are trained to conduct research and communicate it to other scientists, but have no special training in the skills needed to bridge the communication gap between them and nonspecialists. Very often specialists don’t know what evidence would be most valuable to citizens or policymakers or how best to help citizens and policymakers understand the value of the evidence researchers are able to find.

The ultimate responsibility rests more with the policymakers and donors commissioning experiments than with the researchers conducting experiments. They—or whoever they put in charge of hiring researchers to conduct experiments—are the ones with the most power to make sure the communication gaps are addressed.

With experiments getting underway and findings about to come out, it’s important to consider lessons in how to improve the chances that experiments will successfully enlighten the public discussion of UBI. As the book argues, past UBI-related experiments—despite almost always being good science—have a mixed record at increasing the understanding of evidence among nonspecialists. Some succeeded and some failed.

The primary goal of a UBI experiment might simply be to examine a few narrow technical issues that are of particular interest to policymakers commissioning the study or to the research community. There is nothing wrong with the desire to make some goal like this the main focus of a project. But they ignore the public role of UBI experiments at their peril. UBI experiments are too closely tied to the political process and their results are too easily misunderstood for researchers to ignore experiments’ role in the political economy of the UBI discussion without risking misuse and misunderstanding.

Although UBI experiments are scientific endeavors, they are both an outcome of and an input into the political process. The current experiments are—directly or indirectly—a response to the growth of the UBI movement. It is no coincidence that UBI-related experiments have taken place in two intervals (1968–1980 and 2008–the present) corresponding with waves of support for UBI and related policies.⁷

These enormous undertakings require great political support to come about. Social science experiments are usually too big to be funded by an everyday grant from a science foundation. The 1970s experiments were commissioned by acts of national legislatures that were seriously considering the policy. The same is true for the new government-funded experiments, such as those in Finland and Canada. Experiments in Namibia, India, Kenya, and two in the United States are all led or funded by private organizations with a strong interest in the UBI debate, although sometimes a mix of private and public institutional funding has been involved.

Whether researchers like it or not, people on all sides of the UBI discussion all over the world will look to UBI experiments for information about UBI and sometimes for ammunition to use in debate. The experiments will affect the public discussion of UBI. People will seize on findings and say it implies X about whether UBI works or whether we should introduce it. The data will be used this way. The question is whether it will be understood and used appropriately or misunderstood and abused.

To achieve the goal of enlightening discussion, people commissioning and conducting experiments need to know the local discussion well, but they also need to avoid overconfidence in their belief about how well they know it. Journalists and opinion writers who have platforms to write about UBI are not necessarily experts on the UBI discussion. Major media outlets do not contain most of or even the most important parts of that discussion. People commissioning and conducting experiments should not be tempted to believe that no one in the local discussion is interested in the big questions that haven’t been explicitly stressed by prominent writers and speakers involved in the discussion. Ignoring the obvious and rational desire for anyone considering a public policy question to have answers to the big questions about it creates an opportunity for a demagogue to use that lack of information to spin the experiment’s findings to their advantage.

The limitations of UBI experiments, discussed throughout this book, might inspire some people to reject experiments altogether. This is not my message; the message instead is how best to conduct a UBI experiment and communicate its results once the decision to conduct an experiment is made. Experiments are happening; let’s make them as good as possible.

The nature of this book requires me to say a little something about my perspective. I am an academic researcher. I have PhDs in Economics and Political Theory, but my job title is Associate Professor of Philosophy. I’ve supported UBI and related policies since 1980. I started writing about it in 1996 and publishing on it in 1999. I’m convinced by existing evidence that the advantages of UBI are so much greater than the disadvantages that most nations should introduce some form of it as soon as possible.

I also believe strongly in honest argument and evidence-based reasoning. Thus, I’m a committed supporter who tries also to be a dispassionate researcher. I have good knowledge of the topic, but I’m vulnerable to confirmation bias. Also, I might not always know whether I’m framing things in the most accurate way or in a way that spins them toward my existing beliefs. I’ll try to take that into account as I write, and you should too as you read. I believe this book will be equally useful to people on all sides of the public discussion of UBI if readers look skeptically at my argument and evidence.

Although I bring a wide interdisciplinary perspective to this project (having written about UBI as a philosopher, an economist, a political theorist, an applied public policy researcher, and an amateur journalist), my experience is still far narrower than would be ideal for the effort at hand. I don’t believe anyone could claim expertise in all the fields relevant to this book. UBI experiments cross all of the social sciences, many health sciences, as well as some technical fields like statistics, mathematics, and computer programing. To understand the political economy of the public discussion of UBI experiments, one would need practical experience across numerous countries in activism, journalism, science communication, grassroots organizing, political campaigns, and high-level public decision-making. And so, this book will necessarily delve into some topics that are beyond my expertise.

The book makes many specific recommendations, including strategies for conducting an effective test and for combatting spin and misunderstanding. Perhaps the best way to sum up my perspective is the following recommendation: treat experiment(s) as a small part of the effort to answer the questions necessary to evaluate UBI as a policy proposal. This recommendation does not mean that experiments must be conducted in conjunction with many other research efforts to answer all these questions. It means that experiments in isolation cannot be interpreted as saying very much at all about UBI as a policy. The true value of an experiment is making a small contribution to this larger effort. For nonspecialists to understand this: additional evidence has to be discussed, and the limits of experimental methods (and the overall effort to research a policy prior to implementation) have to be stressed.

In addition to many more specific suggestions, the book stresses four broad strategies to help experiments enlighten the discussion of UBI:

• 1. Work back and forth from the public discussion to the experiment. Anyone commissioning, conducting, or writing about experiments should respect the national or regional discussion of UBI. Find out what they can about what people most want to know. Design a study oriented as much as possible toward the questions that are important to the local discussion, with careful attention to the extent to which experiment can and cannot contribute to our understanding of those issues. All reports about experimental findings should relate the information to the big questions that are important to the local discussion. This strategy involves bringing in nonexperimental data and calling attention to the remaining, but it is necessary to help people appreciate the contribution an experiment can make.

• 2. Focus on the effects rather than the side effects of UBI. Research projects have a way of focusing attention on the things they can measure at the expense of more difficult questions that might be more important to the policy issue at hand. For example, past experiments have often focused on quantifiable self-effects, such as labor effort and cost at the expense of more important but less quantifiable issues, such as whether UBI has the positive effects on people’s well-being as supporters predict.

• 3. Focus on the bottom line. Although the public discussion varies enormously over time and place, the desire for an answer to the big questions is ubiquitous, and so I suggest focusing on what I call the bottom line: an overall evaluation of UBI as a long-term, national policy.⁸ Experiments alone cannot provide enough evidence to answer a bottom-line question, but researchers can relate all of their findings to it. Virtually all UBI research has some relevance to the bottom line, but citizens and policymakers often need a great deal of help understanding that relevance meaningfully. Even the best journalists are not always able to provide that help.

• 4. Address the ethical controversy. Researchers cannot resolve the controversy over the ethical evaluation of UBI, nor should they try. But they do the public a disservice by ignoring it. They can better head off spin by recognizing the controversy and explaining what the findings mean to people who hold different ethical positions that are common locally and perhaps internationally as well.

I wish I could say this strategy fully resolves the problem, but that isn’t possible. A social science experiment is a very limited tool, and its implications are inherently difficult to understand. The effort to treat experiments as a small and incomplete part of a wider effort to answer all the important empirical issues about UBI will help but won’t eliminate misunderstanding.

There will always be gaps in understanding between the people involved in the discussion of such a complex issue and such complex evidence. If a nonspecialist learns everything a specialist knows, they become a specialist. But experimentation and communication can always be improved. I hope this research project makes a small contribution to that effort.

This book is organized in 19 chapters, beginning with this introduction.

Chapter 2 defines and explains the workings of UBI and its more easily testable cousin, the Negative Income Tax (NIT).

Chapter 3 discusses some necessary definitions and the pros and cons of the available testing techniques: randomized controlled trials (RCTs), saturation studies, and combinations of the two.

Chapter 4 discusses several general problems that virtually any UBI experiment will have to deal with: community effects, long-term effects, the Hawthorne effect, the streetlight effect, and the difficulty of separating the effects of the size and type of program being studied.

Chapter 5 discusses one big difficulty: the practical impossibility of testing UBI under most circumstances and the problems created by using NIT as an approximation of UBI.

Chapter 6 discusses the five NIT experiments conducted in the 1970s in the United States and Canada, summarizes their findings, and shows how badly they were misunderstood at the time. It argues that although the experiments succeeded in the scientific goal of obtaining useful data, they badly failed in the goal of enlightening the public discussion.

Chapter 7 discusses more recent findings from two experiments conducted in the late 2000s and early 2010s and from newly released data from one of the 1970s experiments, showing how these findings had a more positive impact on the public understanding of UBI.

Chapter 8 briefly discusses some of the now ongoing UBI experiments, proposed UBI experiments, and experiments in policies similar to UBI. The book references these experiments only rarely because its goal is not to analyze or criticize them, but to offer some useful analysis to the people commissioning, designing, conducting, reporting on, and reading about them.

Chapter 9 discusses the surprisingly complex political economy of the decision process that brings about UBI experiments in response to a movement more interested in the immediate introduction of UBI. It shows that experiments’ vulnerability to misunderstanding and misuse make them a risky strategy for the UBI movement.

Chapter 10 examines why the results of experiments are so easily misunderstood, and therefore, vulnerable to misuse. These problems happen because of the inherent complexity of the material and the differences in background knowledge of the people involved.

Chapter 11 explains why UBI experiments cannot resolve the public disagreement about UBI. It argues that experiments can only make a small contribution to the large body of available evidence. The discussion turns less on remaining unknowns about UBI’s effects than on the ethical desirability of UBI’s known effects.

Chapter 12 begins the effort to work backward from the claims important to the public discussion of UBI to the claims experiments are able to examine. It suggests that UBI experiments should relate all findings to the bottom line, the overall cost-effectiveness of a fully implemented national UBI. An issue-specific bottom line for any variable of interest should also be considered.

Chapter 13 proposes a list of important empirical claims made by supporters and opponents of UBI in an effort to identify what empirical questions UBI experiments should focus on and how researchers can relate experimental findings to the things people really want to know about.

Chapter 14 identifies several empirical claims that should not be ignored but that cannot be tested on an experimental scale. Evidence about these claims will have to come from other sources, which will have to be combined with experimental evidence to connect it to the bottom line.

Chapter 15 identifies several claims that cannot be tested on an experimental scale but cannot be left out of the discussion of UBI’s bottom line. It offers suggestions about how to treat them.

Chapter 16 discusses claims that can be examined by UBI tests, but shows that each of them can only be tested partially, indirectly, and/or inconclusively. It discusses the implications these limitations have for conducting a study and communicating its results.

Chapter 17 discusses possible ways to test UBI in light of these issues, working down from the dream test that solves all testing problems to tests that might be possible within the experiment’s budget.

Chapter 18 considers whether it is after all worthwhile to have a UBI experiment, given all the difficulties tests have in addressing the most important issues in the public discussion.

Chapter 19 concludes with a discussion of how to work forward from the experimental results to the public discussion in ways that overcome communication barriers and reduce the problems associated with them. It argues that it is not enough to communicate the findings of experiments on their own terms, but results have to be presented with an understanding of the role they play in the political economy of the UBI discussion.