Abstract
Randomized controlled experimentation has a long history. But the transition from statistical theory to statistical practice remains a difficult one, especially when the experiments are large in scale. In this paper, we review several major randomized social experiments carried out in the United States beginning in the early 1970s. We describe briefly the policy settings out of which these experiments grew, and we indicate how this type of experimentation fits into the historical development of randomized experimentation more generally. We give detailed design features for five different sets of social experiments, and we briefly summarize some of the analyses of the experimental data. We end the paper with a discussion of analytical strategies for social experiments, and possible alternatives to large-scale experimentation, including combining the results of several experiments, and evolutionary designs. Throughout, we place special emphasis on the possible roles of models and of social and economic theory in both the design and the analysis of experiments, and we describe several open research problems requiring the attention of mathematical statisticians and economists.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
Bibliography
Acton, J. P. et al. (1982). Promoting Energy Efficiency Through Improved Electricity Pricing: A Mid-Project Report, N-1843-HF/FF/NSF. Santa Monica, CA: The Rand Corporation.
Acton, J. P., Manning, W. G., Jr., and Mitchell, B. M. (1978). Lessons to be Learned from the Los Angeles Rate Experiment in Electricity. Report R-2113-DWP. Santa Monica, CA: The Rand Corporation.
Acton, J. P. and Mitchell, B. M. (1979). Evaluating Time-of-Day Electricity Rates for Residential Customers. Report R-2509-DWP. Santa Monica, CA: The Rand Corporation.
Aigner, D. J. (1979). “A brief introduction to the methodology of optimal experimental design.” J. Econometrics, 11, 7–27.
Aigner, D. J. (1982). “The residential electricity time-of-use pricing experiments: What have we learned?” In J. A. Hausman and D. Wise (eds.), Social Experimentation, NBER conference volume.
Aigner, D.J. and Learner, E. (1984). “Estimation of time-of-use pricing response in the absence of experimental data: An application of methodology of data transferability.” J. Econometrics, 26, 202–227.
Archibald, R. W. and Newhouse, J. P. (1980). Social Experimentation: Some Whys and Hows. Report R-2479-HEW. Santa Monica, CA: The Rand Corporation.
Atkinson, A. C. (1985). “An introduction to the optimum design of experiments.” In this volume, Chapter 20.
Berry, S. H. (1979). Conducting a Survey Using the Client’s Staff: Evaluation of Interviewer performance in the Electricity Rate Study. Report R-2223-DWP. Santa Monica, CA: The Rand Corporation.
Becker, N. G. and Kersting, G. (1983). “Design problems for the pure birth process.” Adv. in Appl. Probab., 15, 255–273.
Bliss, C.I. (1952). The Statistics of Bioassay. New York: Academic.
Borjas, G. J. and Heckman, J. J. (1978). “Labor supply estimates for public policy evaluation.” In Proceedings of 31st Annual Meeting of Industrial Relations Research Association, 320–331.
Boruch, R. F. (1975). “On common contentions about randomized field experiments.” In R. F. Boruch and H. W. Riecken (eds.), Experimental Tests of Public Policy. Boulders, CO: Westview, 108–145.
Box, G. E. P. (1978). “Experimental design: Response surface methodology.” In W. H. Kruskal and J. M. Tanur (eds.), International Encyclopedia of Statistics, 1. New York: Free Press, 294–299.
Box, G. E. P. and Draper, N. R. (1969). Evolutionary Operation: A Statistical Method for Process Improvement. New York: Wiley.
Box, G. E. P. and Wilson, K. B. (1951). “On experimental attainment of optimal conditions.” J. Roy. Statist. Soc. Ser. B, 13, 1–45.
Bradbury, K. and Downs, A. (eds.) (1981). Do Housing Allowances Work? Washington, D.C.: Brookings Institution.
Cain, G. G. and Watts, H. W. (eds.) (1973). Income Maintenance and Labor Supply. Chicago: Markham.
Campbell, D. T. and Stanley, J. C. (1963). “Experimental and quasi-experimental designs for research on teaching.” In N. L. Gage (ed.), Handbook of Research on Teaching. Chicago: Rand Mally, 171–246.
Cochran, W. G. (1937). “Problems arising in the analysis of a series of similar experiments.” J. Roy. Statist. Soc. Suppl., 4, 102–118.
Cochran, W. G. (1951). “Improvement by means of selection.” In Second Berkeley Symposium on Mathematical Statistics and Probability, 3, Berkeley, CA: Univ. of California Press, 449–470.
Cochran, W. G. (1954). “The combination of estimates from different experiments.” Biometrics, 7, 101–129.
Cohen, E. R., Crowe, K. M., and Dumond, J. W. M. (1957). Fundamental Constants of Physics. New York:Mraw-Hill.
Conlisk, J. and Watts, H. W. (1969). “A model for optimizing experimental designs for estimating response surfaces.” In Proceedings of the Social Statistics Section, American Statistical Association, 150–156.
Conlisk, J. and Watts, H. W. (1979). “A model for optimizing experimental designs for estimating response surfaces.” J. Econometrics, 11, 27–42.
Conlisk, J. and Kurz, M. (1972). The Assignment Model of the Seattle and Denver Income Maintenance Experiment. Research Memo 15. Menlo Park, CA: Stanford Research Institute, Center for the Study of Welfare Policy.
Cosslett, S. R. (1983). “Distribution-free maximum likelihood estimator of the binary choice model.” Econometrica, 51, 765–782.
Cox, D. R. (1972). “Regression models and life tables (with discussion).” J. Roy. Statist. Soc. Ser. B, 34, 187–220.
Cox, G. M. (1958). “Forward.” In Experimental Designs in Industry, V. Chew (ed.), New York: Wiley.
Cronbach, L. J. (1982). Designing Evaluations of Educational and Social Programs. San Francisco: Jossey-Bass.
Duan, N., Manning, W. G., Jr., Morris, C. N., and Newhouse, J. P. (1983). “A comparison of alternative models of the demand for medical care.” J. Business and Economics, 1, 115–126.
Douchel, W. H. and Harris, J. E. (1983). “Bayes methods for combining results of cancer studies in humans and other species (with discussion).” J. Amer. Statist. Assoc., 78, 293–315.
Ericsson, N. R. and Hendry, D. F. (1985). “Econometric modeling of the market for new housing in the United Kingdom.” In this volume, Chapter 11.
Ferber, R. and Hirsch, W. Z. (1979). “Social experiments in economics.” J. Econometrics, 11, 77–115.
Ferber, R. and Hirsch, W. Z. (1982). Social Experimentation and Economic Policy. Cambridge: Cambridge U.P.
Finney, D. J. (1945). “The fractional replication of factorial arrangements.” Ann. Eugenics, 12, 291–301.
Fisher, R. A. (1935). The Design of Experiments. Edinburgh: Oliver and Boyd.
Fisk, J. D. and Roth, D. M. (1980). Work Disincentives and Income Maintenance Programs: Review of the Empirical Evidence. Congressional Research Service, Libary of Congress, Congress of the United States.
Gallant, A. R. and Koenker, R. W. (1983). Some Welfare Econometrics of Peak-Load Pricing of Electricity: A Continuous Time Approach. Unpublished technical report. Murray Hill, N. J.: Bell Laboratories.
Gilbert, J. P., Light, R. J., and Mosteller, F. (1975). “Assessing social innovations: An empirical base for policy.” In C. A. Bennett and A. A. Lumsdaine (eds.), Evaluation and Experiment: Some Critical Issues in Assessing Social Programs. New York: Academic, 39–193.
Glass, G. V., Maw, B., and Smith, M. L. (1981). Meta-analysis in Social Research. Beverly Hills, CA: Sage.
Haggstrom, G. W. (1976). “The pitfalls of manpower experiments.” In H. W. Sinaiko and L. A. Broedling (eds.), Perspectives on Attitude Assessment: Surveys and Their Alternatives. Champaign, IL: Pendleton.
Haggstrom, G. W., Blaschke, T. J., Chow, W. K., and Lisowski, Wm. (1981). The Multiple Option Recruiting Experiment: Report R-2671-MRAL. Santa Monica, CA: The Rand Corporation.
Hannan, M., Tuma, N., and Groeneveld, L. (1976). The Impact of Income Maintenance on the Making and Breaking of Marital Unions: Interim Report. Research Memorandum 28. Menlo Park, CA: Stanford Research Institute, Center for the Study of Welfare Policy.
Hannan, M., Tuma, N., and Groeneveld, L. (1977). “Income and marital events: Evidence from an income maintenance experiment.” Amer. J. Sociology, 82, 1186–1211.
Hausman, J. A. (1981). “The design and analysis of social and economic experiments.” Proc Internat. Statist. Inst., 43rd Session, 49 (Book 2), 1084–1108.
Heckman, J. J. (1978). “Comment on The labor supply response of wage earners,’ by O. Ashenfelter.” In J. Palmer and J. Pechman (eds.) Welfare in Rural Areas. Washington: Brookings Institution, 138–147.
Heckman, J. and Singer, B. (1984). “Econometric duration analysis.” Econometrics, 24, 63–132.
Hedges, L. V. and Olkin, I. (1981). “Analyses, reanalyses, and meta-analysis: Review of Glass, Maw, and Smith.” Contemporary Education Review, 1, 157–165.
Hendricks, W. and Koenker, R. (1979). “Demand for electricity by time-of-day: An evaluation of experimental results.” Paper presented at Rutgers Conference, The Demand for and Pricing of Public Utility Services, October 1979.
Hermalin, A. and Mason, W. (1980). “A strategy for the comparative analysis of WFS data with illustrative examples.” In The United Nations Program for Comparative Analysis of World Fertility Survey Data. New York: United Nations Fund for Population Activities, 90–168.
Immer, F. R., Hayes, H. K., and Powers, L. (1934). “Statistical determination of barley varietal adaptation.” J. Amer. Soc. Agronomy, 26, 403–419.
Kalbfleisch, J. D. and Prentice, R. L. (1980). The Statistical Analysis of Failure Time Data. New York: Wiley.
Keeler, E. and Rolph, J. (1982). The Demand for Episodes of Medical Treatment. Report R-2829-HHS. Santa Monica, CA: The Rand Corporation.
Keeley, M. C. and Robins, P. K. (1978). The Design of Social Experiments: A Critique of the Conlisk-Watts Assignment Model. Research Memorandum 57. Menlo Park, CA: SRI International.
Kershaw, D. N. and Fair, J. (1976). The New Jersey Income-Maintenance Experiment. Vol. 1, Operations, Survey, and Administration. New York: Academic.
Lancaster, T. and Nickell, S. (1980). “The analysis of reemployment probabilities for the unemployed.” J. Roy. Statist. Soc. Ser. A, 143, 141–165.
Lillard, L. A. and Action, J. P. (1980). Seasonal Electricity Demand and Pricing Analysis with a Variable Response Model. Report R-2425-DWP. Santa Monica, CA: Rand Corporation.
Lillard, L. A. and Aigner, D. J. (1984). “Time-of-day electricity consumption response to temperature and the ownership of air conditioning appliances.” J. Business and Economic Statistics, 2, No. 1, 40–54.
Madansky, A. (1980). “Response surface exploration in social experiments.” Paper presented at Second Annual Public Policy Conference, Boston, MA, October 24–25, 1980.
Manning, W. G., and Acton, J. P. (1980). Residential Electricity Demand Under Time-of-Day Pricing: Exploratory Data Analysis from the Los Angeles Rate Study. Report R-2426-DWP/HP. Santa Monica, CA: Rand Corporation.
Manning, W. G., Mitchell, B. M., and Acton, J. P. (1979). “Design of the Los Angeles peak-load pricing experiment for electricity.” J. Econometrics, 11, 131–194.
Mason, W., Wong, G., and Entwistle, B. (1984). “Contextual analysis through the multilevel linear model.” In S. Leinhardt (ed.), Sociological Methodology 1983–1984. San Francisco: Jossey-Bass, 72–103.
Meier, P. (1975). “Statistics and medical experimentation.” Biometrics, 31, 511–529.
Morris, C. (1979). “A finite selection model for experimental design of the health insurance study.” J. Econometrics, 11, 43–61.
Mosteller, F. (1978). “Errors: Nonsampling errors.” In W. H. Kruskal and J. M. Tanur (eds.), International Encyclopedia of Statistics, 1. New York: Free Press, 208–229.
Mosteller, F. and Tukey, J. W. (1982). “Combination of results of stated precision: I. The optimistic case.” Utilitas Mathematics, 21, 155–178.
Mosteller, F. and Tukey, J. W. (1984). “Combination of results of stated precision: II. A more realistic case.” In P.S.R.S. Rao and J. Sedransk (eds.), W. G. Cochran’s Impact on Statistics. New York: Wiley, 223–252.
Neuberg, L. (1985). Social Control Experimentation: Towards a Rationalist and Historical Critique of Empiricist Methodology. Cambridge, England: Cambridge U.P.
Newhouse, J. P. (1974). “A design for a health insurance experiment.” Inquiry, 11, 5–27.
Newhouse, J. P., Marquis, K. H., Morris, C. N., et al. (1979). “Measurement issues in the second generation of social experiments: The Health Insurance Study.” J. Econometrics, 11, 117–130.
Newhouse, J. P., Manning, W. G., Morris, C. N., et al. (1981). “Some interim results from a controlled trial of cost-sharing in health insurance.” New England J. Medicine 305, 1501–1507.
Pottinger, G. (1983). “Explanation, rationality, and microeconomic theory.” Behavioral Sci., 28, 109–125.
Riecken, H. W., Boruch, R. F., Campbell, D. T., Caplan, N, Glennan, T. K., Pratt, J. W., Rees, A., and Williams, W. (1974). Social Experiments: A Method for Planning and Evaluating Social Programs. New York: Seminar Press.
Rivlin, A. M. (1974). “How can experiments be more useful?” Amer. Econ. Rev., 64, 346–354.
Rosenthal, R. and Rubin, D. B. (1978). “Interpersonal expectancy effects: The first 345 studies.” Behavioral and Brain Sciences, 3, 377–415.
Rossi, P. H., Berk, R. A., and Lenihan, K. J. (1980). Money, Work, and Crime. New York: Academic.
Rossi, P. H. and Lyall, K. C. (1976). Reforming Public Welfare: A Critique of the Negative Income Experiment. New York: Russell Sage Foundation.
Scriven, M. (1969). “Evaluating educational programs.” Urban Rev., 3, 20–22.
Simon, H. A. (1957). Models of Man. New York: Wiley.
Student (1931). “The Lanarkshire milk experiment.” Biometrika, 23, 398–406.
Tharp, R. G. and Gallimore, R. (1979). “The ecology of program research and evaluation: A model of evaluation succession.” In L. Sechrest et al. (eds.), Evaluation Studies Review Annual, 4. Beverly Hills, CA: Sage.
Titterington, D. M. (1980). “Aspects of optimal design in dynamic systems.” Technometrics, 22, 289–299.
University Group Diabetes Program (1970). “A study of the effects of hypoglycemic agents in vascular complications in patients with adult-onset diabetes.” Diabetes (Suppl. 2), 19, 747–830.
U.S. Department of Energy (1978). Analytical Master Plan for the Analysis of Data from the Electric Utility Rate Demonstration Projects. Prepared by Research Triangle Institute, Research Triangle Park, NC.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1985 Springer-Verlag New York Inc.
About this paper
Cite this paper
Fienberg, S.E., Singer, B., Tanur, J.M. (1985). Large-Scale Social Experimentation in the United States. In: Atkinson, A.C., Fienberg, S.E. (eds) A Celebration of Statistics. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-8560-8_12
Download citation
DOI: https://doi.org/10.1007/978-1-4613-8560-8_12
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4613-8562-2
Online ISBN: 978-1-4613-8560-8
eBook Packages: Springer Book Archive