An NBER paper by Sanbonmatsu, Kling, Duncan, and Brooks-Gunn looks at the Moving to Opportunity Program, a program that distributes housing vouchers to low-income households:
Neighborhoods and Academic Achievement: Results from the Moving to Opportunity Experiment, by Lisa Sanbonmatsu, Jeffrey R. Kling, Greg J. Duncan, Jeanne Brooks-Gunn, NBER WP 11909, January 2006: Abstract Families originally living in public housing were assigned housing vouchers by lottery, encouraging moves to neighborhoods with lower poverty rates. Although we had hypothesized that reading and math test scores would be higher among children in families offered vouchers (with larger effects among younger children), the results show no significant effects on test scores for any age group among over 5000 children ages 6 to 20 in 2002 who were assessed four to seven years after randomization. Program impacts on school environments were considerably smaller than impacts on neighborhoods, suggesting that achievement-related benefits from improved neighborhood environments are alone small.
I want to talk about the Moving to Opportunity experiment more including earlier econometric results by Katz, Kling, and Liebman (and many others) on the program's effectiveness. But before talking about the Moving to Opportunity experiment in particular (and its earlier version, the Gautreaux experiment) , I thought it would be useful to talk in general about three large-scale social experiments that have been conducted to evaluate voucher programs in housing and health insurance, and to evaluate the effect a minimum income guarantee for low-income households has on work effort. Fortunately, there is a nice discussion of this topic in Michael Murray's new econometrics text. I will try to follow this up over the next few days with more on the Moving to Opportunity and other programs. From reading comments, etc., I have the impression that many think that economists simply fight ideological battles over social programs and other government policies without regard to actual evidence. Hopefully this discussion will help to uncover some of the evidence behind policy debates. Here's the discussion from Murray (pgs. 649-651):
The Social Experiments: Labor Supply, Housing Allowances, and Health Insurance
In the 1970s and 1980s, the U.S. government sponsored three large social science experiments. First were a series of four negative-income tax experiments that sought to understand how American households would alter their work effort if given a guaranteed minimum income. Second were two housing allowance experiments that probed into how low-income American households would respond to housing vouchers that work much like food stamps. Third was a national health insurance experiment that examined how American households would respond to universal health insurance.
These randomized social experiments all randomly sampled U.S. households in the subpopulations of interest. Selected households were then randomly assigned to treatment and control groups. In each instance, the experiments focused on potential public policies of great scope. Many conservative and liberal social analysts hoped that the patchwork quilt of American social welfare programs could be replaced by a single program, a guaranteed minimum income that would be slowly taxed away as one's income rose (the benefit a household received would be based on a formula: benefit = minimum income - [tax rate]·[earned income]). Others, skeptical about the prospects for a guaranteed income and dissatisfied with public housing projects as a solution to the housing needs of the poor, hoped for a system of housing vouchers that would pay part of the rent of low-income families as they found their homes in the private housing market. Still others saw health insurance as a national need that could be provided best by a universal, subsidized insurance program. Before such complex programs could garner the political support they would need, the most serious doubts about the programs had to be addressed, which was the goal of the experiments.
Would a guaranteed minimum income cause millions of people to go on the dole and abandon work? Would housing vouchers increase the demand for housing, only, to see rents skyrocket and low-income households made no better off by the allowances? Would a free national health plan bankrupt government? Would national health insurance improve health outcomes in the United States? The available economic and medical evidence on all these questions was too fragile to serve as the basis for vastly expensive public policies. The hope was that randomized experiments, freed of the biases that threaten the validity of most social science empirical work, could provide the informational basis for sound policy making.
The experiments were complex undertakings. In the four negative-income tax (NIT) experiments, more than 8,000 households received guaranteed incomes, some receiving guaranteed incomes for up to 20 years. One housing allowance experiment, called the supply experiment, offered housing allowances to all income-eligible households in two small cities, South Bend, Indiana, and Green Bay, Wisconsin. The other housing allowance experiment offered allowances to a random sample of income-eligible households in two larger cities, Phoenix and Pittsburgh. The health insurance experiment sampled some 5,000 people across six cities. Each experiment lasted a minimum of several years.
The NIT experiments established that a guaranteed minimum income would have nonnegligible, but also noncatastrophic, effects on work effort. Male heads of two-parent households would not react much to the minimum income guarantee itself, but would cut their work hours because of the tax rate on earnings that draws down the subsidy as income rises. Men in the program who faced the highest marginal tax rates decreased their hours worked by about 11%. Single female household heads who participated would cut their hours worked by as little as nothing or as much as 15%; these women would react to both the guarantee itself (an income effect that increases the consumption of leisure, which is a normal good) and to the tax rate (a substitution effect that follows on a decrease in the opportunity cost of leisure). Participating women in two-parent households would cut their hours of work most dramatically, by as much as 20% to 30%. (Because married women have become more attached to market work in the past 30 years, this last result might prove different today; married women might behave more like their husbands now.)
Despite these results, the political will for a guaranteed minimum income did not materialize. When Richard Nixon proposed his Family Assistance Plan, a variant on guaranteed minimum income, it fell to a coalition of conservatives who disliked the idea altogether and liberals who found the plan insufficiently generous.
The experimental data from the NIT experiments did, however, provide labor economists with a rich trove of information to explore with economic models and econometric tools. The division of reductions in work effort into income and substitution effects required econometric estimates of labor supply models that went well beyond comparing the mean hours worked of treatment groups and controls. The sophistication of labor economics grew dramatically during the 1970s and 1980s as a direct result of analyzing the experiment.
The housing allowance supply experiment established that a city-wide housing allowance program need not drive up rents. Four factors contributed to keeping prices in check in South Bend and Green Bay. First, eligible households only slowly took advantage of their vouchers, giving the market time to adjust to increased demands. Second, housing vacancies served as a buffer, absorbing new demands for better housing. Third, landlords upgraded units to provide additional housing that met the requirements of the allowance program. Fourth, among those who used their vouchers, the program subsidies spurred housing demand less than anticipated. Much doubt remained, nonetheless, about whether housing market experiences in small places such as South Bend and Green Bay told us enough about large-city markets such as New York or Los Angeles.
The housing allowance demand experiment rigorously established that price and income elasticities of demand for housing are smaller than had been previously believed. Voucher plans that would pay a percentage of households' rents would spur demand less than anticipated. Program requirements that insisted on housing units meeting specific physical standards (such as treads on the building's stairways) did increase the proportion of households meeting those standards, but the correlation among standards met was rather low, so the only way to ensure a standard would be met was to set the standard explicitly. Unfortunately, physical standards discouraged substantial numbers of otherwise eligible households from finding compliant housing. Perhaps the most important finding of the experiment was that housing allowances enabled households to obtain housing as good as that found in public housing at a much lower cost to government.
In the years since the housing allowance experiment, federally subsidized housing programs have increasingly used voucher programs, and less and less used subsidized construction for the poor. Because the voucher programs are cheaper, more households are getting housing subsidies today than in the past, while the quality of the housing that subsidized households receive has not been adversely affected.
The National Health Insurance Experiment (NHIE) established that income has a positive effect on outpatient services and a negative effect on Inpatient services for insured households, leading to a shallow U-shaped relationship between income and total expenditures. Furthermore, the experimental results imply that changes in insurance coverage over time accounted for only a small fraction of the rise in health care costs since World War II. But perhaps the most important finding of the NHIE was that modest cost sharing by patients (with a cap on maximum out-of-pocket expenditures) reduces medical expenditures but does not measurably change health outcomes.
The NHIE did not lead to a national health insurance program. The lesson about cost sharing seems to have changed insurance practices in the United States, however. The fraction of major companies with cost-sharing insurance plans rose from 30% to 63% in the years immediately following the publication of the experimental results. Within a decade of the experiment, the resulting cost savings on the consequent avoided medical expenses was about $7 billion.
All of the large social experiments had to grapple with serious logistical and statistical problems. Do short-duration experimental programs tell us much about actual long-term programs? Do attrition bias and nonreporting bias invalidate the inferences from the experiments? Are the locations chosen for the experiments representative of the nation as a whole - can we generalize from South Bend to New York or from Seattle to Atlanta? Researchers overcame enough of these problems by applying well-known statistical methods and by inventing new methods, so that all the experiments added richly to our understanding of the social policies they addressed.
The great social experiments of the 1970s and 1980s did not lead to the vast new programs that their sponsors had hoped for, but each has had a lasting positive impact on social policy. Despite this success, few additional social experiments have followed in the wake of these first efforts. The cost of social experiments is high-the NHIE cost almost $150 million-and government seldom funds such big ticket items in times of fiscal distress. But if we compare the $150 million cost of the NHIE with the $7 billion cost savings from knowing to use modest cost sharing in health insurance policies, we might conclude that additional social experiments can be worthwhile social Investments.
If that is accurate, if a $150 million investment yielded a $7 billion saving, or anything remotely close, then we should do a lot more of these experiments. It would save money and give our policy debates a much better empirical foundation.