Recently, the CBO Director's Blog posted an entry that seemed to contradict many of the results on the volatility of household risk that come from the work of Jacob Hacker:
Income volatility, Peter Orszag, CBO Director's Blog: Substantial interest has arisen recently regarding how much household income and workers’ earnings bounce around from year to year, prompted in part by the work of Jacob Hacker at Yale University. This topic is important not only to understand potential sources of household anxiety, but also in designing social insurance systems and the tax code.
In previous work released in 2007, CBO examined the volatility of workers’ earnings. That report concluded that earnings were surprisingly volatile, but had been roughly as volatile since the early 1980s — in other words, earnings volatility had not increased.
In preliminary work that I discussed in the latter half of a talk hosted by the Society of Government Economists at the ASSA meetings in New Orleans over the weekend, CBO has now examined the volatility of household income (rather than workers’ earnings volatility, the subject of our study in 2007). The preliminary results suggest that household income is much less volatile than individual worker’s earnings, and that household income volatility has not increased over time — and perhaps even declined slightly. Some other recent studies relying on other data sources have suggested increases in household and family income volatility, but various problems in the surveys used in those studies may be contaminating those results. CBO will soon be releasing our final report on the topic.
CBO’s work on income and earnings volatility is led by Molly Dahl and Jonathan Schwabish of CBO (along with Thomas DeLeire of the University of Wisconsin-Madison) ...
I will let Jacob speak for himself. Here is his response, in full:
I have received numerous queries about a recent post on the Congressional Budget Office’s (CBO) blog that indicates that the CBO has examined the instability (also known as “volatility” or “variability”) of U.S. family incomes from 1984/85 to 2001/2002 and, in preliminary results, has found no consistent increase over that period.
This preliminary finding obviously runs counter to the results that I present in my book, The Great Risk Shift (revised and expanded, 2008), for the 1973-2004 period. In the revised edition of the book, and a forthcoming brief from the Economic Policy Institute, written with Elisabeth Jacobs of the Brookings Institution, I show that the volatility of family incomes has roughly doubled over the 1973-2004 period—a finding that is closely in line with a number of other recent studies.
In my book, I discuss many other indicators of declining economic security among American families: dwindling health coverage and the rising financial threat posed by medical costs, the steady demise of pension plans offering a guaranteed benefit for the remainder of a retired workers’ life; the growing costs of job dislocations and high levels of involuntary job displacement; the rising levels of household debt, growing prevalence of bankruptcy and mortgage foreclosures; and the increasingly threadbare character of public benefits for American workers, especially in light of the increasing number of workers juggling household duties and paid employment due to the movement of women into the workforce. It is the confluence of these trends, not just rising family income volatility, that I believe helps account for rising levels of public anxiety about economic security. In this memo, however, I focus on family economic volatility alone.
To jump to a summary of recent studies, click here.
To get the bottom-line results of my recent analyses (conducted with Elisabeth Jacobs of the Brookings Institution), click here.
So why might the CBO be finding something different?
To fully answer that question, I will need to gain access to the CBO’s underlying analysis, which I have requested. However, what I know of the analysis leads me to believe that there are some straightforward reasons why the CBO results differ from all prior analyses, including my own – and reasons why my analysis is likely to be a more accurate depiction of the nature of family income instability over the last three and a half decades.
For those looking for a quick and dirty answer to the question of why the CBO’s findings differ from mine and nearly all of the other empirical work on family income volatility conducted to date, I offer two (relatively) straightforward reasons:
First, the CBO analysis begins right after two deep recessions, and after fifteen years in which (according to my analyses) family income volatility has already risen dramatically.
Second, the CBO adjusts the data in a way that is likely to lead to spurious results. The basic dataset that it uses is the Survey of Income and Program Participation, or SIPP—a different source than I employ. Other researchers have found an increase in family income volatility between 1984 and 2002 using the SIPP. However, the CBO attempts to match people in the SIPP with Social Security wage records. This might sound like a clever way of verifying people’s reports of their own earnings—except that it results in exclusion of a substantial share of people in the SIPP (more than 40% of those in the SIPP in 2002, up from around 15% in 1984). There is every reason to think that it is the exclusion of such a large share of the sample (and an increasing share over time) that is driving the CBO’s findings.
By contrast, the findings I report in my book (and forthcoming brief with Elisabeth Jacobs) not only go back to the early 1970s. They also are based on as broad a cross-section of Americans as possible given existing data. Moreover, this cross-section, as I show later in this memo, is highly representative of the population I am looking at: non-immigrants aged 25 to 62.
A fuller discussion of these issues and others will appear in my forthcoming brief for the Economic Policy Institute with Elisabeth Jacobs. In addition, upon release of that brief, I plan to make available at www.greatriskshift.com all of the codes and data we used in reaching the conclusions reported in the brief and summarized below.
If you want to go straight to my full discussion of the preliminary CBO results, click here.
At the risk of moving into technical territory, let me begin by emphasizing that prior to my work, no one had attempted to examine the changing volatility of family incomes over the entire last generation, and thus it was unclear what the proper method and data for such analyses would be. Now that we have a number of good studies, how to resolve these technical issues is becoming clearer. Nonetheless, there are technical differences between existing studies that can make comparing their results inherently difficult.
Four technical differences between existing studies stand out and require mention up front:
1. Dataset. Analyzing family income volatility requires multiple observations of the family income of the same individuals over time, something that is available only in a very limited number of datasets, those with so-called “panel data.” The Panel Study of Income Dynamics—a national representative longitudinal study of family incomes that has been conducted since the late 1960s, following the same families and individuals over time—is the most widely used of such datasets, and the one that I use. Two other datasets that have been used to look at family income volatility are the Current Population Survey (CPS) and the Survey of Income and Program Participation (SIPP). The CPS is not a true panel dataset, but it can be used to look at the same people over short periods of time through what is known as “matching”—linking people across waves of the survey. Unfortunately, people who move from their current residence cannot be matched, and matching is difficult and imperfect. The SIPP, by contrast, is a true panel data set. However, it is only available from 1984 on—which brings us to the second difference.
2. Time period. The time period studied is extremely relevant for judging changes over time in family volatility, since the upward trend in volatility has not been smooth. Family income volatility appears to have risen quickly from the early 1970s through the mid-1980s, when it began to decline (though never to anywhere near its 1970s lows). It then turned back upward in the early 1990s, declined in the late 1990s, and then rose again in the early part of this decade—roughly in line with the business cycle. Thus, studies of family income volatility with different starting and ending points may reach very different conclusions, especially when those starting and ending points fall during different parts of the business cycle. In general, studies with longer-time periods should be preferred, as they reduce the chance that the idiosyncrasies of starting and ending years greatly color the results. If there is an upward trend, it should be visible over thirty or forty years. Analyses like mine that use the Panel Study of Income Dynamics—a national representative longitudinal study of family incomes that has been conducted since the late 1960s, following the same families and individuals over time—examine changing family income volatility from 1969 until the mid-2000s (2004 is the last year of available data at present). The other main data sources for looking at family income over time encompass different time periods. In theory, matched CPS data could be used to go back to at least the late 1960s, but the only available study of family income volatility using matched CPS data (which finds a substantial increase) only goes back to 1985 (see http://www.american.edu/cas/econ/workingpapers/2007-17.pdf). The SIPP can only be used from 1984 on (it has also, unlike the PSID and CPS, been discontinued).
3. Method. Very briefly, three basic approaches have emerged among scholars measuring family income (or earnings) volatility. The first, based on the work of the economists Robert Moffitt and Peter Gottschalk, attempts to decompose family income changes into two parts—temporary changes in income and permanent shifts in income. My core analyses utilize a version of this decomposition of family income in order to measure income volatility. Basically, I look at how much family income fluctuates around its four-year average (for more on this method, click here). The second main approach is to look at some measure of the dispersion of changes in family income over time—for example, the standard deviation of annual changes in income. This approach does not distinguish between permanent and short-term income shifts. It also requires figuring out how to measure income changes in a way that is not sensitive to changes in the average level of income. A common approach is to use the log of income. Another is to look at percentage changes in income, although this approach has the problem that equal dollar changes in income will produce very different percentage changes depending on whether they are positive or negative—which means that results may be sensitive to whether and at what level percentage changes are capped. (For instance, a drop in income from $1000 to $100 is a 90 percent decline, while the return from $100 to $1000 represents a 900 percent increase.) A third approach, which I also used in my core analyses for the Great Risk Shift, is to look at the chance of very large percentage drops in family income over a distinct time period—say 50% or greater drops in family income over a two-year period. The advantage of this last approach is not only its intuitive simplicity, but also the fact that it alone captures downward income volatility, which is likely to be most salient to individuals. Both the transitory variance and standard deviation approaches discussed above capture movement both above and below average income. For the most part, these three approaches do not produce fundamentally different results: All methods applied to the PSID show a substantial increase in family income volatility. However, they do have different underlying units of measurement, which leads to the final point.
4. Unit of Measurement. In general, discussion has centered on what the percentage change in family income volatility has been since the early 1970s. In the brief, as noted, my coauthor and I find a roughly 100% increase between 1973 and 2004 (1973 is the first year we can calculate 4-year average income, since the survey data we use starts in 1969, and 2004 is the last year for which the data are available). The exact increase we find is 95%. But the percentage change in any measure is obviously dependent on what the units of measurement are. The volatility measure we use is expressed in terms of the variance of log income. By contrast, the standard deviation of income changes is expressed in terms of the standard deviation of log or percentage changes in income. The standard deviation is the square root of variance. Thus, expressed in terms of standard deviations, the 95 percent rise in family income volatility that we find from 1973 to 2004 would be around 40 percent—which is very close to what we and others find using the standard deviation of change metric to examine the PSID.
To make these points clearer, I have produced a chart that puts all the existing studies of the PSID, Matched-CPS data, and SIPP onto the same dimension. It can be accessed here. The lines show the linear trend in family income volatility for each study, with the units of measurement adjusted to match the approach that I take in The Great Risk Shift. For those studies that start after 1973, I simply set their start value so they branch off my volatility trendline in their starting year. As can be seen, the percentage change in the various volatility measures—once one adjusts for their different unit of measurement—is quite similar across studies. All show a substantial increase in family income volatility. This is true even of the two studies that start in the mid-1980s.
The bottom line, then, is that once one takes into account differences in measurement units and time period, we are talking about a relatively constrained range of estimates of the percentage rise in family income volatility over time—ones that appear relatively similar across datasets and methods, though of course not identical.
Indeed, it is possible to sum up the findings of the existing studies, including my analyses in The Great Risk Shift and the work with Elisabeth Jacobs reported in our forthcoming EPI brief, in a series of fairly simple observations:
- The instability of family incomes has risen substantially over the last three decades. Much of the rise in income volatility occurred prior to 1985, and volatility dropped substantially in the late 1990s. But it has risen in recent years to exceed its 1980s peak.
- The proportion of working-age individuals experiencing a large drop in their family income (50 percent or greater) has climbed more steadily, rising (in our analyses) from around 3 to 4 percent in the early 1970s to roughly 7 to 9 percent in the early 2000s. The probability of large income drops varies predictably with the business cycle. Yet it has also trended strongly upward over time. For instance, the 2001 recession, which was fairly mild in macroeconomic terms, was associated with a higher chance of large income drops than the recession of the early 1980s.
- There is an important distinction between family income (total earnings, asset income, and transfer income for all members of a family) and individual earnings. While the instability of individual workers’ earnings rose sharply between the 1970s and 1980s, it has been more or less stable since then, trending up and down with the business cycle through the 1980s and 1990s. However, individual workers’ earnings instability appears to have risen in the early 2000s. This basic trend—a rise in earnings variability in the 1970s, little clear trend from the early 1980s to the late 1990s, and an upswing in the early 2000s—has been confirmed by numerous analyses, including a recent study by the Congressional Budget Office. Moreover, this same basic pattern can be seen in data from both the survey-based Panel Study of Income Dynamics (which we use in this brief) and the administratively-collected Continuous Work History Sample of the Social Security Administration (used by the CBO).
- Contrary to assertions in the popular press, women’s increased workforce participation has not been a major factor contributing to the rise in family income volatility. Female earnings have, if anything, become somewhat more stable since the 1980s. Male workers have experienced a larger and more sustained rise in earnings instability. And because men’s earnings account for a larger percentage of total household income than do women’s earnings, on average, rising instability in male earnings helps account for the increase in family income volatility. In short, the stabilizing influence on family income of the decrease in female earnings instability is overwhelmed by the rise in men’s earnings instability.
- In addition to the increase in male earnings variability, other likely causes of rising family income volatility include the growing variability of cash transfers and the limited cushioning effect of having a second earner in the household. Although the evidence is limited, early analyses suggest that a second family earner is less of a benefit in terms of income protection today than it was prior to the 1990s.
- Finally, while less educated and poorer Americans have less stable family incomes than their better-educated and wealthier peers, the increase in family income volatility appears to have affected all major demographic and economic groups—and, indeed, looks most sharp among college-educated Americans in recent years.
Back to the CBO. The emerging consensus on these findings—especially the basic finding of rising family income volatility over the last generation—makes it somewhat surprising that the CBO is touting preliminary results using the SIPP to suggest that family income volatility did not rise between 1985 and 2002. I have yet to receive enough of the details about the CBO’s study to form a complete impression, and I have requested more information. But what I know leads me to have very grave doubts about what they are up to.
Given my earlier discussion of the importance of comparing results across the same time period (to go back to it, click here), it could be that the period chosen by the CBO is the only reason for the discrepancy. The preliminary CBO results are available for 1984/1985, 1993/1994, 1997/1998, and 2001/2002. Were it not for the fact that the SIPP began in October 1983, 1984 would be a very odd year to use as a baseline, given that it has come after not one but two national recessions—one of which was the worst economic downturn since the Great Depression. In any case, the CBO results show that volatility declined between 1985 and 1994, stayed low in 1998, and rose back roughly to 1985 levels in 2002. The chance of large (50%) drops in income rises more sharply between 1998 and 2002, slightly exceeding the 1985 level.
This is not so different from the trend I find using the PSID. As the figure here summarizing my findings shows, 1985 turns out to be pretty close to a local peak in family income volatility, with much of the rise in family income volatility between the early 1970s and early 2000s occurring by the mid-1980s. Nonetheless, I and others using the PSID, as well as others using the SIPP, do find a continuing increase between the 1980s and 2000s—so there must be something more going on.
The most plausible interpretation (again, without the underlying analyses in hand) has to do with the procedure that the CBO apparently used. According to those in attendance when the CBO’s preliminary results were presented at the American Economic Association’s annual meeting in January, the CBO actually found an increase in family income volatility using the SIPP alone, just as other scholars have.
But they then tried to match SIPP records with Social Security wage data (the CWHS) in order to use actual Social Security reports of earnings rather than what people said they earned. And here things seem to have gone awry. The justification for incorporating actual wage data is that the self-reports of earnings in the SIPP may be unreliable. In particular, the CBO seems wary of so-called imputed data in the SIPP—when the SIPP replaces missing data with values that ensure that the missing data (which is certainly not random) does not skew the distribution of income and other variables in the SIPP. Interestingly, according to those who saw the CBO presentation, the imputed data does not show higher volatility than the non-imputed data—which runs strongly against the grain of the CBO’s suggestion that imputation (in the SIPP or the PSID) is behind the finding of rising volatility.
Based on reports from those attending the CBO presentation earlier this month, I suspect the most substantial problem, however, is that the CBO ended up throwing out a good deal of the SIPP sample in order to bring in the Social Security data for wages in place of the SIPP’s own wage data. This problem may be especially salient in 2002, when (again, according to those who saw the presentation) the match rate was 57 percent (down from 85 percent in 1985). It is difficult to believe that nearly half of the SIPP sample was not in employment covered by Social Security, and so it is reasonable to suspect that the CBO research excludes a great many valid households from their analysis. I suspect that the CBO analysis may as a result exclude many of the most volatile cases. This biased exclusion may explain why the CBO finds that the rising family income volatility that they and other scholars have found in the unadjusted SIPP simply “disappears” when they fold in the administrative data and throw out the unmatched observations.
One reason to suspect that the matching process is biasing the CBO results is that other work using the SIPP to investigate economic volatility suggests that the unadjusted SIPP earnings data give similar results to other studies, including the CBO’s earlier work. According to the only existing study of family income volatility in the SIPP (Gosselin and Zimmerman, listed in the linked list of studies here), the SIPP shows a very modest rise in the volatility of earnings between 1984 and 2002—around 10% versus an almost 50% rise in family income volatility. This result is consistent with the findings of the CBO’s earlier study as well as a nice recent paper by Donggyun Shin and Gary Solon (see the list of recent studies), and it matches the results of my own analyses reported above—after rising sharply between the early 1970s and early 1980s, earnings instability did not rise consistently over the 1980s and 1990s. Thus, whatever is driving the rise in family income volatility in the SIPP, it is not rising earnings instability. This makes it odd that simply substituting in earnings records from Social Security would cause the rise in family income volatility in the SIPP to disappear. The more likely reason why substituting the earnings data makes a difference is that it results in the exclusion of the most volatile households.
Finally, the CBO apparently throw out all households in the SIPP with any so-called imputed data, which further reduces the share of the SIPP sample the CBO is using. Moreover, the need for imputation in the SIPP has risen, so once again the share of people excluded from analysis increases over time. Without going into the technical details of this procedure, it has large potential effects in a dataset like the SIPP, which has many questions and returns to ask people in the same households questions repeatedly (indeed, every four months). While there are clearly concerns with imputed data, throwing out all cases with any imputed data raises just as serious concerns, since the very reason for imputation in the first place is to reduce the bias caused by nonrandom missing data. If missing data are indicative of some pattern, then deletion can introduce substantial bias.
Two other concerns: First, the CBO cuts a substantial chunk of observations out of both the bottom and the top of the income distribution (2 percent of the distribution on both ends) when it analyzes the SIPP, even after it cuts out so many people in the matching process. A key justification for trimming outliers when using survey-based data like the PSID is that very low or high values caused by measurement error may skew the results. There seems to be less justification for doing so when using administrative data, which, in theory at least, is more reliable. (I say “in theory”, because some scholars have raised questions about the reliability of even the Social Security wage records.) Dropping people who have very high or low incomes in either of the two years used for the analysis of year-to-year income dynamics will obviously reduce estimates of volatility substantially.
Second, it seems difficult to reconcile the findings of the CBO’s earlier findings on earnings instability and these preliminary results on family income volatility. In its earlier findings, the CBO found 10-15% of people experiencing year-to-year labor income drops of 50% or greater. In its current analysis, it finds family income drops on the order of 4%—which appear quite low, suggesting a seemingly unrealistic amount of cushioning through government transfers and the work effort of other family members. This is especially true because the CBO does not include the effect of taxes, which means the cushioning effect of the Earned Income Tax Credit is not taken into account.
However, let me emphasize that I have yet to receive enough information about the CBO report to fully assess its validity, in terms of both methods and results. The emerging consensus in the literature—especially among scholars who use the PSID, which provides the longest picture of changing family income dynamics—gives us good reason to view the CBO’s results with skepticism. But I welcome new contributions to the discussion and hope they and others will continue to refine our understanding of this vital trend.
A final note for true data junkies: On the blog post the CBO suggests that the PSID may have errors due to the fact that it is based on surveys. This is no doubt the case (and it also applies to all the SIPP variables that the CBO uses in conjunction with the Social Security earnings reports). The issue is whether those errors have worsened over time. There is little reason to think that PSID data, which are considered among the best available on income, wealth, active savings and average annual hourly earnings, have gotten consistently more error-prone over time in a way that would call into question the finding of rising family income volatility. My own comparisons suggest that the family income reports in the PSID closely track both the Census Bureau’s March Current Population Survey (CPS) and the Congressional Budget Office (CBO)’s income dataset, which is based in part on administrative tax data and hence considered quite reliable.
The figure here compares the variances of the PSID’s family income data for working-age adults and the CPS’s, with adjustments to each to ensure consistency across the two datasets. As can be seen the two track each other extremely closely, which should not be so surprising, as the PSID’s weights are based in part on the CPS.
As a number of scholars have discovered, the one notable point of incongruence between the PSID and other datasets comes during the 1990s, with the differences most pronounced at the bottom of the income ladder. Although the 1991 recession does result in a decrease in incomes at the bottom of the distribution, the sharpness and persistence of the drop raises questions about the quality of the data during this period. This is particularly true because the PSID changed its survey administration procedures during this time. I am working with the PSID to address these problems. Moreover, in the analyses reported in the revised edition book and the EPI brief, my treatment of the data—namely, my dropping of very low income reports—reduces the impact of the small number of questionable observations. Most important, as the figure just cited suggests, the problem appears to be largely limited to the early to mid-1990s. By the end of the decade, the PSID once again looks highly representative.
In short, with the exception of the spike in low-income observations in the mid-1990s that I attempt to compensate for in the analyses (and which do not provide grounds for questioning the long-term trend), the PSID appears highly representative, lending credence to the finding of rising family income volatility—especially since the trends in family income volatility that we find using the PSID compare closely to recent estimates made by other scholars.