More econometrics: Special Session: Model Selection and Inference (Technical)
Sunday, June 05, 2016
Wednesday, April 20, 2016
101 Boosterism: I see that @drvox is writing a big piece on carbon pricing – and agonizing over length and time. I don’t want to step on his forthcoming message, but what he’s said so far helped crystallize something I’ve meant to write about for a while, a phenomenon I’ll call “101 boosterism.”
The name is a takeoff on Noah Smith’s clever writing about “101ism”, in which economics writers present Econ 101 stuff about supply, demand, and how great markets are as gospel, ignoring the many ways in which economists have learned to qualify those conclusions in the face of market imperfections. His point is that while Econ 101 can be a very useful guide, it is sometimes (often) misleading when applied to the real world.
My point is somewhat different: even when Econ 101 is right, that doesn’t always mean that it’s important – certainly not that it’s the most important thing about a situation. In particular, economists may delight in talking about issues where 101 refutes naïve intuition, but that doesn’t at all mean that these are the crucial policy issues we face. ...
He goes on to talk about this in the context of international trade and carbon pricing (too hard to excerpt without leaving out important parts of his discussion).
Thursday, March 31, 2016
This is from the B of E's Bank Underground:
Modelling banking sector shocks and unconventional policy: new wine in old bottles?, by James Cloyne, Ryland Thomas, and Alex Tuckett: The financial crisis has thrown up a huge number of empirical challenges for academic and professional economists. The search is on for a framework with a rich enough variety of financial and real variables to examine both the financial shocks that caused the Great Recession and the unconventional policies, such as Quantitative Easing (QE), that were designed to combat it. In a new paper we show how using an older structural econometric modelling approach can be used to provide insights into these questions in ways other models currently cannot. So what are the advantages of going back to an older tradition of modelling? An ongoing issue for central bank economists is that they typically want to look at a wide range of financial sector variables and at a more granular, sector-based level of aggregation than typically found in macroeconomic models with credit and asset market frictions. For example, we often want to distinguish between the credit provided to firms separately from that provided to households or between secured lending and unsecured lending. We may also want to compare and contrast a number of policy instruments that work through different channels such as central bank asset purchases (QE) and macroprudential tools such as countercyclical capital requirements.
It is a tough challenge to incorporate all of these effects in the theoretical and empirical models that are typically used by macroeconomists, such as structural vector autoregression (SVAR) models and micro-founded general equilibrium (DSGE) models. For these reasons turning back to the older tradition of building structural econometric models (SEMs) – built from blocks of simultaneously estimated equations with structural identifying restrictions – can be useful. This approach can be thought of as a blend of the more theory-free VAR methods and a more structural model-based approach. The main advantage of the structural econometric frameworks are that they produce quantitative results at a sector level, which can still be aggregated up to produce a general equilibrium response. They also allow models to be built up in a modular way that allows replacing and improving sets of equations for particular blocks of the model without necessarily undermining the logic of the model as a whole. This older school approach to modelling has begun to appear in a variety of modern vintages. ...
Friday, February 12, 2016
Making Inferences about Tropics, Germs, and Crops: One of the long-running arguments in growth is "geography versus institutions". A lot of ink and a tiny amount of computing power was thrown at this question. The early stages of this involved a lot of cross-country regressions that attempted to figure out empirically whether measures of institutions or geography had explanatory power for GDP per capita. (By the way, I'm talking here just about the economic growth literature's take on this. The general question goes back centuries.)
A tweet by Garett Jones recently reminded me of one of the entries in this literature, Bill Easterly and Ross Levine's "Tropics, Germs, and Crops" paper. EL want to assess the role of geography in explaining cross-country incomes per capita. Their main questions are summed up nicely in the abstract.
Does economic development depend on geographic endowments like temperate instead of tropical location, the ecological conditions shaping diseases, or an environment good for grains or certain cash crops? Or do these endowments of tropics, germs, and crops affect economic development only through institutions or policies?
Their conclusion is that geography has no effect, other than through its relationship to institutions. This conclusion, though, doesn't follow from the empirical tests they run. Let's run through the empirics on these questions, and see how to answer them.
I'm going to do this with precisely the dataset that EL use. I don't recall when or where I picked up the data, but I believe it was from Easterly's old website, where he had a nice set of links to datasets. Regardless, it's the right dataset because I can perfectly replicate their results. ...
Monday, February 08, 2016
Cecchetti & Schoenholtz:
The Scandal is What's Legal: If you haven’t seen The Big Short, you should. The acting is superb and the story enlightening: a few brilliant outcasts each discover just how big the holes are that eventually bury the U.S. financial system in the crisis of 2007-2009. If you’re like most people we know, you’ll walk away delighted by the movie and disturbed by the reality it captures. ...
But we’re not film critics. The movie—along with some misleading criticism—prompts us to clarify what we view as the prime causes of the financal crisis. The financial corruption depicted in the movie is deeply troubling (we’ve written about fraud and conflicts of interest in finance here and here). But what made the U.S. financial system so fragile a decade ago, and what made the crisis so deep, were practices that were completely legal. The scandal is that we still haven’t addressed these properly.
We can’t “cover” the causes of the crisis in a blog post, but we can briefly explain our top three candidates: (1) insufficient capital and liquidity reflecting poor risk management and incentives; (2) the ability of complex, highly interconnected intermediaries to take on and conceal enormous amounts of risk; and (3) an absurdly byzantine regulatory structure that made it virtually impossible for anyone, however inclined, to understand (let alone manage) the system’s fragilities. ...[long explanationss of each]...
To say that this is a scandal that makes the system less safe is to dramatically understate the case.
Now, we could go on. There are plenty of other problems that policymakers have ignored and are allowing to fester (how about the government-sponsored enterprises?). But we focused on our top three: the need for financial intermediaries to have more capital and liquid assets; the need to improve the ability of both financial market participants and authorities to assess and control risk concentrations through a combination of central clearing and better information collection; and the need to simplify the structure and organization of the U.S. regulatory system itself.
Only if people learn how far the financial system remains from these ideals, only if they understand that the scandal is almost always what is legal, will there be much chance of making the next crisis less severe. ...
Monday, October 26, 2015
Why can't we make good predictions? Maybe we've been doing it wrong:
Scientists devise new method to solve significant variables conundrum, Columbia University: Scientists at Columbia University, the University of California, San Diego (UCSD) and Harvard University have presented an alternative method to address the challenge of using significant variables to make useful predictions in areas such as complex disease.
Shaw-Hwa Lo and Tian Zheng of Columbia, Adeline Lo of UCSD and Herman Chernoff of Harvard present findings in ... Proceedings of the National Academy of Sciences ... that demonstrates that statistically significant variables are not necessarily predictive. In addition, very predictive variables do not necessarily have to appear significant and thereby evade a researcher using statistical significance as a criterion to evaluate variables for prediction.
Statistical significance is a traditional, long-standing measure in any researcher's toolbox but thus far, scientists have been puzzled by the inability to use results of statistically significant variants in complex diseases to make predictions useful for personalized medicine. Why aren't significant variables leading to good prediction of outcomes? This conundrum affects both simple and complex data in a broad range of science and social science fields.
In their findings, the authors demonstrate that what makes variables good for prediction versus significance depends on different properties of the underlying distributions. They suggest that progress in prediction requires efforts toward a new research agenda of searching for a novel criterion to retrieve highly predictive variables rather than highly significant variables.
They also present an alternative approach, the Partition Retention method, which displays strong power in prediction. ...
"What we're saying here is that using the previously very well-known methods might not be appropriate when we care about predicting important outcomes," says Professor Lo. "Our alternative approach seems to do very well in prediction, and is relevant for many scientific fields."
Thursday, October 01, 2015
For a long time, I have been making the argument that part of the reason for the inequality problem is distortions in the distribution of income driven by market imperfections such as monopoly power that allows prices to exceed marginal costs. What I didn't realize is that this can also affect measurements of productivity growth:
The relationship between U.S. productivity growth and the decline in the labor share of national income, by Nick Bunker: One of the ongoing debates about the state of the U.S. economy is the extent to which the profits from productivity gains are increasingly going to the owners of capital instead of wage earners. These researchers are debating the extent to which the labor share of income, once considered a constant by economists, is on the decline.
But what if the decline of national income going to labor actually affects the measured rate of U.S. productivity growth? In a blog post published last week, University of Houston economist Dietz Vollrath sketches out a model showing just that scenario. ...
Vollrath argues that businesses with more market power are able to charge higher markups on their goods and services, meaning their pricing is higher than the cost of producing an additional goods or services compared to pricing in a perfectly competitive market. So in this situation where markups are high, goods and services are being produced less efficiently, with the increased profits going to the owners of capital.
Vollrath argues that this is how measured productivity growth is affected by the decline of the labor share of income. Market power is important for thinking about measured productivity growth because, as Vollrath says, it “dictates how efficiently we use our inputs.” ... Impeding the most efficient use of capital and labor via marked-up prices will reduce measured productivity. ... Perhaps this could explain some of the reason why measured productivity growth looks so meager in the seeming age of innovation...
But Vollrath’s story isn’t a complete explanation of the fall in measured productivity, as he acknowledges...
But Vollrath’s market power explanation for falling productivity growth, alongside the falling share of national income going to wage earners, is supported by some evidence. Work by Massachusetts Institute of Technology graduate student Matt Rognlie, for example, found evidence of higher markups.
Whether and how the decline of the labor share of income affects productivity growth is obviously a topic far too large for a couple of blog posts. But Vollrath’s model is especially interesting for connecting two important trends in recent years: the slowdown in productivity growth and the declining labor share. It’s worth, at the very least, a bit more investigation.
Monday, September 28, 2015
No sense hiding from evidence that works against my support of immigration. This is from George Borjas (if you are unfamiliar with the Mariel boatlift, see here):
The Wage Impact of the Marielitos: A Reappraisal, by George J. Borjas, NBER Working Paper No. 21588 [open link]: This paper brings a new perspective to the analysis of the Mariel supply shock, revisiting the question and the data armed with the accumulated insights from the vast literature on the economic impact of immigration. A crucial lesson from this literature is that any credible attempt to measure the wage impact of immigration must carefully match the skills of the immigrants with those of the pre-existing workforce. The Marielitos were disproportionately low-skill; at least 60 percent were high school dropouts. A reappraisal of the Mariel evidence, specifically examining the evolution of wages in the low-skill group most likely to be affected, quickly overturns the finding that Mariel did not affect Miami’s wage structure. The absolute wage of high school dropouts in Miami dropped dramatically, as did the wage of high school dropouts relative to that of either high school graduates or college graduates. The drop in the relative wage of the least educated Miamians was substantial (10 to 30 percent), implying an elasticity of wages with respect to the number of workers between -0.5 and -1.5. In fact, comparing the magnitude of the steep post-Mariel drop in the low-skill wage in Miami with that observed in all other metropolitan areas over an equivalent time span between 1977 and 2001 reveals that the change in the Miami wage structure was a very unusual event. The analysis also documents the sensitivity of the estimated wage impact to the choice of a placebo. The measured impact is much smaller when the placebo consists of cities where pre-Mariel employment growth was weak relative to Miami.
Saturday, September 12, 2015
Wednesday, June 10, 2015
Can economic opportunity be separated from economic outcomes?:
Inequality of opportunity as a policy construct: ...Concluding remarks
Any attempt to separate circumstances from effort – to identify that portion of the inequality of outcomes which is a legitimate target for redistribution – is fraught with empirical and conceptual difficulties. Fine-grained distinctions between inequality of opportunity and inequality of outcomes do not hold water in practice, and we are likely to greatly underestimate inequality of opportunity and hence the need for intervention.
Further, what if one person’s effort becomes another person’s circumstance, as when income generated through parents’ effort provides a better start in life for some children? Or when freely made choices by one group of upper-income house buyers push up prices for others who may have lower incomes? Is it legitimate or is it not legitimate to intervene in this case?
These arguments support the case for generalised social protection in dimensions such as income, health and education, irrespective of whether the outcomes can be specifically attributed to circumstance or to effort.
The important questions then relate to what the best available policy instruments are for delivering this social protection, what effects they have on incentives, and how best they can be deployed. To be sure, we may make some Type I and Type II errors in doing so; we may penalize effort when we should not, and we may not fully compensate for circumstances when we should. But this is preferable to being frozen into perpetually underestimating the need for intervention by a focus on that will o’ the wisp, inequality of opportunity.
Saturday, June 06, 2015
Seems like much the same can be said about modern macroeconomics (except perhaps the "given the field its credibility" part):
A Crisis at the Edge of Physics, by Adam Frank and Marcelo Gleiser, NY Times: Do physicists need empirical evidence to confirm their theories?
You may think that the answer is an obvious yes, experimental confirmation being the very heart of science. But a growing controversy at the frontiers of physics and cosmology suggests that the situation is not so simple.
A few months ago in the journal Nature, two leading researchers, George Ellis and Joseph Silk, published a controversial piece called “Scientific Method: Defend the Integrity of Physics.” They criticized a newfound willingness among some scientists to explicitly set aside the need for experimental confirmation of today’s most ambitious cosmic theories — so long as those theories are “sufficiently elegant and explanatory.” Despite working at the cutting edge of knowledge, such scientists are, for Professors Ellis and Silk, “breaking with centuries of philosophical tradition of defining scientific knowledge as empirical.”
Whether or not you agree with them, the professors have identified a mounting concern in fundamental physics: Today, our most ambitious science can seem at odds with the empirical methodology that has historically given the field its credibility. ...
Friday, June 05, 2015
Monday, June 01, 2015
Maurizio Bovi summarizes a new published paper he wrote with Roy Cerqueti. The paper examines lay agents' forecasts amid great recessions (with a special focus on Greece). The paper is Bovi, M. and R. Cerqueti (2015) "Forecasting macroeconomic fundamentals in economic crises" Annals of Operations Research DOI 10.1007/s10479-015-1879-4:
Forecasting in Economic Crises: Expectations are a key factor in economics and heterogeneous forecasts are a fact of life. Just to mention, there are quite significant and well-known incentives to become a sport champion or to win a Nobel Prize, yet very few persons succeed in the endeavor. The brutal truth is that the majority lags behind or gives up - heterogeneity is the rule, not the exception. By the same token lay forecasters may learn, but it is unrealistic to think that all of them—even in the long run—will achieve Muth-optimal and, hence, homogeneous forecasts. The situation is made even more complex, and more interesting to study, when the fundamental to predict, the real GDP growth rate, is well below zero and highly volatile.
In recent work (Bovi and Cerqueti, 2015) we address the topic of heterogeneous forecasting performances amid deep recessions. Lay agents are assumed to have different predictive ability in that they have equal loss functions, but different asymmetry parameters that are used as control to minimize their forecasting errors. Simulating poor performing economies populated by three groups of forecasters, we have obtained the following results.
The less sophisticated forecasters in our setting – the “medians” (using passive rule-of-thumb) - never perform as the best predictors – the “muthians” – whereas “second best” (SB) agents (acting as attentive econometricians) do that only occasionally. This regardless the size of the crisis. Thus, as in the real world, in our artificial economy heterogeneity is a structural trait. More intriguingly, simulations also show that the medians’ behavior tend to be relatively smoother than that of SB agents, and that the difference between them widens in the case very serious crises. In particular, great recessions make SB agents’ predictions relatively more biased. An explanation is that dramatic crises extend the available information set (e.g., due to greater mass media coverage), and this leads SB agents, who are more prompt to revise their forecasts than medians.
Our results are somewhat in line with Simon’s famous statement about the fact that more information does not necessarily mean better forecasting performances. Furthermore, our outcomes shed some light on what has been happening in the freak macroeconomic expectations in Greece these years. The current crisis, in fact, may be thought of as a sort of natural experiment to understand how lay decision makers react to very dramatic years. In particular, due to its terrible recent downturn, Greece is one of the most suitable cases, raising the following question: How do Greeks perceive their own personal financial situation with respect to that of their country? Clearly, the representative citizen cannot by definition systematically drift apart from that of the country where she lives, given that the nation-wide economic situation is the (weighted) sum of the individual ones in the country. Yet, it may be hard to remain objective in the course of very deep and prolonged economic crises. The evidence depicted in the following graph looks rather suggestive of the effects of deep recessions on the rationality of people’s expectations, something that conform with our findings.
Wednesday, May 20, 2015
From the blog Three-Toed Sloth by Cosma Shalizi (this also appeared in yesterday's links):
Any P-Value Distinguishable from Zero is Insufficiently Informative: After ten years of teaching statistics, I feel pretty confident in saying that one of the hardest points to get through to undergrads is what "statistically significant" actually means. (The word doesn't help; "statistically detectable" or "statistically discernible" might've been better.) They have a persistent tendency to think that parameters which are significantly different from 0 matter, that ones which are insignificantly different from 0 don't matter, and that the smaller the p-value, the more important the parameter. Similarly, if one parameter is "significantly" larger than another, then they'll say the difference between them matters, but if not, not. If this was just about undergrads, I'd grumble over a beer with my colleagues and otherwise suck it up, but reading and refereeing for non-statistics journals shows me that many scientists in many fields are subject to exactly the same confusions as The Kids, and talking with friends in industry makes it plain that the same thing happens outside academia, even to "data scientists". ... To be fair, one meets some statisticians who succumb to these confusions.
One reason for this, I think, is that we fail to teach well how, with enough data, any non-zero parameter or difference becomes statistically significant at arbitrarily small levels. The proverbial expression of this, due I believe to Andy Gelman, is that "the p-value is a measure of sample size". More exactly, a p-value generally runs together the size of the parameter, how well we can estimate the parameter, and the sample size. The p-value reflects how much information the data has about the parameter, and we can think of "information" as the product of sample size and precision (in the sense of inverse variance) of estimation, say n/σ2. In some cases, this heuristic is actually exactly right, and what I just called "information" really is the Fisher information.
Rather than working on grant proposals Egged on by a friend As a public service, I've written up some notes on this... [The mathematics comes next.]
Saturday, May 16, 2015
Koen Jochmans of Sciences Po speaks to Mark Thoma about his research and winning the Sargan Prize for outstanding research in the Econometric Journal. http://www.res.org.uk/details/econome...
The interview was recorded at the Royal Economic Society annual conference at The University of Manchester in April 2015 and produced by Econ Films.
[This is the first of three interviews I did -- will save the best for last.]
Friday, April 24, 2015
John Cochrane weighs in on the discussion of unit roots:
Unit roots, redux: Arnold Kling's askblog and Roger Farmer have a little exchange on GDP and unit roots. My two cents here.
I did a lot of work on this topic a long time ago, in How Big is the Random Walk in GNP? (the first one) Permanent and Transitory Components of GNP and Stock Prices” (The last, and I think best one) "Multivariate estimates" with Argia Sbordone, and "A critique of the application of unit root tests", particularly appropriate to Roger's battery of tests.
The conclusions, which I still think hold up today:
Log GDP has both random walk and stationary components. Consumption is a pretty good indicator of the random walk component. This is also what the standard stochastic growth model predicts: a random walk technology shock induces a random walk component in output but there are transitory dynamics around that value.
A linear trend in GDP is only visible ex-post, like a "bull" or "bear" market. It's not "wrong" to detrend GDP, but it is wrong to forecast that GDP will return to the linear trend or to take too seriously correlations of linearly detrended series, as Arnold mentions. Treating macro series as cointegrated with one common trend is a better idea.
Log stock prices have random walk and stationary components. Dividends are a pretty good indicator of the random walk component. (Most recently, here.) ...
Both Arnold and Roger claim that unemployment has a unit root. Guys, you must be kidding. ...
He goes on to explain.
Tuesday, March 17, 2015
Some of you may be interested in this:
Causal Inference in Social Science An elementary introduction, by Hal R. Varian, Google, Inc, Jan 2015, Revised: March 7, 2015: Abstract This is a short and very elementary introduction to causal inference in social science applications targeted to machine learners. I illustrate the techniques described with examples chosen from the economics and marketing literature.
Thursday, March 05, 2015
Economists' Biggest Failure: One of the biggest things that economists get grief about is their failure to predict big events like recessions. ...
Pointing this out usually leads to the eternal (and eternally fun) debate over whether economics is a real science. The profession's detractors say that if you don’t make successful predictions, you aren’t a science. Economists will respond that seismologists can’t forecast earthquakes, and meteorologists can’t forecast hurricanes, and who cares what’s really a “science” anyway.
The debate, however, misses the point. Forecasts aren’t the only kind of predictions a science can make. In fact, they’re not even the most important kind.
Take physics for example. Sometimes physicists do make forecasts -- for example, eclipses. But those are the exception. Usually, when you make a new physics theory, you use it to predict some new phenomenon... For example, quantum mechanics has gained a lot of support from predicting the strange new things like quantum tunneling or quantum teleportation.
Other times, a theory will predict things we have seen before, but will describe them in terms of other things that we thought were totally separate, unrelated phenomena. This is called unification, and it’s a key part of what philosophers think science does. For example, the theory of electromagnetism says that light, electric current, magnetism, radio waves are all really the same phenomenon. Pretty neat! ...
So that’s physics. What about economics? Actually, econ has a number of these successes too. When Dan McFadden used his Random Utility Model to predict how many people would ride San Francisco's Bay Area Rapid Transit system,... he got it right. And he got many other things right with the same theory -- it wasn’t developed to explain only train ridership.
Unfortunately, though, this kind of success isn't very highly regarded in the economics world... Maybe now, with the ascendance of empirical economics and a decline in theory, we’ll see a focus on producing fewer but better theories, more unification, and more attempts to make novel predictions. Someday, maybe macroeconomists will even be able to make forecasts! But let’s not get our hopes up.
I've addressed this question many times, e.g. in 2009, and to me the distinction is between forecasting the future, and understanding why certain phenomena occur (re-reading, it's a bit repetitive):
Are Macroeconomic Models Useful?: There has been no shortage of effort devoted to predicting earthquakes, yet we still can't see them coming far enough in advance to move people to safety. When a big earthquake hits, it is a surprise. We may be able to look at the data after the fact and see that certain stresses were building, so it looks like we should have known an earthquake was going to occur at any moment, but these sorts of retrospective analyses have not allowed us to predict the next one. The exact timing and location is always a surprise.
Does that mean that science has failed? Should we criticize the models as useless?
No. There are two uses of models. One is to understand how the world works, another is to make predictions about the future. We may never be able to predict earthquakes far enough in advance and with enough specificity to allow us time to move to safety before they occur, but that doesn't prevent us from understanding the science underlying earthquakes. Perhaps as our understanding increases prediction will be possible, and for that reason scientists shouldn't give up trying to improve their models, but for now we simply cannot predict the arrival of earthquakes.
However, even though earthquakes cannot be predicted, at least not yet, it would be wrong to conclude that science has nothing to offer. First, understanding how earthquakes occur can help us design buildings and make other changes to limit the damage even if we don't know exactly when an earthquake will occur. Second, if an earthquake happens and, despite our best efforts to insulate against it there are still substantial consequences, science can help us to offset and limit the damage. To name just one example, the science surrounding disease transmission helps use to avoid contaminated water supplies after a disaster, something that often compounds tragedy when this science is not available. But there are lots of other things we can do as well, including using the models to determine where help is most needed.
So even if we cannot predict earthquakes, and we can't, the models are still useful for understanding how earthquakes happen. This understanding is valuable because it helps us to prepare for disasters in advance, and to determine policies that will minimize their impact after they happen.
All of this can be applied to macroeconomics. Whether or not we should have predicted the financial earthquake is a question that has been debated extensively, so I am going to set that aside. One side says financial market price changes, like earthquakes, are inherently unpredictable -- we will never predict them no matter how good our models get (the efficient markets types). The other side says the stresses that were building were obvious. Like the stresses that build when tectonic plates moving in opposite directions rub against each other, it was only a question of when, not if. (But even when increasing stress between two plates is observable, scientists cannot tell you for sure if a series of small earthquakes will relieve the stress and do little harm, or if there will be one big adjustment that relieves the stress all at once. With respect to the financial crisis, economists expected lots of little, small harm causing adjustments, instead we got the "big one," and the "buildings and other structures" we thought could withstand the shock all came crumbling down. On prediction in economics, perhaps someday improved models will allow us to do better than we have so far at predicting the exact timing of crises, and I think that earthquakes provide some guidance here. You have to ask first if stress is building in a particular sector, and then ask if action needs to be taken because the stress has reached dangerous levels, levels that might result in a big crash rather than a series of small stress relieving adjustments. I don't think our models are very good at detecting accumulating stress...
Whether the financial crisis should have been predicted or not, the fact that it wasn't predicted does not mean that macroeconomic models are useless any more than the failure to predict earthquakes implies that earthquake science is useless. As with earthquakes, even when prediction is not possible (or missed), the models can still help us to understand how these shocks occur. That understanding is useful for getting ready for the next shock, or even preventing it, and for minimizing the consequences of shocks that do occur.
But we have done much better at dealing with the consequences of unexpected shocks ex-post than we have at getting ready for these a priori. Our equivalent of getting buildings ready for an earthquake before it happens is to use changes in institutions and regulations to insulate the financial sector and the larger economy from the negative consequences of financial and other shocks. Here I think economists made mistakes - our "buildings" were not strong enough to withstand the earthquake that hit. We could argue that the shock was so big that no amount of reasonable advance preparation would have stopped the "building" from collapsing, but I think it's more the case that enough time has passed since the last big financial earthquake that we forgot what we needed to do. We allowed new buildings to be constructed without the proper safeguards.
However, that doesn't mean the models themselves were useless. The models were there and could have provided guidance, but the implied "building codes" were ignored. Greenspan and others assumed no private builder would ever construct a building that couldn't withstand an earthquake, the market would force them to take this into consideration. But they were wrong about that, and even Greenspan now admits that government building codes are necessary. It wasn't the models, it was how they were used (or rather not used) that prevented us from putting safeguards into place.
We haven't failed at this entirely though. For example, we have had some success at putting safeguards into place before shocks occur, automatic stabilizers have done a lot to insulate against the negative consequences of the recession (though they could have been larger to stop the building from swaying as much as it has). So it's not proper to say that our models have not helped us to prepare in advance at all, the insulation social insurance programs provide is extremely important to recognize. But it is the case that we could have and should have done better at preparing before the shock hit.
I'd argue that our most successful use of models has been in cleaning up after shocks rather than predicting, preventing, or insulating against them through pre-crisis preparation. When despite our best effort to prevent it or to minimize its impact a priori, we get a recession anyway, we can use our models as a guide to monetary, fiscal, and other policies that help to reduce the consequences of the shock (this is the equivalent of, after a disaster hits, making sure that the water is safe to drink, people have food to eat, there is a plan for rebuilding quickly and efficiently, etc.). As noted above, we haven't done a very good job at predicting big crises, and we could have done a much better job at implementing regulatory and institutional changes that prevent or limit the impact of shocks. But we do a pretty good job of stepping in with policy actions that minimize the impact of shocks after they occur. This recession was bad, but it wasn't another Great Depression like it might have been without policy intervention.
Whether or not we will ever be able to predict recessions reliably, it's important to recognize that our models still provide considerable guidance for actions we can take before and after large shocks that minimize their impact and maybe even prevent them altogether (though we will have to do a better job of listening to what the models have to say). Prediction is important, but it's not the only use of models.
Saturday, January 24, 2015
Did the Keynesians Get It Wrong in Predicting a Recession in 2013?: I have had several readers send me a blogpost from Scott Sumner saying that the Keynesians have been dishonest in not owning up to the fact that they were wrong in predicting a recession in 2013. The argument is that supposedly us Keynesian types all said that the budget cuts and the ending of the payroll tax cut at the start of 2013 would throw the economy back into recession. (Jeffrey Sachs has made similar claims.)
That isn't my memory of what I said at the time, but hey we can check these things. I looked at a few of my columns from the fall of 2012 and they mostly ran in the opposite direction. The Washington insider types were hyping the threat of the "fiscal cliff" in the hope of pressuring President Obama and the Democrats to make big concessions on Social Security and Medicare. They were saying that even the risk of falling off the cliff could have a big impact on growth in the third and fourth quarter of 2012.
My columns and blogposts (e.g. here, here, here, here, and here) were largely devoted to saying this was crap. I certainly agreed that budget cutbacks and the end of the payroll tax cuts would dampen growth, but the number was between 0.5-0.8 percentage points. This left us far from recession. (All my columns and blogposts from this time are at the CEPR website, so folks can verify that I didn't do any cherry picking here.)
I know Paul Krugman is the real target here, not me, but we've been seeing the economy pretty much the same way since the beginning of the recession. If he had a different story at the time I think I would remember it. But his columns and blogposts are archived too. I really don't think anyone will find him predicting a recession in 2013, although I'm sure he also said that budget cuts and tax increases would dampen growth.
Anyhow, I'm generally happy to stand behind the things I've said, and when they are proven wrong I hope I own up to it. But I don't see any apologies in order. No recession happened in 2013 and none was predicted here.
I don't recall predicting a recession either (the "they" intended to tar all Keynesians refers to just a few people), just that it would be a drag on growth (the CBO predicted 0.6%). In any case, not much can be said unless one takes the time to estimate a model, use it as a baseline, and then ask the model how the economy would have done in an alternative world where policy was different. Just because we still had growth after the budget cuts does not prove or disprove anything. Even if growth rises under austerity, you can't say it would not have risen a bit more more without austerity (all else is far from equal) unless you have done the hard work of estimating a defensible model and then asking it these questions. Similarly, you can't say much about the degree of monetary offset unless you have taken the time to do the econometrics to support it. But with changes this small -- the impact was predicted to be much less than one percent of growth by most models -- it is very hard to get statistically significant differences in any case.
The problem is that there is no model that all economists agree is "best" for these purposes, and the answer one gets depends upon the particular choice of models. Choose a model that delivers small fiscal multipliers and you get one answer, use a model with bigger multipliers and the answer changes. But even the models with the largest multipliers did not predict a recession, only a drag on growth (generally less than one percent) so the fact that we still had growth says nothing about the impact of the policy, or the degree of monetary offset.
Wednesday, December 31, 2014
I'm pretty sure my dissertation advisor (Greg Duncan, a McFadden student at Berkeley) helped to create the model used to make the BART prediction discussed below (I think they used a program called QUAIL, and precursor to LIMDEP):
Here's What Economics Gets Right, by Noah Smith: Criticizing economics for not being scientific enough is a crime of which many of us -- I’ve done it -- are guilty. But there’s a right way to do it and a wrong way to do it. Alex Rosenberg and Tyler Curtain, writing in the New York Times, have done it the wrong way.
Here are Rosenberg and Curtain:
Over time, the question of why economics has not (yet) qualified as a science has become an obsession among theorists, including philosophers of science like us...The trouble with economics is that it lacks the most important of science’s characteristics — a record of improvement in predictive range and accuracy...In fact, when it comes to economic theory’s track record, there isn’t much predictive success to speak of at all.
Economics doesn’t have predictive success, eh? This is something a lot of people claim, but once you look beyond the well-publicized fact that economists can’t predict recessions, you can see that the claim just isn’t true. Economics can predict plenty of things.
My favorite example is the story of Daniel McFadden and the BART...
Wednesday, November 05, 2014
It's a mystery why inflation is such a mystery:
Forecasting Inflation with Fundamentals . . . It’s Hard!, by Jan Groen, Liberty Street Economics, NY Fed: Controlling inflation is at the core of monetary policymaking, and central bankers would like to have access to reliable inflation forecasts to assess their progress in achieving this goal. Producing accurate inflation forecasts, however, turns out not to be a trivial exercise. This posts reviews the key challenges in inflation forecasting and discusses some recent developments that attempt to deal with these challenges. ...
Wednesday, October 22, 2014
Persuasion with statistics: Mark Thoma's point that apparently strong econometric results are often the product of specification mining prompts Lars Syll to remind us that eminent economists have long been wary of what econometrics can achieve.
I doubt if many people have ever thought "Crikey, the t stats are high here. That means I must abandon my long-held beliefs about an important matter." More likely, the reaction is to recall Dave Giles' commandments nine and 10. (Apparently?) impressive econometric findings might be good enough to get you published. But there's a big difference between being published and being read, let alone being persuasive.
This poses a question: how, then, do statistics persuade people to change their general beliefs (as distinct from beliefs about single facts)?
Let me take an example of an issue where I've done just this. I used to believe in the efficient market hypothesis. ...[explains why that changed]...
He concludes with:
The above is not a story about statistical significance (pdf). Single studies are rarely persuasive. Instead, the process of persuading people to change their mind requires diversity - a diversity of data sets, and a diversity of theories. Am I wrong? Feel free to offer counter-examples.
Thursday, October 16, 2014
Brad DeLong wonders why Cliff Asness in clinging to a theoretical model that has clearly been rejected by the data:
... What is not normal is to claim that your analysis back in 2010 that quantitative easing was generating major risks of inflation was dead-on.
What is not normal is to adopt the mental pose that your version of classical austerian economics cannot fail--that it can only be failed by an uncooperative and misbehaving world.
What is not normal is, after 4 1/2 years, in a week, a month, a six-month period in which market expectations of long-run future inflation continue on a downward trajectory, to refuse to mark your beliefs to market and demand that the market mark its beliefs to you. To still refuse to bring your mind into agreement with reality and demand that reality bring itself into agreement with your mind. To still refuse to say: "my intellectual adversaries back in 2010 had a definite point" and to say only: "IT'S NOT OVER YET!!!!"
There's a version of this in econometrics, i.e. you know the model is correct, you are just having trouble finding evidence for it. It goes as follows. You are testing a theory you came up with, but the data are uncooperative and say you are wrong. But instead of accepting that, you tell yourself "My theory is right, I just haven't found the right econometric specification yet. I need to add variables, remove variables, take a log, add an interaction, square a term, do a different correction for misspecification, try a different sample period, etc., etc., etc." Then, after finally digging out that one specification of the econometric model that confirms your hypothesis, you declare victory, write it up, and send it off (somehow never mentioning the intense specification mining that produced the result).
Too much econometric work proceeds along these lines. Not quite this blatantly, but that is, in effect, what happens in too many cases. I think it is often best to think of econometric results as the best case the researcher could make for a particular theory rather than a true test of the model.
Monday, September 29, 2014
Why are economic forecasts wrong so often?: The Queen of England famously asked why economists failed to foresee the financial crisis in 2008. "Why did nobody notice it?" was her question when she visited the London School of Economics that year.
Economists' failure to accurately predict the economy's course isn't limited to the financial crisis and the Great Recession that followed. Macroeconomic computer models also aren't very useful for predicting how variables such as GDP, employment, interest rates and inflation will evolve over time.
Forecasting most things is fraught with difficulty. See the current dust-up between Nate Silver and Sam Wang over their conflicting predictions about the coming Senate elections. Why is forecasting so hard?
Because so many things can go wrong. For example...
Monday, September 22, 2014
The NY Fed hopes that someday the FRBNY DSGE model will be useful for forecasting. Presently, the model has "huge margins for improvement. The list of flaws is long..." (first in a five-part series):
Forecasting with the FRBNY DSGE Model, by Marco Del Negro, Bianca De Paoli, Stefano Eusepi, Marc Giannoni, Argia Sbordone, and Andrea Tambalotti, Liberty Economics, FRBNY: The Federal Reserve Bank of New York (FRBNY) has built a DSGE model as part of its efforts to forecast the U.S. economy. On Liberty Street Economics, we are publishing a weeklong series to provide some background on the model and its use for policy analysis and forecasting, as well as its forecasting performance. In this post, we briefly discuss what DSGE models are, explain their usefulness as a forecasting tool, and preview the forthcoming pieces in this series.
The term DSGE, which stands for dynamic stochastic general equilibrium, encompasses a very broad class of macro models, from the standard real business cycle (RBC) model of Nobel prizewinners Kydland and Prescott to New Keynesian monetary models like the one of Christiano, Eichenbaum, and Evans. What distinguishes these models is that rules describing how economic agents behave are obtained by solving intertemporal optimization problems, given assumptions about the underlying environment, including the prevailing fiscal and monetary policy regime. One of the benefits of DSGE models is that they can deliver a lens for understanding the economy’s behavior. The third post in this series will show an example of this role with a discussion of the forces behind the Great Recession and the following slow recovery.
DSGE models are also quite abstract representations of reality, however, which in the past severely limited their empirical appeal and forecasting performance. This started to change with work by Schorfheide and Smets and Wouters. First, they popularized estimation (especially Bayesian estimation) of these models, with parameters chosen in a way that increased the ability of these models to describe the time series behavior of economic variables. Second, these models were enriched with both endogenous and exogenous propagation mechanisms that allowed them to better capture patterns in the data. For this reason, estimated DSGE models are increasingly used within the Federal Reserve System (the Board of Governors and the Reserve Banks of Chicago and Philadelphia have versions) and by central banks around the world (including the New Area-Wide Model developed at the European Central Bank, and models at the Norges Bank and the Sveriges Riksbank). The FRBNY DSGE model is a medium-scale model in the tradition of Christiano, Eichenbaum, and Evans and Smets and Wouters that also includes credit frictions as in the financial accelerator model developed by Bernanke, Gertler, and Gilchrist and further investigated by Christiano, Motto, and Rostagno. The second post in this series elaborates on what DSGE models are and discusses the features of the FRBNY model.
Perhaps some progress was made in the past twenty years toward empirical fit, but is it enough to give forecasts from DSGE models any credence? Aren’t there many critics out there (here is one) telling us these models are a failure? As it happens, not many people seem to have actually checked the extent to which these model forecasts are off the mark. Del Negro and Schorfheide do undertake such an exercise in a chapter of the recent Handbook of Economic Forecasting. Their analysis compares the real-time forecast accuracy of DSGE models that were available prior to the Great Recession (such as the Smets and Wouters model) to that of the Blue Chip consensus forecasts, using a period that includes the Great Recession. They find that, for nowcasting (forecasting current quarter variables) and short-run forecasting, DSGE models are at a disadvantage compared with professional forecasts. Over the medium- and long-run terms, however, DSGE model forecasts for both output and inflation become competitive with—if not superior to—professional forecasts. They also find that including timely information from financial markets such as credit spreads can dramatically improve the models’ forecasts, especially in the Great Recession period.
These results are based on what forecasters call “pseudo-out-of-sample” forecasts. These are not truly “real time” forecasts, because they were not produced at the time. (To our knowledge, there is little record of truly real time DSGE forecasts for the United States, partly because these models were only developed in the mid-2000s.) For this reason, in the fourth post of this series, we report forecasts produced in real time using the FRBNY DSGE model since 2010. These forecasts have been included in internal New York Fed documents, but were not previously made public. Although the sample is admittedly short, these forecasts show that while consensus forecasts were predicting a relatively rapid recovery from the Great Recession, the DSGE model was correctly forecasting a more sluggish recovery.
The last post in the series shows the current FRBNY DSGE forecasts for output growth and inflation and discusses the main economic forces driving the predictions. Bear in mind that these forecasts are not the official New York Fed staff forecasts; the DSGE model is only one of many tools employed for prediction and policy analysis at the Bank.
DSGE models in general and the FRBNY model in particular have huge margins for improvement. The list of flaws is long, ranging from the lack of heterogeneity (the models assume a representative household) to the crude representation of financial markets (the models have no term premia). Nevertheless, we are sticking our necks out and showing our forecasts, not because we think we have a “good” model of the economy, but because we want to have a public record of the model’s successes and failures. In doing so, we can learn from both our past performance and readers’ criticism. The model is a work in progress. Hopefully, it can be improved over time, guided by current economic and policy questions and benefiting from developments in economic theory and econometric tools.
Sunday, September 14, 2014
Via email, I was asked if this is the "stupidest article ever published?":
If not, it's certainly in the running.
Monday, September 08, 2014
Matthew O. Jackson, Stanford University Social and Economic Networks: Backgound
Daron Acemoglu, MIT Networks: Games over Networks and Peer Effects
Matthew O. Jackson, Stanford University Diffusion, Identification, Network Formation
Daron Acemoglu, MIT Networks: Propagation of Shocks over Economic Networks
Wednesday, September 03, 2014
Thursday, August 14, 2014
Jennifer Castle and David Hendry on data mining
‘Data mining’ with more variables than observations: While ‘fool’s gold’ (iron pyrites) can be found by mining, most mining is a productive activity. Similarly, when properly conducted, so-called ‘data mining’ is no exception –despite many claims to the contrary. Early criticisms, such as the review of Tinbergen (1940) by Friedman (1940) for selecting his equations “because they yield high coefficients of correlation”, and by Lovell (1983) and Denton (1985) of data mining based on choosing ‘best fitting’ regressions, were clearly correct. It is also possible to undertake what Gilbert (1986) called ‘strong data mining’, whereby an investigator tries hundreds of empirical estimations, and reports the one she or he ‘prefers’ – even when such results are contradicted by others that were found. As Leamer (1983) expressed the matter: “The econometric art as it is practiced at the computer terminal involves fitting many, perhaps thousands, of statistical models. One or several that the researcher finds pleasing are selected for reporting purposes”. That an activity can be done badly does not entail that all approaches are bad, as stressed by Hoover and Perez (1999), Campos and Ericsson (1999), and Spanos (2000) – driving with your eyes closed is a bad idea, but most car journeys are safe.
Why is ‘data mining’ needed?
Econometric models need to handle many complexities if they are to have any hope of approximating the real world. There are many potentially relevant variables, dynamics, outliers, shifts, and non-linearities that characterise the data generating process. All of these must be modelled jointly to build a coherent empirical economic model, necessitating some form of data mining – see the approach described in Castle et al. (2011) and extensively analysed in Hendry and Doornik (2014).
Any omitted substantive feature will result in erroneous conclusions, as other aspects of the model attempt to proxy the missing information. At first sight, allowing for all these aspects jointly seems intractable, especially with more candidate variables (denoted N) than observations (T denotes the sample size). But help is at hand with the power of a computer. ...[gives technical details]...
Appropriately conducted, data mining can be a productive activity even with more candidate variables than observations. Omitting substantively relevant effects leads to mis-specified models, distorting inference, which large initial specifications should mitigate. Automatic model selection algorithms like Autometrics offer a viable approach to tackling more candidate variables than observations, controlling spurious significance.
Monday, August 11, 2014
...The rather boring truth is that it is entirely predictable that forecasters will miss major recessions, just as it is equally predictable that each time this happens we get hundreds of articles written asking what has gone wrong with macro forecasting. The answer is always the same - nothing. Macroeconomic model based forecasts are always bad, but probably no worse than intelligent guesses.
Monday, July 28, 2014
Alan Blinder and Mark Watson:
Presidents and the U.S. Economy: An Econometric Exploration, by Alan S. Blinder and Mark W. Watson, NBER Working Paper No. 20324 [open link]: The U.S. economy has grown faster—and scored higher on many other macroeconomic metrics—when the President of the United States is a Democrat rather than a Republican. For many measures, including real GDP growth (on which we concentrate), the performance gap is both large and statistically significant, despite the fact that postwar history includes only 16 complete presidential terms. This paper asks why. The answer is not found in technical time series matters (such as differential trends or mean reversion), nor in systematically more expansionary monetary or fiscal policy under Democrats. Rather, it appears that the Democratic edge stems mainly from more benign oil shocks, superior TFP performance, a more favorable international environment, and perhaps more optimistic consumer expectations about the near-term future. Many other potential explanations are examined but fail to explain the partisan growth gap.
Friday, July 18, 2014
Further thoughts on Phillips curves: In a post from a few days ago I looked at some recent evidence on Phillips curves, treating the Great Recession as a test case. I cast the discussion as a debate between rational and adaptive expectations. Neither is likely to be 100% right of course, but I suggested the evidence implied rational expectations were more right than adaptive. In this post I want to relate this to some other people’s work and discussion. (See also this post from Mark Thoma.) ...
The first issue is why look at just half a dozen years, in only a few countries. As I noted in the original post, when looking at CPI inflation there are many short term factors that may mislead. Another reason for excluding European countries which I did not mention is the impact of austerity driven higher VAT rates (and other similar taxes or administered prices), nicely documented by Klitgaard and Peck. Surely all this ‘noise’ is an excellent reason to look over a much longer time horizon?
One answer is given in this recent JEL paper by Mavroeidis, Plagborg-Møller and Stock. As Plagborg-Moller notes in an email to Mark Thoma: “Our meta-analysis finds that essentially any desired parameter estimates can be generated by some reasonable-sounding specification. That is, estimation of the NKPC is subject to enormous specification uncertainty. This is consistent with the range of estimates reported in the literature….traditional aggregate time series analysis is just not very informative about the nature of inflation dynamics.” This had been my reading based on work I’d seen.
This is often going to be the case with time series econometrics, particularly when key variables appear in the form of expectations. Faced with this, what economists often look for is some decisive and hopefully large event, where all the issues involving specification uncertainty can be sidelined or become second order. The Great Recession, for countries that did not suffer a second recession, might be just such an event. In earlier, milder recessions it was also much less clear what the monetary authority’s inflation target was (if it had one at all), and how credible it was. ...
I certainly agree with the claim that a "decisive and hopefully large event" is needed to empirically test econometric models since I've made the same point many times in the past. For example, "...the ability to choose one model over the other is not quite as hopeless as I’ve implied. New data and recent events like the Great Recession push these models into unchartered territory and provide a way to assess which model provides better predictions. However, because of our reliance on historical data this is a slow process – we have to wait for data to accumulate – and there’s no guarantee that once we are finally able to pit one model against the other we will be able to crown a winner. Both models could fail..."
Anyway...he goes on to discuss "How does what I did relate to recent discussions by Paul Krugman?," and concludes with:
My interpretation suggests that the New Keynesian Phillips curve is a more sensible place to start from than the adaptive expectations Friedman/Phelps version. As this is the view implicitly taken by most mainstream academic macroeconomics, but using a methodology that does not ensure congruence with the data, I think it is useful to point out when the mainstream does have empirical support. ...
Monday, July 14, 2014
Via email, a comment on my comments about the difficulty of settling questions about the Phillips curve empirically:
Dear Professor Thoma,
I saw your recent post on the difficulty of empirically testing the Phillips Curve, and I just wanted to alert you to a survey paper on this topic that I wrote with Sophocles Mavroeidis and Jim Stock: "Empirical Evidence on Inflation Expectations in the New Keynesian Phillips Curve". It was published in the Journal of Economic Literature earlier this year (ungated working paper).
In the paper we estimate a vast number of specifications of the New Keynesian Phillips Curve (NKPC) on a common U.S. data set. The specification choices include the data series, inflation lag length, sample period, estimator, and so on. A subset of the specifications amount to traditional backward-looking (adaptive expectation) Phillips Curves. We are particularly interested in two key parameters: the extent to which price expectations are forward-looking, and the slope of the curve (how responsive inflation is to real economic activity).
Our meta-analysis finds that essentially any desired parameter estimates can be generated by some reasonable-sounding specification. That is, estimation of the NKPC is subject to enormous specification uncertainty. This is consistent with the range of estimates reported in the literature. Even if one were to somehow decide on a given specification, the uncertainty surrounding the parameter estimates is typically large. We give theoretical explanations for these empirical findings in the paper. To be clear: Our results do not reject the validity of the NKPC (or more generally, the presence of a short-run inflation/output trade-off), but traditional aggregate time series analysis is just not very informative about the nature of inflation dynamics.
PhD candidate in economics, Harvard University
Tuesday, June 03, 2014
Why Economists Can’t Always Trust Data, by Mark Thoma, The Fiscal Times: To make progress in economics, it is essential that theoretical models be subjected to empirical tests that determine how well they can explain actual data. The tests that are used must be able to draw a sharp distinction between competing theoretical models, and one of the most important factors is the quality of the data used in the tests. Unfortunately, the quality of the data that economists employ is less than ideal, and this gets in the way of the ability of economists to improve the models they use. There are several reasons for the poor quality of economic data...
Tuesday, May 20, 2014
At MoneyWatch, what is econometrics?:
Using Econometrics to Figure Out How the World Really Works, by Mark Thoma: Many people believe there has been no progress in economics, but that isn't true. ...
Friday, May 02, 2014
From the Journal of Economic Perspectives' Symposium on Big Data:
"Big Data: New Tricks for Econometrics," by Hal R. Varian: Computers are now involved in many economic transactions and can capture data associated with these transactions, which can then be manipulated and analyzed. Conventional statistical and econometric techniques such as regression often work well, but there are issues unique to big datasets that may require different tools. First, the sheer size of the data involved may require more powerful data manipulation tools. Second, we may have more potential predictors than appropriate for estimation, so we need to do some kind of variable selection. Third, large datasets may allow for more flexible relationships than simple linear models. Machine learning techniques such as decision trees, support vector machines, neural nets, deep learning, and so on may allow for more effective ways to model complex relationships. In this essay, I will describe a few of these tools for manipulating and analyzing big data. I believe that these methods have a lot to offer and should be more widely known and used by economists. Full-Text Access | Supplementary Materials
Monday, April 14, 2014
This may be of interest:
“There will be growth in the spring”: How well do economists predict turning points?, by Hites Ahir and Prakash Loungani: Forecasters have a poor reputation for predicting recessions. This column quantifies their ability to do so, and explores several reasons why both official and private forecasters may fail to call a recession before it happens.
Thursday, April 10, 2014
"Past performance is not an indicator of future results":
Pseudo-mathematics and financial charlatanism, EurekAlert: Your financial advisor calls you up to suggest a new investment scheme. Drawing on 20 years of data, he has set his computer to work on this question: If you had invested according to this scheme in the past, which portfolio would have been the best? His computer assembled thousands of such simulated portfolios and calculated for each one an industry-standard measure of return on risk. Out of this gargantuan calculation, your advisor has chosen the optimal portfolio. After briefly reminding you of the oft-repeated slogan that "past performance is not an indicator of future results", the advisor enthusiastically recommends the portfolio, noting that it is based on sound mathematical methods. Should you invest?
The somewhat surprising answer is, probably not. Examining a huge number of sample past portfolios---known as "backtesting"---might seem like a good way to zero in on the best future portfolio. But if the number of portfolios in the backtest is so large as to be out of balance with the number of years of data in the backtest, the portfolios that look best are actually just those that target extremes in the dataset. When an investment strategy "overfits" a backtest in this way, the strategy is not capitalizing on any general financial structure but is simply highlighting vagaries in the data. ...
Unfortunately, the overfitting of backtests is commonplace not only in the offerings of financial advisors but also in research papers in mathematical finance. One way to lessen the problems of backtest overfitting is to test how well the investment strategy performs on data outside of the original dataset on which the strategy is based; this is called "out-of-sample" testing. However, few investment companies and researchers do out-of-sample testing. ...
Tuesday, March 11, 2014
Inequality in Capitalist Systems Is Not Inevitable, by Mark Thoma: Capitalism is the best economic system yet discovered for giving people the goods and services they desire at the lowest possible price, and for producing innovative economic growth. But there is a cost associated with these benefits, the boom and bust cycles inherent in capitalist systems, and those costs hit working class households – who have done nothing to deserve such a fate – very hard. Protecting innocent households from the costs of recessions is an important basis for our social insurance programs.
It is becoming more and more evident that there is another cost of capitalist systems, the inevitable rising inequality documented by Thomas Piketty in “Capital in the Twenty-First Century, that our social insurance system will need to confront. ...
Thursday, January 23, 2014
On R-squared and economic prediction: Recently I've heard a number of otherwise intelligent people assess an economic hypothesis based on the R2 of an estimated regression. I'd like to point out why that can often be very misleading. ...
Here's what you'd find if you calculated a regression of this month's stock price (pt) on last month's stock price (pt-1). Standard errors of the regression coefficients are in parentheses.
The adjusted R-squared for this relation is 0.997. ... On the other hand, another way you could summarize the same relation is by using the change in the stock price (Δpt = pt - pt-1) as the left-hand variable in the regression:
This is in fact the identical model of stock prices as the first regression. The standard errors of the regression coefficients are identical for the two regressions, and the standard error of the estimate ... is identical for the two regressions because indeed the residuals are identical for every observation. ...
Whatever you do, don't say that the first model is good given its high R-squared and the second model is bad given its low R-squared, because equations (1) and (2) represent the identical model. ...
That's not a bad empirical description of stock prices-- nobody can really predict them. ... This is actually a feature of a broad class of dynamic economic models, which posit that ... the deviation between what actually happens and what the decision-maker intended ... should be impossible to predict if the decision-maker is behaving rationally. For example, if everybody knew that a recession is coming 6 months down the road, the Fed should be more expansionary today... The implication is that when recessions do occur, they should catch the Fed and everyone else by surprise.
It's very helpful to look critically at which magnitudes we can predict and which we can't, and at whether that predictability or lack of predictability is consistent with our economic understanding of what is going on. But if what you think you learned in your statistics class was that you should always judge how good a model is by looking at the R-squared of a regression, then I hope that today you learned something new.
[There's an additional example and more explanation in the original post.]
Thursday, January 02, 2014
... Metrika began as a small village - little more than a coach-stop and a mandatory tavern at a junction in the highway running from the ancient data mines in the South, to the great city of Enlightenment, far to the North. In Metrika, the transporters of data of all types would pause overnight on their long journey; seek refreshment at the tavern; and swap tales of their experiences on the road.
To be fair, the data transporters were more than just humble freight carriers. The raw material that they took from the data mines was largely unprocessed. The vast mountains of raw numbers usually contained valuable gems and nuggets of truth, but typically these were buried from sight. The data transporters used the insights that they gained from their raucous, beer-fired discussions and arguments (known locally as "seminars") with the Metrika yokels locals at the tavern to help them to sift through the data and extract the valuable jewels. With their loads considerably lightened, these "data-miners" then continued on their journey to the City of Enlightenment in a much improved frame of mind, hangovers nothwithstanding!
Over time, the town of Metrika prospered and grew as the talents of its citizens were increasingly recognized and valued by those in the surrounding districts, and by the data miners transporters.
Young Joe grew up happily, supported by his family of econometricians, and he soon developed the skills that were expected of his societal class. He honed his computing skills; developed a good nose for "dodgy" data; and studiously broadened and deepened his understanding of the various tools wielded by the artisans in the neighbouring town of Statsbourg.
In short, he was a model child!
But - he was torn! By the time that he reached the tender age of thirteen, he felt the need to make an important, life-determining, decision.
Should he align his talents with the burly crew who frequented the gym near his home - the macroeconometricians - or should he throw in his lot with the physically challenged bunch of empirical economists known locally as the microeconometricians? ...
Full story here.
Saturday, December 07, 2013
I tweeted this link, and it's getting far, far more retweets than I would have expected, so I thought I'd note it here:
Econometrics and "Big Data", by Dave Giles: In this age of "big data" there's a whole new language that econometricians need to learn. ... What do you know about such things as:
- Decision trees
- Support vector machines
- Neural nets
- Deep learning
- Classification and regression trees
- Random forests
- Penalized regression (e.g., the lasso, lars, and elastic nets)
- Spike and slab regression?
Probably not enough!
If you want some motivation to rectify things, a recent paper by Hal Varian ... titled, "Big Data: New Tricks for Econometrics" ... provides an extremely readable introduction to several of these topics.
He also offers a valuable piece of advice:
"I believe that these methods have a lot to offer and should be more widely known and used by economists. In fact, my standard advice to graduate students these days is 'go to the computer science department and take a class in machine learning'."
Monday, September 09, 2013
Ten Things for Applied Econometricians to Keep in Mind, by Dave Giles: No "must do" list is ever going to be complete, let alone perfect. This is certainly true when it comes to itemizing essential ground-rules for all of us when we embark on applying our knowledge of econometrics.
That said, here's a list of ten things that I like my students to keep in mind:
- Always, but always, plot your data.
- Remember that data quality is at least as important as data quantity.
- Always ask yourself, "Do these results make economic/common sense"?
- Check whether your "statistically significant" results are also "numerically/economically significant".
- Be sure that you know exactly what assumptions are used/needed to obtain the results relating to the properties of any estimator or test that you use.
- Just because someone else has used a particular approach to analyse a problem that looks like, that doesn't mean they were right!
- "Test, test, test"! (David Hendry). But don't forget that "pre-testing" raises some important issues of its own.
- Don't assume that the computer code that someone gives to you is relevant for your application, or that it even produces correct results.
- Keep in mind that published results will represent only a fraction of the results that the author obtained, but is not publishing.
- Don't forget that "peer-reviewed" does NOT mean "correct results", or even "best practices were followed".
I'm sure you can suggest how this list can be extended!
I'll add two that I heard often in grad school:
Don't take econometric techniques in search of questions. Instead, start with the important questions and then develop the econometrics needed to answer them.
Model the process that generates the data.
Any further suggestions?
Sunday, April 21, 2013
There will be a big revision of macroeconomic data in July:
Data shift to lift US economy by 3%, by Robin Harding, FT: The US economy will officially become 3 per cent bigger in July as part of a shake-up that will for the first time see government statistics take into account 21st century components such as film royalties and spending on research and development. ...
In an interview with the Financial Times, Brent Moulton, who manages the national accounts at the Bureau of Economic Analysis, said the update is the biggest since computer software was added to the accounts in 1999.
“We are carrying these major changes all the way back in time – which for us means to 1929 – so we are essentially rewriting economic history,” said Mr Moulton.
The changes will affect everything from the measured GDP of different US states to the stability of the inflation measure targeted by the US Federal Reserve. They will force economists to revisit policy debates about everything from corporate profits to the causes of economic growth. ...
The changes are in addition to a comprehensive revision of the national accounts that takes place every five years... Steve Landefeld, the BEA director, said it was hard to predict the overall outcome given the mixture of new methodology and data updates. ... But while the level of GDP may change,... “I wouldn’t be looking for large changes in trends or cycles,” said Mr Landefeld. ...
When working with macroeconomic data, we don't generally assume that there are large measurement errors in the data when assessing the significance of the results. Maybe we should.
Wednesday, April 17, 2013
The blow-up over the Reinhart-Rogoff results reminds me of a point I’ve been meaning to make about our ability to use empirical methods to make progress in macroeconomics. This isn't about the computational mistakes that Reinhart and Rogoff made, though those are certainly important, especially in small samples, it's about the quantity and quality of the data we use to draw important conclusions in macroeconomics.
Everybody has been highly critical of theoretical macroeconomic models, DSGE models in particular, and for good reason. But the imaginative construction of theoretical models is not the biggest problem in macro – we can build reasonable models to explain just about anything. The biggest problem in macroeconomics is the inability of econometricians of all flavors (classical, Bayesian) to definitively choose one model over another, i.e. to sort between these imaginative constructions. We like to think or ourselves as scientists, but if data can’t settle our theoretical disputes – and it doesn’t appear that it can – then our claim for scientific validity has little or no merit.
There are many reasons for this. For example, the use of historical rather than “all else equal” laboratory/experimental data makes it difficult to figure out if a particular relationship we find in the data reveals an important truth rather than a chance run that mimics a causal relationship. If we could do repeated experiments or compare data across countries (or other jurisdictions) without worrying about the “all else equal assumption” we’d could perhaps sort this out. It would be like repeated experiments. But, unfortunately, there are too many institutional differences and common shocks across countries to reliably treat each country as an independent, all else equal experiment. Without repeated experiments – with just one set of historical data for the US to rely upon – it is extraordinarily difficult to tell the difference between a spurious correlation and a true, noteworthy relationship in the data.
Even so, if we had a very, very long time-series for a single country, and if certain regularity conditions persisted over time (e.g. no structural change), we might be able to answer important theoretical and policy questions (if the same policy is tried again and again over time within a country, we can sort out the random and the systematic effects). Unfortunately, the time period covered by a typical data set in macroeconomics is relatively short (so that very few useful policy experiments are contained in the available data, e.g. there are very few data points telling us how the economy reacts to fiscal policy in deep recessions).
There is another problem with using historical as opposed to experimental data, testing theoretical models against data the researcher knows about when the model is built. In this regard, when I was a new assistant professor Milton Friedman presented some work at a conference that impressed me quite a bit. He resurrected a theoretical paper he had written 25 years earlier (it was his plucking model of aggregate fluctuations), and tested it against the data that had accumulated in the time since he had published his work. It’s not really fair to test a theory against historical macroeconomic data, we all know what the data say and it would be foolish to build a model that is inconsistent with the historical data it was built to explain – of course the model will fit the data, who would be impressed by that? But a test against data that the investigator could not have known about when the theory was formulated is a different story – those tests are meaningful (Friedman’s model passed the test using only the newer data).
As a young time-series econometrician struggling with data/degrees of freedom issues I found this encouraging. So what if in 1986 – when I finished graduate school – there were only 28 quarterly observations for macro variables (112 total observations, reliable data on money, which I almost always needed, doesn’t begin until 1959). By, say, the end of 2012 there would be almost double that amount (216 versus 112!!!). Asymptotic (plim-type) results here we come! (Switching to monthly data doesn’t help much since it’s the span of the data – the distance between the beginning and the end of the sample – rather than the frequency the data are sampled that determines many of the “large-sample results”).
By today, I thought, I would have almost double the data I had back then and that would improve the precision of tests quite a bit. I could also do what Friedman did, take really important older papers that give us results “everyone knows” and see if they hold up when tested against newer data.
It didn’t work out that way. There was a big change in the Fed’s operating procedure in the early 1980s, and because of this structural break today 1984 is a common starting point for empirical investigations (start dates can be anywhere in the 79-84 range though later dates are more common). Data before this time-period are discarded.
So, here we are 25 years or so later and macroeconomists don’t have any more data at our disposal than we did when I was in graduate school. And if the structure of the economy keeps changing – as it will – the same will probably be true 25 years from now. We will either have to model the structural change explicitly (which isn’t easy, and attempts to model structural beaks often induce as much uncertainty as clarity), or continually discard historical data as time goes on (maybe big data, digital technology, theoretical advances, etc. will help?).
The point is that for a variety of reasons – the lack of experimental data, small data sets, and important structural change foremost among them – empirical macroeconomics is not able to definitively say which competing model of the economy best explains the data. There are some questions we’ve been able to address successfully with empirical methods, e.g., there has been a big change in views about the effectiveness of monetary policy over the last few decades driven by empirical work. But for the most part empirical macro has not been able to settle important policy questions. The debate over government spending multipliers is a good example. Theoretically the multiplier can take a range of values from small to large, and even though most theoretical models in use today say that the multiplier is large in deep recessions, ultimately this is an empirical issue. I think the preponderance of the empirical evidence shows that multipliers are, in fact, relatively large in deep recessions – but you can find whatever result you like and none of the results are sufficiently definitive to make this a fully settled issue.
I used to think that the accumulation of data along with ever improving empirical techniques would eventually allow us to answer important theoretical and policy questions. I haven’t completely lost faith, but it’s hard to be satisfied with our progress to date. It’s even more disappointing to see researchers overlooking these well-known, obvious problems – for example the lack pf precision and sensitivity to data errors that come with the reliance on just a few observations – to oversell their results.
Thursday, April 04, 2013
I should have noted this when I posted the conference schedule. If you want to watch a live feed of the sessions, it's at:
Remember that all times listed are Hong Kong Time (15 hours ahead of PST, 12 hours ahead of EST). Videos of each session will also be posted (same address as the link above).
Friday, March 29, 2013
Is "Intellectual Property" a Misnomer?, by Tim Taylor: The terminology of "intellectual property" goes back to the eighteenth century. But some modern critics of how the patent and copyright law have evolved have come to view the term as a tendentious choice. One you have used the "property" label, after all, you are implicitly making a claim about rights that should be enforced by the broader society. But "intellectual property" is a much squishier subject than more basic applications of property, like whether someone can move into your house or drive away in your car or empty your bank account. ...
Is it really true that using someone else's invention is the actually the same thing as stealing their sheep? If I steal your sheep, you don't have them any more. If I use your idea, you still have the idea, but are less able to profit from using it. The two concepts may be cousins, but they not identical.
Those who believe that patent protection has in some cases gone overboard, and is now in many industries acting more to protect established firms than to encourage new innovators, thus refer to "intellectual property as a "propaganda term." For a vivid example of these arguments, see "The Case Against Patents," by Michele Boldrin and David K. Levine, in the Winter 2013 issue of my own Journal of Economic Perspectives. (Like all articles in JEP back to the first issue in 1987, it is freely available on-line courtesy of the American Economic Association.)
Mark Lemley offers a more detailed unpacking of the concept of "intellectual property" in a 2005 article he wrote for the Texas Law Review called "Property, Intellectual Property, and Free Riding" Lemley writes: ""My worry is that the rhetoric of property has a clear meaning in the minds of courts, lawyers and commentators as “things that are owned by persons,” and that fixed meaning will make all too tempting to fall into the trap of treating intellectual property just like “other” forms of property. Further, it is all too common to assume that because something is property, only private and not public rights are implicated. Given the fundamental differences in the economics of real property and intellectual property, the use of the property label is simply too likely to mislead."
As Lemley emphasizes, intellectual property is better thought of as a kind of subsidy to encourage innovation--although the subsidy is paid in the form of higher prices by consumers rather than as tax collected from consumers and then spent by the government. A firm with a patent is able to charge more to consumers, because of the lack of competition, and thus earn higher profits. There is reasonably broad agreement among economists that it makes sense for society to subsidize innovation in certain ways, because innovators have a hard time capturing the social benefits they provide in terms of greater economic growth and a higher standard of living, so without some subsidy to innovation, it may well be underprovided.
But even if you buy that argument, there is room for considerable discussion of the most appropriate ways to subsidize innovation. How long should a patent be? Should the length or type of patent protection differ by industry? How fiercely or broadly should it be enforced by courts? In what ways might U.S. patent law be adapted based on experiences and practices in other major innovating nations like Japan or Germany? What is the role of direct government subsidies for innovation in the form of government-sponsored research and development? What about the role of indirect government subsidies for innovation in the form of tax breaks for firms that do research and development, or in the form of support for science, technology, and engineering education? Should trade secret protection be stronger, and patent protection be weaker, or vice versa?
These are all legitimate questions about the specific form and size of the subsidy that we provide to innovation. None of the questions about "intellectual property" can be answered yelling "it's my property."
The phrase "intellectual property" has been around a few hundred years, so it clearly has real staying power and widespread usage I don't expect the term to disappear. But perhaps we can can start referring to intellectual "property" in quotation marks, as a gentle reminder that an overly literal interpretation of the term would be imprudent as a basis for reasoning about economics and public policy.
Thursday, March 07, 2013
Dean Baker's blog is called "Beat the Press," but he praised this effort (the original is quite a bit longer, and makes additional points):
The War On Entitlements, by Thomas Edsall, Commentary, NY Times: ...Currently, earned income in excess of $113,700 is entirely exempt from the 6.2 percent payroll tax that funds Social Security benefits... Simply by eliminating the payroll tax earnings cap — and thus ending this regressive exemption for the top 5.2 percent of earners — would, according to the Congressional Budget Office, solve the financial crisis facing the Social Security system.
So why don’t we talk about raising or eliminating the cap – a measure that has strong popular, though not elite, support? ... The Washington cognoscenti are more inclined to discuss two main approaches...: means-testing of benefits and raising the age of eligibility for Social Security and Medicare. ... Means-testing and raising the age of eligibility as methods of cutting spending appeal to ideological conservatives for a number of reasons.
First, insofar as benefits for the affluent are reduced or eliminated under means-testing, social insurance programs are no longer universal and are seen, instead, as a form of welfare. Public support would almost certainly decline, encouraging further cuts in the future. Second, the focus on means-testing and raising the age of eligibility diverts attention from a much simpler and more equitable approach: raising the payroll tax to apply to the earnings of the well-to-do, a step strongly opposed by the ideological right. ... Third, and most important in terms of the policy debate, while both means-testing and eliminating the $113,700 cap on earnings subject to the payroll tax hurt the affluent, the latter would inflict twice as much pain. ...
Theda Skocpol ... of ... Harvard and an authority on the history of the American welfare state contended ... that policy elites avoid addressing the sharply regressive nature of social welfare taxes because, “at one level, it’s very, very privileged people wanting to make sure they cut spending on everybody else” while “holding down their own taxes.” ...
Wednesday, July 11, 2012
Stephen Ziliak, via email:
Does graphing improve prediction and increase understanding of uncertainty? When making economic forecasts, are scatter plots better than t-statistics, p-values, and other commonly required regression output?
A recent paper by Emre Soyer and Robin Hogarth suggests the answers are yes, that in fact we are far better forecasters when staring at plots of data than we are when dishing out – as academic journals normally do – tables of statistical significance. [Here is a downloadable version of the Soyer-Hogarth article.]
“The Illusion of Predictability: How Regression Statistics Mislead Experts” was published by Soyer and Hogarth in a symposium of the International Journal of Forecasting (vol. 28, no. 3, July 2012). The symposium includes published comments by J. Scott Armstrong, Daniel Goldstein, Keith Ord, N. Nicholas Taleb, and me, together with a reply from Soyer and Hogarth.
Soyer and Hogarth performed an experiment on the forecasting ability of more than 200 well-published econometricians worldwide to test their ability to predict economic outcomes using conventional outputs of linear regression analysis: standard errors, t-statistics, and R-squared.
The chief finding of the Soyer-Hogarth experiment is that the expert econometricians themselves—our best number crunchers—make better predictions when only graphical information—such as a scatter plot and theoretical linear regression line—is provided to them. Give them t-statistics and fits of R-squared for the same data and regression model and their forecasting ability declines. Give them only t-statistics and fits of R-squared and predictions fall from bad to worse.
It’s a finding that hits you between the eyes, or should. R-squared, the primary indicator of model fit, and t-statistic, the primary indicator of coefficient fit, are in the leading journals of economics - such as the AER, QJE, JPE, and RES - evidently doing more harm than good.
Soyer and Hogarth find that conventional presentation mode actually damages inferences from models. This harms decision-making by reducing the econometrician’s (and profit seeker’s) understanding of the total error of the experiment—or of what might be called the real standard error of the regression, where “real” is defined as the sum (in percentage terms, say) of both systematic and random sources of uncertainty in the whole model. If Soyer and Hogarth are correct, academic journals should allocate more space to visual plots of data and less to tables of statistical significance.
In the blogosphere the statistician Andrew Gelman, INET’s Robert Johnson, and journalists Justin Fox (Harvard Business Review) and Felix Salmon (Reuters) have commented favorably on Soyer's and Hogarth's striking results.
But historians of economics and statistics, joined by scientists in other fields – engineering and physics, for example – will not be surprised by the power of visualizing uncertainty. As I explain in my published comment, Karl Pearson himself—a founding father of English-language statistics—tried beginning in the 1890s to make “graphing” the foundation of statistical method. Leading economists of the day such as Francis Edgeworth and Alfred Marshall sympathized strongly with the visual approach.
And as Keynes (1937, QJE) observed, in economics “there is often no scientific basis on which to form any calculable probability whatever. We simply do not know.” Examples of variables we do not know well enough to forecast include, he said, “the obsolescence of a new invention”, “the price of copper” and “the rate of interest twenty years hence” (Keynes, p. 214).
That sounds about right - despite currently fashionable claims about the role of statistical significance in finding a Higgs boson. Unfortunately, Soyer and Hogarth did not include time series forecasting in their novel experiment though in future work I suspect they and others will.
But with extremely powerful, dynamic, and high-dimensional visualization software such as “GGobi” – which works with R and is currently available for free on-line - economists can join engineers and rocket scientists and do a lot more gazing at data than we currently do (http://www.ggobi.org).
At least, that is, if our goal is to improve decisions and to identify relationships that hit us between the eyes.
Stephen T. Ziliak
Professor of Economics
Tuesday, July 10, 2012
Marcus Nunes, I think properly, concludes that Williamson’s graph is wrong, because Williamson ignores the fact that there was a rising trend of NGDP during the 1970s, while during the Great Moderation, NGDP was stationary... Furthermore, Scott Sumner questions whether the application of the Hodrick-Prescott filter to the entire 1947-2011 period was appropriate, given the collapse of NGDP after 2008, thereby distorting estimates of the trend…
First off, I am very cautious about mixing pre- and post-1985 data because of the impact of the Great Moderation on business cylce dynamics. This applies to Jim Hamilton's reply to my thoughts about the positive impact from housing. Hamilton points out that prior to the Great Moderation, housing would make significant contributions to GDP growth as the economy jumped back to trend. True enough; Hamilton might prove correct. But I would add that large contributions prior to 1985 would typically come in the early stages of the business cycle. I don't think the same kinds of cycles are currently at play, and that it is a little late to be expecting a V-shaped boost from housing.
As to the issue of the HP filter, this was on my radar because St. Louis Federal Reserve President James Bullard likes to rely on this technique to support his claim that the US economy is operating near potential. As he said today:
The housing bubble and the ensuing financial crisis probably did some lasting damage to the economy, suggesting that the output gap in the U.S. is not as large as commonly believed and that the growth rate of potential output is modest. This helps explain why U.S. growth continues to be sluggish, why U.S. inflation has remained close to target instead of dropping precipitously and why U.S. unemployment has fallen over the last year—from a level of 9.1 percent in June 2011 to 8.2 percent in June 2012.
I think there is more wrong than right in these two sentences. I don't see how a slower rate of potential growth necessarily implies lower actual growth in the short run. Clearly we have many instances of both above and below trend growth over the years. The failure of inflation to fall further can easily be explained by nominal wage rigidities. And the drop in the unemployment rate, in itself not impressive, should be taken in context with the stagnation of the labor force participation rate.
Bullard likes to rely on this chart as support:
For some reason, Bullard rejects entirely CBO estimates of potential output, which would reveal a smaller output gap then his linear trend decomposition. My version of this chart:
To deal with the endpoint problem, I used a GDP forecast from an ARIMA(1,1,1) model to extend the data beyond 2012:1. If you don't deal with the endpoint problem, you get this:
I believe most people would believe this result (that output is solidly above potential) to be a nonsensical. By itself, the issue of dealing with the endpoint problem should raise red flags about using the HP filter to draw policy conclusions about recent economic dynamics.
Relatedly, notice that the HP filter reveals a period of substantial above trend growth through the middle of 2008. This should be a red flag for Bullard. If he wants to argue that steady inflation now implies that growth is close to potential, he needs to explain why inflation wasn't skyrocketing in 2005. Or 2006. Or 2007. Most importantly, we should have seen the rise in headline inflation confirmed by core-inflation. The record:
Core-inflation remained remarkably well-behave for an economy operating so far above potential, don't you think?
At issue is the tendency of the HP filter to generate revisionist history. Consider the view of the world using data through 2007:4:
Suddenly, the output gap disappears almost entirely in 2005. And 2006. And 2007. Which is much more consistent with the inflation story during that period.
Bottom Line: Use the HP filter with great caution, especially around large shocks. Such shocks will distort your estimates of the underlying trends, both before and after the shock.