Economist's View

On Prediction

This is from Josh Hendrickson at The Everyday Economist:

On Prediction: Suppose that you are a parent of a young child. Every night you give your child a glass of milk with their dinner. When your child is very young, they have a lid on their cup to prevent it from spilling. However, there comes a time when you let them drink without the lid. The absence of a lid presents a possible problem: spilled milk. Initially there is not much you can do to prevent milk from being spilled. However, over time, you begin to notice things that predict when the milk is going to be spilled. For example, certain placements of the cup on the table might make it more likely that the milk is spilled. Similarly, when your child reaches across the table, this also increases the likelihood of spilled milk. The fact that you are able to notice these “risk factors” means that, over time, you will be able to limit the number of times milk is spilled. You begin to move the cup away from troublesome spots before the spill. You institute a rule that the child is not allowed to reach across the table to get something they want. By doing so, the spills become less frequent. You might even get so good at predicting when the milk will spill and preventing it from happening that when it does happen, you and your spouse might argue about it. In fact, with the benefit of hindsight, one of you might say to the other “how did you not see that the milk was going to spill?”

Now suppose that there was an outside observer who studied the spilling of milk at your house. They are tasked with difficult questions: How good are you at successfully predicting when milk is spilled? Were any of your methods to prevent spilling actually successful? ...

An Empirical Turn in Economics Research

From the AEA research highlights:

An empirical turn in economics research: Over the past few decades, economists have increasingly been cited in the press and sought by Congress to give testimony on the issues of the day. This could be due in part to the increasingly empirical nature of economics research.

Aided by internet connections that allow datasets to be assembled from disparate sources and cheap computing power to crunch the numbers, economists are more and more often turning to real-world data to complement and test theoretical models.

This trend was documented in a 2013 article from the Journal of Economic Literature...

In the spirit of empirical inquiry, the authors of a study appearing in the May issue of the American Economic Review: Papers & Proceedings used machine learning techniques to expand this analysis to a much larger set of 135,000 papers published across 80 academic journals cited frequently in the American Economic Review. ...

The ... prevalence of empirical work as determined by the authors’ model has been rising across fields since 1980. The authors note that the empirical turn is not a result of certain more empirical fields overtaking other more theoretical ones, but instead every field becoming more empirically-minded.

Monte Carlo Simulation Basics, I: Historical Notes

Dave Giles:

Monte Carlo Simulation Basics, I: Historical Notes: Monte Carlo (MC) simulation provides us with a very powerful tool for solving all sorts of problems. In classical econometrics, we can use it to explore the properties of the estimators and tests that we use. More specifically, MC methods enable us to mimic (computationally) the sampling distributions of estimators and test statistics in situations that are of interest to us. In Bayesian econometrics we use this tool to actually construct the estimators themselves. I'll put the latter to one side in what follows.

To get the discussion going, suppose that we know that a particular test statistic has a sampling distribution that is Chi Square asymptotically - that is when the sample size is infinitely large - when the null hypothesis is true. That's helpful, up to a point, but it provides us with no hard evidence about that sampling distribution if the sample size that we happen to be faced with is just n = 10 (say). And if we don't know what the sampling distribution is, then we don't know what critical region is appropriate when we apply the test.

In the same vein, we usually avoid using estimators that are are "inconsistent". This implies that our estimators are (among other things) asymptotically unbiased. Again, however, this is no guarantee that they are unbiased, or even have acceptably small bias, if we're working with a relatively small sample of data. If we want to determine the bias (or variance) of an estimator for a particular finite sample size (n), then once again we need to know about the estimator's sampling distribution. Specifically, we need to determine the mean and the variance of that sampling distribution.

And there's no "magic number" above which the sample size can be considered large enough for the asymptotic results to hold. It varies from problem to problem, and often it depends on the unknown parameters that we're trying to draw inferences about. In some situations a sample size of n = 30 will suffice, but in other cases thousands of observations are needed before we can appeal to the asymptotic results. 

If we can't figure the details of the sampling distribution for an estimator or a test statistic by analytical means - and sometimes that can be very, very, difficult - then one way to go forward is to conduct some sort of MC simulation experiment. Indeed, using such simulation techniques hand-in-hand with analytical tools can be very productive when we're exploring new territory.

I'll be devoting several upcoming posts to this important topic. In those posts I'll try and provide an easy-to-follow introduction to MC simulation for econometrics students.

Before proceeding further, let's take a brief look at some of the history of MC simulation. ...

Using Big Data: An Interview with Christian Hansen (Video)

More econometrics: Special Session: Model Selection and Inference (Technical)

101 Boosterism

Paul Krugman:

101 Boosterism: I see that @drvox is writing a big piece on carbon pricing – and agonizing over length and time. I don’t want to step on his forthcoming message, but what he’s said so far helped crystallize something I’ve meant to write about for a while, a phenomenon I’ll call “101 boosterism.”

The name is a takeoff on Noah Smith’s clever writing about “101ism”, in which economics writers present Econ 101 stuff about supply, demand, and how great markets are as gospel, ignoring the many ways in which economists have learned to qualify those conclusions in the face of market imperfections. His point is that while Econ 101 can be a very useful guide, it is sometimes (often) misleading when applied to the real world.

My point is somewhat different: even when Econ 101 is right, that doesn’t always mean that it’s important – certainly not that it’s the most important thing about a situation. In particular, economists may delight in talking about issues where 101 refutes naïve intuition, but that doesn’t at all mean that these are the crucial policy issues we face. ...

He goes on to talk about this in the context of international trade and carbon pricing (too hard to excerpt without leaving out important parts of his discussion).

'Modelling Banking Sector Shocks and Unconventional Policy: New Wine in Old Bottles?'

This is from the B of E's Bank Underground:

Modelling banking sector shocks and unconventional policy: new wine in old bottles?, by James Cloyne, Ryland Thomas, and Alex Tuckett: The financial crisis has thrown up a huge number of empirical challenges for academic and professional economists. The search is on for a framework with a rich enough variety of financial and real variables to examine both the financial shocks that caused the Great Recession and the unconventional policies, such as Quantitative Easing (QE), that were designed to combat it. In a new paper we show how using an older structural econometric modelling approach can be used to provide insights into these questions in ways other models currently cannot. So what are the advantages of going back to an older tradition of modelling? An ongoing issue for central bank economists is that they typically want to look at a wide range of financial sector variables and at a more granular, sector-based level of aggregation than typically found in macroeconomic models with credit and asset market frictions. For example, we often want to distinguish between the credit provided to firms separately from that provided to households or between secured lending and unsecured lending. We may also want to compare and contrast a number of policy instruments that work through different channels such as central bank asset purchases (QE) and macroprudential tools such as countercyclical capital requirements.

It is a tough challenge to incorporate all of these effects in the theoretical and empirical models that are typically used by macroeconomists, such as structural vector autoregression (SVAR) models and micro-founded general equilibrium (DSGE) models. For these reasons turning back to the older tradition of building structural econometric models (SEMs) – built from blocks of simultaneously estimated equations with structural identifying restrictions – can be useful. This approach can be thought of as a blend of the more theory-free VAR methods and a more structural model-based approach. The main advantage of the structural econometric frameworks are that they produce quantitative results at a sector level, which can still be aggregated up to produce a general equilibrium response. They also allow models to be built up in a modular way that allows replacing and improving sets of equations for particular blocks of the model without necessarily undermining the logic of the model as a whole. This older school approach to modelling has begun to appear in a variety of modern vintages. ...

'Making Inferences about Tropics, Germs, and Crops'

Dietrich Vollrath:

Making Inferences about Tropics, Germs, and Crops: One of the long-running arguments in growth is "geography versus institutions". A lot of ink and a tiny amount of computing power was thrown at this question. The early stages of this involved a lot of cross-country regressions that attempted to figure out empirically whether measures of institutions or geography had explanatory power for GDP per capita. (By the way, I'm talking here just about the economic growth literature's take on this. The general question goes back centuries.)

A tweet by Garett Jones recently reminded me of one of the entries in this literature, Bill Easterly and Ross Levine's "Tropics, Germs, and Crops" paper. EL want to assess the role of geography in explaining cross-country incomes per capita. Their main questions are summed up nicely in the abstract.

Does economic development depend on geographic endowments like temperate instead of tropical location, the ecological conditions shaping diseases, or an environment good for grains or certain cash crops? Or do these endowments of tropics, germs, and crops affect economic development only through institutions or policies?

Their conclusion is that geography has no effect, other than through its relationship to institutions. This conclusion, though, doesn't follow from the empirical tests they run. Let's run through the empirics on these questions, and see how to answer them.

I'm going to do this with precisely the dataset that EL use. I don't recall when or where I picked up the data, but I believe it was from Easterly's old website, where he had a nice set of links to datasets. Regardless, it's the right dataset because I can perfectly replicate their results. ...

'The Scandal is What's Legal'

Cecchetti & Schoenholtz:

The Scandal is What's Legal: If you haven’t seen The Big Short, you should. The acting is superb and the story enlightening: a few brilliant outcasts each discover just how big the holes are that eventually bury the U.S. financial system in the crisis of 2007-2009. If you’re like most people we know, you’ll walk away delighted by the movie and disturbed by the reality it captures. ...

But we’re not film critics. The movie—along with some misleading criticism—prompts us to clarify what we view as the prime causes of the financal crisis. The financial corruption depicted in the movie is deeply troubling (we’ve written about fraud and conflicts of interest in finance here and here). But what made the U.S. financial system so fragile a decade ago, and what made the crisis so deep, were practices that were completely legal. The scandal is that we still haven’t addressed these properly.

We can’t “cover” the causes of the crisis in a blog post, but we can briefly explain our top three candidates: (1) insufficient capital and liquidity reflecting poor risk management and incentives; (2) the ability of complex, highly interconnected intermediaries to take on and conceal enormous amounts of risk; and (3) an absurdly byzantine regulatory structure that made it virtually impossible for anyone, however inclined, to understand (let alone manage) the system’s fragilities. ...[long explanationss of each]...

To say that this is a scandal that makes the system less safe is to dramatically understate the case.

Now, we could go on. There are plenty of other problems that policymakers have ignored and are allowing to fester (how about the government-sponsored enterprises?). But we focused on our top three: the need for financial intermediaries to have more capital and liquid assets; the need to improve the ability of both financial market participants and authorities to assess and control risk concentrations through a combination of central clearing and better information collection; and the need to simplify the structure and organization of the U.S. regulatory system itself.

Only if people learn how far the financial system remains from these ideals, only if they understand that the scandal is almost always what is legal, will there be much chance of making the next crisis less severe. ...

'New Method to Solve Significant Variables (Prediction) Conundrum'

Why can't we make good predictions? Maybe we've been doing it wrong:

Scientists devise new method to solve significant variables conundrum, Columbia University: Scientists at Columbia University, the University of California, San Diego (UCSD) and Harvard University have presented an alternative method to address the challenge of using significant variables to make useful predictions in areas such as complex disease.

Shaw-Hwa Lo and Tian Zheng of Columbia, Adeline Lo of UCSD and Herman Chernoff of Harvard present findings in ... Proceedings of the National Academy of Sciences ... that demonstrates that statistically significant variables are not necessarily predictive. In addition, very predictive variables do not necessarily have to appear significant and thereby evade a researcher using statistical significance as a criterion to evaluate variables for prediction.

Statistical significance is a traditional, long-standing measure in any researcher's toolbox but thus far, scientists have been puzzled by the inability to use results of statistically significant variants in complex diseases to make predictions useful for personalized medicine. Why aren't significant variables leading to good prediction of outcomes? This conundrum affects both simple and complex data in a broad range of science and social science fields.

In their findings, the authors demonstrate that what makes variables good for prediction versus significance depends on different properties of the underlying distributions. They suggest that progress in prediction requires efforts toward a new research agenda of searching for a novel criterion to retrieve highly predictive variables rather than highly significant variables.

They also present an alternative approach, the Partition Retention method, which displays strong power in prediction. ...

"What we're saying here is that using the previously very well-known methods might not be appropriate when we care about predicting important outcomes," says Professor Lo. "Our alternative approach seems to do very well in prediction, and is relevant for many scientific fields."

Market Power, Labor's Share of Income, and Measured Productivity

For a long time, I have been making the argument that part of the reason for the inequality problem is distortions in the distribution of income driven by market imperfections such as monopoly power that allows prices to exceed marginal costs. What I didn't realize is that this can also affect measurements of productivity growth:

The relationship between U.S. productivity growth and the decline in the labor share of national income, by Nick Bunker: One of the ongoing debates about the state of the U.S. economy is the extent to which the profits from productivity gains are increasingly going to the owners of capital instead of wage earners. These researchers are debating the extent to which the labor share of income, once considered a constant by economists, is on the decline.

But what if the decline of national income going to labor actually affects the measured rate of U.S. productivity growth? In a blog post published last week, University of Houston economist Dietz Vollrath sketches out a model showing just that scenario. ...

Vollrath argues that businesses with more market power are able to charge higher markups on their goods and services, meaning their pricing is higher than the cost of producing an additional goods or services compared to pricing in a perfectly competitive market. So in this situation where markups are high, goods and services are being produced less efficiently, with the increased profits going to the owners of capital.

Vollrath argues that this is how measured productivity growth is affected by the decline of the labor share of income. Market power is important for thinking about measured productivity growth because, as Vollrath says, it “dictates how efficiently we use our inputs.” ... Impeding the most efficient use of capital and labor via marked-up prices will reduce measured productivity. ... Perhaps this could explain some of the reason why measured productivity growth looks so meager in the seeming age of innovation...

But Vollrath’s story isn’t a complete explanation of the fall in measured productivity, as he acknowledges...

But Vollrath’s market power explanation for falling productivity growth, alongside the falling share of national income going to wage earners, is supported by some evidence.  Work by Massachusetts Institute of Technology graduate student Matt Rognlie, for example, found evidence of higher markups.

Whether and how the decline of the labor share of income affects productivity growth is obviously a topic far too large for a couple of blog posts. But Vollrath’s model is especially interesting for connecting two important trends in recent years: the slowdown in productivity growth and the declining labor share. It’s worth, at the very least, a bit more investigation.

'The Wage Impact of the Marielitos: A Reappraisal'

No sense hiding from evidence that works against my support of immigration. This is from George Borjas (if you are unfamiliar with the Mariel boatlift, see here):

The Wage Impact of the Marielitos: A Reappraisal, by George J. Borjas, NBER Working Paper No. 21588 [open link]: This paper brings a new perspective to the analysis of the Mariel supply shock, revisiting the question and the data armed with the accumulated insights from the vast literature on the economic impact of immigration. A crucial lesson from this literature is that any credible attempt to measure the wage impact of immigration must carefully match the skills of the immigrants with those of the pre-existing workforce. The Marielitos were disproportionately low-skill; at least 60 percent were high school dropouts. A reappraisal of the Mariel evidence, specifically examining the evolution of wages in the low-skill group most likely to be affected, quickly overturns the finding that Mariel did not affect Miami’s wage structure. The absolute wage of high school dropouts in Miami dropped dramatically, as did the wage of high school dropouts relative to that of either high school graduates or college graduates. The drop in the relative wage of the least educated Miamians was substantial (10 to 30 percent), implying an elasticity of wages with respect to the number of workers between -0.5 and -1.5. In fact, comparing the magnitude of the steep post-Mariel drop in the low-skill wage in Miami with that observed in all other metropolitan areas over an equivalent time span between 1977 and 2001 reveals that the change in the Miami wage structure was a very unusual event. The analysis also documents the sensitivity of the estimated wage impact to the choice of a placebo. The measured impact is much smaller when the placebo consists of cities where pre-Mariel employment growth was weak relative to Miami.

Video: Sir David Hendry on Big Data and Econometrics

'Inequality of Opportunity: Useful Policy Construct or Will o’ the Wisp?'

Can economic opportunity be separated from economic outcomes?:

Inequality of opportunity as a policy construct: ...Concluding remarks

Any attempt to separate circumstances from effort – to identify that portion of the inequality of outcomes which is a legitimate target for redistribution – is fraught with empirical and conceptual difficulties.[4] Fine-grained distinctions between inequality of opportunity and inequality of outcomes do not hold water in practice, and we are likely to greatly underestimate inequality of opportunity and hence the need for intervention.

Further, what if one person’s effort becomes another person’s circumstance, as when income generated through parents’ effort provides a better start in life for some children? Or when freely made choices by one group of upper-income house buyers push up prices for others who may have lower incomes? Is it legitimate or is it not legitimate to intervene in this case?

These arguments support the case for generalised social protection in dimensions such as income, health and education, irrespective of whether the outcomes can be specifically attributed to circumstance or to effort.

The important questions then relate to what the best available policy instruments are for delivering this social protection, what effects they have on incentives, and how best they can be deployed. To be sure, we may make some Type I and Type II errors in doing so; we may penalize effort when we should not, and we may not fully compensate for circumstances when we should. But this is preferable to being frozen into perpetually underestimating the need for intervention by a focus on that will o’ the wisp, inequality of opportunity.

'A Crisis at the Edge of Physics'

Seems like much the same can be said about modern macroeconomics (except perhaps the "given the field its credibility" part):

A Crisis at the Edge of Physics, by Adam Frank and Marcelo Gleiser, NY Times: Do physicists need empirical evidence to confirm their theories?

You may think that the answer is an obvious yes, experimental confirmation being the very heart of science. But a growing controversy at the frontiers of physics and cosmology suggests that the situation is not so simple.

A few months ago in the journal Nature, two leading researchers, George Ellis and Joseph Silk, published a controversial piece called “Scientific Method: Defend the Integrity of Physics.” They criticized a newfound willingness among some scientists to explicitly set aside the need for experimental confirmation of today’s most ambitious cosmic theories — so long as those theories are “sufficiently elegant and explanatory.” Despite working at the cutting edge of knowledge, such scientists are, for Professors Ellis and Silk, “breaking with centuries of philosophical tradition of defining scientific knowledge as empirical.”

Whether or not you agree with them, the professors have identified a mounting concern in fundamental physics: Today, our most ambitious science can seem at odds with the empirical methodology that has historically given the field its credibility. ...

No Slow Down in Global Warming

NOAA Study Confirms Global Warming Speed-Up Is Imminent

Forecasting in Economic Crises

Maurizio Bovi summarizes a new published paper he wrote with Roy Cerqueti. The paper examines lay agents' forecasts amid great recessions (with a special focus on Greece). The paper is Bovi, M. and R. Cerqueti (2015) "Forecasting macroeconomic fundamentals in economic crises" Annals of Operations Research DOI 10.1007/s10479-015-1879-4:

Forecasting in Economic Crises: Expectations are a key factor in economics and heterogeneous forecasts are a fact of life. Just to mention, there are quite significant and well-known incentives to become a sport champion or to win a Nobel Prize, yet very few persons succeed in the endeavor. The brutal truth is that the majority lags behind or gives up - heterogeneity is the rule, not the exception. By the same token lay forecasters may learn, but it is unrealistic to think that all of them—even in the long run—will achieve Muth-optimal and, hence, homogeneous forecasts. The situation is made even more complex, and more interesting to study, when the fundamental to predict, the real GDP growth rate, is well below zero and highly volatile.

In recent work (Bovi and Cerqueti, 2015) we address the topic of heterogeneous forecasting performances amid deep recessions. Lay agents are assumed to have different predictive ability in that they have equal loss functions, but different asymmetry parameters that are used as control to minimize their forecasting errors. Simulating poor performing economies populated by three groups of forecasters, we have obtained the following results.

The less sophisticated forecasters in our setting – the “medians” (using passive rule-of-thumb) - never perform as the best predictors – the “muthians” – whereas “second best” (SB) agents (acting as attentive econometricians) do that only occasionally. This regardless the size of the crisis. Thus, as in the real world, in our artificial economy heterogeneity is a structural trait. More intriguingly, simulations also show that the medians’ behavior tend to be relatively smoother than that of SB agents, and that the difference between them widens in the case very serious crises. In particular, great recessions make SB agents’ predictions relatively more biased. An explanation is that dramatic crises extend the available information set (e.g., due to greater mass media coverage), and this leads SB agents, who are more prompt to revise their forecasts than medians.

Our results are somewhat in line with Simon’s famous statement about the fact that more information does not necessarily mean better forecasting performances. Furthermore, our outcomes shed some light on what has been happening in the freak macroeconomic expectations in Greece these years. The current crisis, in fact, may be thought of as a sort of natural experiment to understand how lay decision makers react to very dramatic years. In particular, due to its terrible recent downturn, Greece is one of the most suitable cases, raising the following question: How do Greeks perceive their own personal financial situation with respect to that of their country? Clearly, the representative citizen cannot by definition systematically drift apart from that of the country where she lives, given that the nation-wide economic situation is the (weighted) sum of the individual ones in the country. Yet, it may be hard to remain objective in the course of very deep and prolonged economic crises. The evidence depicted in the following graph looks rather suggestive of the effects of deep recessions on the rationality of people’s expectations, something that conform with our findings.

[Click on figure to enlarge]

'Any P-Value Distinguishable from Zero is Insufficiently Informative'

From the blog Three-Toed Sloth by Cosma Shalizi (this also appeared in yesterday's links):

Any P-Value Distinguishable from Zero is Insufficiently Informative: After ten years of teaching statistics, I feel pretty confident in saying that one of the hardest points to get through to undergrads is what "statistically significant" actually means. (The word doesn't help; "statistically detectable" or "statistically discernible" might've been better.) They have a persistent tendency to think that parameters which are significantly different from 0 matter, that ones which are insignificantly different from 0 don't matter, and that the smaller the p-value, the more important the parameter. Similarly, if one parameter is "significantly" larger than another, then they'll say the difference between them matters, but if not, not. If this was just about undergrads, I'd grumble over a beer with my colleagues and otherwise suck it up, but reading and refereeing for non-statistics journals shows me that many scientists in many fields are subject to exactly the same confusions as The Kids, and talking with friends in industry makes it plain that the same thing happens outside academia, even to "data scientists". ... To be fair, one meets some statisticians who succumb to these confusions.

One reason for this, I think, is that we fail to teach well how, with enough data, any non-zero parameter or difference becomes statistically significant at arbitrarily small levels. The proverbial expression of this, due I believe to Andy Gelman, is that "the p-value is a measure of sample size". More exactly, a p-value generally runs together the size of the parameter, how well we can estimate the parameter, and the sample size. The p-value reflects how much information the data has about the parameter, and we can think of "information" as the product of sample size and precision (in the sense of inverse variance) of estimation, say n/σ². In some cases, this heuristic is actually exactly right, and what I just called "information" really is the Fisher information.

Rather than working on grant proposals Egged on by a friend As a public service, I've written up some notes on this... [The mathematics comes next.]

Video: On Econometrics - Koen Jochmans & Mark Thoma - RES 2015

Koen Jochmans of Sciences Po speaks to Mark Thoma about his research and winning the Sargan Prize for outstanding research in the Econometric Journal. http://www.res.org.uk/details/econome...

The interview was recorded at the Royal Economic Society annual conference at The University of Manchester in April 2015 and produced by Econ Films.

[This is the first of three interviews I did -- will save the best for last.]

'Unit Roots, Redux'

John Cochrane weighs in on the discussion of unit roots:

Unit roots, redux: Arnold Kling's askblog and Roger Farmer have a little exchange on GDP and unit roots. My two cents here.

I did a lot of work on this topic a long time ago, in How Big is the Random Walk in GNP?  (the first one)  Permanent and Transitory Components of GNP and Stock Prices” (The last, and I think best one) "Multivariate estimates" with Argia Sbordone, and "A critique of the application of unit root tests", particularly appropriate to Roger's battery of tests.

The conclusions, which I still think hold up today:

Log GDP has both random walk and stationary components. Consumption is a pretty good indicator of the random walk component. This is also what the standard stochastic growth model predicts: a random walk technology shock induces a random walk component in output but there are transitory dynamics around that value.

A linear trend in GDP is only visible ex-post, like a "bull" or "bear" market.  It's not "wrong" to detrend GDP, but it is wrong to forecast that GDP will return to the linear trend or to take too seriously correlations of linearly detrended series, as Arnold mentions. Treating macro series as cointegrated with one common trend is a better idea.

Log stock prices have random walk and stationary components. Dividends are a pretty good indicator of the random walk component. (Most recently, here.) ...

Both Arnold and Roger claim that unemployment has a unit root. Guys, you must be kidding. ...

He goes on to explain.

Varian: Causal Inference in Social Science An elementary introduction

Some of you may be interested in this:

Causal Inference in Social Science An elementary introduction, by Hal R. Varian, Google, Inc, Jan 2015, Revised: March 7, 2015: Abstract This is a short and very elementary introduction to causal inference in social science applications targeted to machine learners. I illustrate the techniques described with examples chosen from the economics and marketing literature.

'Economists' Biggest Failure'

Noah Smith:

Economists' Biggest Failure: One of the biggest things that economists get grief about is their failure to predict big events like recessions. ... 

Pointing this out usually leads to the eternal (and eternally fun) debate over whether economics is a real science. The profession's detractors say that if you don’t make successful predictions, you aren’t a science. Economists will respond that seismologists can’t forecast earthquakes, and meteorologists can’t forecast hurricanes, and who cares what’s really a “science” anyway. 

The debate, however, misses the point. Forecasts aren’t the only kind of predictions a science can make. In fact, they’re not even the most important kind. 

Take physics for example. Sometimes physicists do make forecasts -- for example, eclipses. But those are the exception. Usually, when you make a new physics theory, you use it to predict some new phenomenon... For example, quantum mechanics has gained a lot of support from predicting the strange new things like quantum tunneling or quantum teleportation.

Other times, a theory will predict things we have seen before, but will describe them in terms of other things that we thought were totally separate, unrelated phenomena. This is called unification, and it’s a key part of what philosophers think science does. For example, the theory of electromagnetism says that light, electric current, magnetism, radio waves are all really the same phenomenon. Pretty neat! ...

So that’s physics. What about economics? Actually, econ has a number of these successes too. When Dan McFadden used his Random Utility Model to predict how many people would ride San Francisco's Bay Area Rapid Transit system,... he got it right. And he got many other things right with the same theory -- it wasn’t developed to explain only train ridership. 

Unfortunately, though, this kind of success isn't very highly regarded in the economics world... Maybe now, with the ascendance of empirical economics and a decline in theory, we’ll see a focus on producing fewer but better theories, more unification, and more attempts to make novel predictions. Someday, maybe macroeconomists will even be able to make forecasts! But let’s not get our hopes up.

I've addressed this question many times, e.g. in 2009, and to me the distinction is between forecasting the future, and understanding why certain phenomena occur (re-reading, it's a bit repetitive):

Are Macroeconomic Models Useful?: There has been no shortage of effort devoted to predicting earthquakes, yet we still can't see them coming far enough in advance to move people to safety. When a big earthquake hits, it is a surprise. We may be able to look at the data after the fact and see that certain stresses were building, so it looks like we should have known an earthquake was going to occur at any moment, but these sorts of retrospective analyses have not allowed us to predict the next one. The exact timing and location is always a surprise.

Does that mean that science has failed? Should we criticize the models as useless?

No. There are two uses of models. One is to understand how the world works, another is to make predictions about the future. We may never be able to predict earthquakes far enough in advance and with enough specificity to allow us time to move to safety before they occur, but that doesn't prevent us from understanding the science underlying earthquakes. Perhaps as our understanding increases prediction will be possible, and for that reason scientists shouldn't give up trying to improve their models, but for now we simply cannot predict the arrival of earthquakes.

However, even though earthquakes cannot be predicted, at least not yet, it would be wrong to conclude that science has nothing to offer. First, understanding how earthquakes occur can help us design buildings and make other changes to limit the damage even if we don't know exactly when an earthquake will occur. Second, if an earthquake happens and, despite our best efforts to insulate against it there are still substantial consequences, science can help us to offset and limit the damage. To name just one example, the science surrounding disease transmission helps use to avoid contaminated water supplies after a disaster, something that often compounds tragedy when this science is not available. But there are lots of other things we can do as well, including using the models to determine where help is most needed.

So even if we cannot predict earthquakes, and we can't, the models are still useful for understanding how earthquakes happen. This understanding is valuable because it helps us to prepare for disasters in advance, and to determine policies that will minimize their impact after they happen.

All of this can be applied to macroeconomics. Whether or not we should have predicted the financial earthquake is a question that has been debated extensively, so I am going to set that aside. One side says financial market price changes, like earthquakes, are inherently unpredictable -- we will never predict them no matter how good our models get (the efficient markets types). The other side says the stresses that were building were obvious. Like the stresses that build when tectonic plates moving in opposite directions rub against each other, it was only a question of when, not if. (But even when increasing stress between two plates is observable, scientists cannot tell you for sure if a series of small earthquakes will relieve the stress and do little harm, or if there will be one big adjustment that relieves the stress all at once. With respect to the financial crisis, economists expected lots of little, small harm causing adjustments, instead we got the "big one," and the "buildings and other structures" we thought could withstand the shock all came crumbling down. On prediction in economics, perhaps someday improved models will allow us to do better than we have so far at predicting the exact timing of crises, and I think that earthquakes provide some guidance here. You have to ask first if stress is building in a particular sector, and then ask if action needs to be taken because the stress has reached dangerous levels, levels that might result in a big crash rather than a series of small stress relieving adjustments. I don't think our models are very good at detecting accumulating stress...

Whether the financial crisis should have been predicted or not, the fact that it wasn't predicted does not mean that macroeconomic models are useless any more than the failure to predict earthquakes implies that earthquake science is useless. As with earthquakes, even when prediction is not possible (or missed), the models can still help us to understand how these shocks occur. That understanding is useful for getting ready for the next shock, or even preventing it, and for minimizing the consequences of shocks that do occur. 

But we have done much better at dealing with the consequences of unexpected shocks ex-post than we have at getting ready for these a priori. Our equivalent of getting buildings ready for an earthquake before it happens is to use changes in institutions and regulations to insulate the financial sector and the larger economy from the negative consequences of financial and other shocks. Here I think economists made mistakes - our "buildings" were not strong enough to withstand the earthquake that hit. We could argue that the shock was so big that no amount of reasonable advance preparation would have stopped the "building" from collapsing, but I think it's more the case that enough time has passed since the last big financial earthquake that we forgot what we needed to do. We allowed new buildings to be constructed without the proper safeguards.

However, that doesn't mean the models themselves were useless. The models were there and could have provided guidance, but the implied "building codes" were ignored. Greenspan and others assumed no private builder would ever construct a building that couldn't withstand an earthquake, the market would force them to take this into consideration. But they were wrong about that, and even Greenspan now admits that government building codes are necessary. It wasn't the models, it was how they were used (or rather not used) that prevented us from putting safeguards into place.

We haven't failed at this entirely though. For example, we have had some success at putting safeguards into place before shocks occur, automatic stabilizers have done a lot to insulate against the negative consequences of the recession (though they could have been larger to stop the building from swaying as much as it has). So it's not proper to say that our models have not helped us to prepare in advance at all, the insulation social insurance programs provide is extremely important to recognize. But it is the case that we could have and should have done better at preparing before the shock hit.

I'd argue that our most successful use of models has been in cleaning up after shocks rather than predicting, preventing, or insulating against them through pre-crisis preparation. When despite our best effort to prevent it or to minimize its impact a priori, we get a recession anyway, we can use our models as a guide to monetary, fiscal, and other policies that help to reduce the consequences of the shock (this is the equivalent of, after a disaster hits, making sure that the water is safe to drink, people have food to eat, there is a plan for rebuilding quickly and efficiently, etc.). As noted above, we haven't done a very good job at predicting big crises, and we could have done a much better job at implementing regulatory and institutional changes that prevent or limit the impact of shocks. But we do a pretty good job of stepping in with policy actions that minimize the impact of shocks after they occur. This recession was bad, but it wasn't another Great Depression like it might have been without policy intervention.

Whether or not we will ever be able to predict recessions reliably, it's important to recognize that our models still provide considerable guidance for actions we can take before and after large shocks that minimize their impact and maybe even prevent them altogether (though we will have to do a better job of listening to what the models have to say). Prediction is important, but it's not the only use of models.

'Did the Keynesians Get It Wrong in Predicting a Recession in 2013?'

Dean Baker:

Did the Keynesians Get It Wrong in Predicting a Recession in 2013?:  I have had several readers send me a blogpost from Scott Sumner saying that the Keynesians have been dishonest in not owning up to the fact that they were wrong in predicting a recession in 2013. The argument is that supposedly us Keynesian types all said that the budget cuts and the ending of the payroll tax cut at the start of 2013 would throw the economy back into recession. (Jeffrey Sachs has made similar claims.)

That isn't my memory of what I said at the time, but hey we can check these things. I looked at a few of my columns from the fall of 2012 and they mostly ran in the opposite direction. The Washington insider types were hyping the threat of the "fiscal cliff" in the hope of pressuring President Obama and the Democrats to make big concessions on Social Security and Medicare. They were saying that even the risk of falling off the cliff could have a big impact on growth in the third and fourth quarter of 2012.

My columns and blogposts (e.g. here, here, here, here, and here) were largely devoted to saying this was crap. I certainly agreed that budget cutbacks and the end of the payroll tax cuts would dampen growth, but the number was between 0.5-0.8 percentage points. This left us far from recession. (All my columns and blogposts from this time are at the CEPR website, so folks can verify that I didn't do any cherry picking here.)

I know Paul Krugman is the real target here, not me, but we've been seeing the economy pretty much the same way since the beginning of the recession. If he had a different story at the time I think I would remember it. But his columns and blogposts are archived too. I really don't think anyone will find him predicting a recession in 2013, although I'm sure he also said that budget cuts and tax increases would dampen growth. 

Anyhow, I'm generally happy to stand behind the things I've said, and when they are proven wrong I hope I own up to it. But I don't see any apologies in order. No recession happened in 2013 and none was predicted here.

I don't recall predicting a recession either (the "they" intended to tar all Keynesians refers to just a few people), just that it would be a drag on growth (the CBO predicted 0.6%). In any case, not much can be said unless one takes the time to estimate a model, use it as a baseline, and then ask the model how the economy would have done in an alternative world where policy was different. Just because we still had growth after the budget cuts does not prove or disprove anything. Even if growth rises under austerity, you can't say it would not have risen a bit more more without austerity (all else is far from equal) unless you have done the hard work of estimating a defensible model and then asking it these questions. Similarly, you can't say much about the degree of monetary offset unless you have taken the time to do the econometrics to support it. But with changes this small -- the impact was predicted to be much less than one percent of growth by most models -- it is very hard to get statistically significant differences in any case.

The problem is that there is no model that all economists agree is "best" for these purposes, and the answer one gets depends upon the particular choice of models. Choose a model that delivers small fiscal multipliers and you get one answer, use a model with bigger multipliers and the answer changes. But even the models with the largest multipliers did not predict a recession, only a drag on growth (generally less than one percent) so the fact that we still had growth says nothing about the impact of the policy, or the degree of monetary offset.

'Here's What Economics Gets Right'

I'm pretty sure my dissertation advisor (Greg Duncan, a McFadden student at Berkeley) helped to create the model used to make the BART prediction discussed below (I think they used a program called QUAIL, and precursor to LIMDEP):

Here's What Economics Gets Right, by  Noah Smith: Criticizing economics for not being scientific enough is a crime of which many of us -- I’ve done it -- are guilty.  But there’s a right way to do it and a wrong way to do it.  Alex Rosenberg and Tyler Curtain, writing in the New York Times, have done it the wrong way.

Here are Rosenberg and Curtain:

Over time, the question of why economics has not (yet) qualified as a science has become an obsession among theorists, including philosophers of science like us...The trouble with economics is that it lacks the most important of science’s characteristics — a record of improvement in predictive range and accuracy...In fact, when it comes to economic theory’s track record, there isn’t much predictive success to speak of at all.

Economics doesn’t have predictive success, eh? This is something a lot of people claim, but once you look beyond the well-publicized fact that economists can’t predict recessions, you can see that the claim just isn’t true. Economics can predict plenty of things.

My favorite example is the story of Daniel McFadden and the BART...

'Forecasting Inflation with Fundamentals . . . It’s Hard!'

It's a mystery why inflation is such a mystery:

Forecasting Inflation with Fundamentals . . . It’s Hard!, by Jan Groen, Liberty Street Economics, NY Fed: Controlling inflation is at the core of monetary policymaking, and central bankers would like to have access to reliable inflation forecasts to assess their progress in achieving this goal. Producing accurate inflation forecasts, however, turns out not to be a trivial exercise. This posts reviews the key challenges in inflation forecasting and discusses some recent developments that attempt to deal with these challenges. ...

'Persuasion with Statistics'

Chris Dillow:

Persuasion with statistics: Mark Thoma's point that apparently strong econometric results are often the product of specification mining prompts Lars Syll to remind us that eminent economists have long been wary of what econometrics can achieve.

I doubt if many people have ever thought "Crikey, the t stats are high here. That means I must abandon my long-held beliefs about an important matter." More likely, the reaction is to recall Dave Giles' commandments nine and 10. (Apparently?) impressive econometric findings might be good enough to get you published. But there's a big difference between being published and being read, let alone being persuasive.

This poses a question: how, then, do statistics persuade people to change their general beliefs (as distinct from beliefs about single facts)?

Let me take an example of an issue where I've done just this. I used to believe in the efficient market hypothesis. ...[explains why that changed]...

He concludes with:

The above is not a story about statistical significance (pdf). Single studies are rarely persuasive. Instead, the process of persuading people to change their mind requires diversity - a diversity of data sets, and a diversity of theories.  Am I wrong? Feel free to offer counter-examples.

'Those Whom a God Wishes to Destroy He First Drives Mad'

Brad DeLong wonders why Cliff Asness in clinging to a theoretical model that has clearly been rejected by the data:

... What is not normal is to claim that your analysis back in 2010 that quantitative easing was generating major risks of inflation was dead-on.

What is not normal is to adopt the mental pose that your version of classical austerian economics cannot fail--that it can only be failed by an uncooperative and misbehaving world.

What is not normal is, after 4 1/2 years, in a week, a month, a six-month period in which market expectations of long-run future inflation continue on a downward trajectory, to refuse to mark your beliefs to market and demand that the market mark its beliefs to you. To still refuse to bring your mind into agreement with reality and demand that reality bring itself into agreement with your mind. To still refuse to say: "my intellectual adversaries back in 2010 had a definite point" and to say only: "IT'S NOT OVER YET!!!!"

There's a version of this in econometrics, i.e. you know the model is correct, you are just having trouble finding evidence for it. It goes as follows. You are testing a theory you came up with, but the data are uncooperative and say you are wrong. But instead of accepting that, you tell yourself "My theory is right, I just haven't found the right econometric specification yet. I need to add variables, remove variables, take a log, add an interaction, square a term, do a different correction for misspecification, try a different sample period, etc., etc., etc." Then, after finally digging out that one specification of the econometric model that confirms your hypothesis, you declare victory, write it up, and send it off (somehow never mentioning the intense specification mining that produced the result).

Too much econometric work proceeds along these lines. Not quite this blatantly, but that is, in effect, what happens in too many cases. I think it is often best to think of econometric results as the best case the researcher could make for a particular theory rather than a true test of the model.

'Why Are Economic Forecasts Wrong So Often?'

At MoneyWatch:

Why are economic forecasts wrong so often?: The Queen of England famously asked why economists failed to foresee the financial crisis in 2008. "Why did nobody notice it?" was her question when she visited the London School of Economics that year.

Economists' failure to accurately predict the economy's course isn't limited to the financial crisis and the Great Recession that followed. Macroeconomic computer models also aren't very useful for predicting how variables such as GDP, employment, interest rates and inflation will evolve over time.

Forecasting most things is fraught with difficulty. See the current dust-up between Nate Silver and Sam Wang over their conflicting predictions about the coming Senate elections. Why is forecasting so hard?

Because so many things can go wrong. For example...

'Forecasting with the FRBNY DSGE Model'

The NY Fed hopes that someday the FRBNY DSGE model will be useful for forecasting. Presently, the model has "huge margins for improvement. The list of flaws is long..." (first in a five-part series):

Forecasting with the FRBNY DSGE Model, by Marco Del Negro, Bianca De Paoli, Stefano Eusepi, Marc Giannoni, Argia Sbordone, and Andrea Tambalotti, Liberty Economics, FRBNY: The Federal Reserve Bank of New York (FRBNY) has built a DSGE model as part of its efforts to forecast the U.S. economy. On Liberty Street Economics, we are publishing a weeklong series to provide some background on the model and its use for policy analysis and forecasting, as well as its forecasting performance. In this post, we briefly discuss what DSGE models are, explain their usefulness as a forecasting tool, and preview the forthcoming pieces in this series.

The term DSGE, which stands for dynamic stochastic general equilibrium, encompasses a very broad class of macro models, from the standard real business cycle (RBC) model of Nobel prizewinners Kydland and Prescott to New Keynesian monetary models like the one of Christiano, Eichenbaum, and Evans. What distinguishes these models is that rules describing how economic agents behave are obtained by solving intertemporal optimization problems, given assumptions about the underlying environment, including the prevailing fiscal and monetary policy regime. One of the benefits of DSGE models is that they can deliver a lens for understanding the economy’s behavior. The third post in this series will show an example of this role with a discussion of the forces behind the Great Recession and the following slow recovery.

DSGE models are also quite abstract representations of reality, however, which in the past severely limited their empirical appeal and forecasting performance. This started to change with work by Schorfheide and Smets and Wouters. First, they popularized estimation (especially Bayesian estimation) of these models, with parameters chosen in a way that increased the ability of these models to describe the time series behavior of economic variables. Second, these models were enriched with both endogenous and exogenous propagation mechanisms that allowed them to better capture patterns in the data. For this reason, estimated DSGE models are increasingly used within the Federal Reserve System (the Board of Governors and the Reserve Banks of Chicago and Philadelphia have versions) and by central banks around the world (including the New Area-Wide Model developed at the European Central Bank, and models at the Norges Bank and the Sveriges Riksbank). The FRBNY DSGE model is a medium-scale model in the tradition of Christiano, Eichenbaum, and Evans and Smets and Wouters that also includes credit frictions as in the financial accelerator model developed by Bernanke, Gertler, and Gilchrist and further investigated by Christiano, Motto, and Rostagno. The second post in this series elaborates on what DSGE models are and discusses the features of the FRBNY model.

Perhaps some progress was made in the past twenty years toward empirical fit, but is it enough to give forecasts from DSGE models any credence? Aren’t there many critics out there (here is one) telling us these models are a failure? As it happens, not many people seem to have actually checked the extent to which these model forecasts are off the mark. Del Negro and Schorfheide do undertake such an exercise in a chapter of the recent Handbook of Economic Forecasting. Their analysis compares the real-time forecast accuracy of DSGE models that were available prior to the Great Recession (such as the Smets and Wouters model) to that of the Blue Chip consensus forecasts, using a period that includes the Great Recession. They find that, for nowcasting (forecasting current quarter variables) and short-run forecasting, DSGE models are at a disadvantage compared with professional forecasts. Over the medium- and long-run terms, however, DSGE model forecasts for both output and inflation become competitive with—if not superior to—professional forecasts. They also find that including timely information from financial markets such as credit spreads can dramatically improve the models’ forecasts, especially in the Great Recession period.

These results are based on what forecasters call “pseudo-out-of-sample” forecasts. These are not truly “real time” forecasts, because they were not produced at the time. (To our knowledge, there is little record of truly real time DSGE forecasts for the United States, partly because these models were only developed in the mid-2000s.) For this reason, in the fourth post of this series, we report forecasts produced in real time using the FRBNY DSGE model since 2010. These forecasts have been included in internal New York Fed documents, but were not previously made public. Although the sample is admittedly short, these forecasts show that while consensus forecasts were predicting a relatively rapid recovery from the Great Recession, the DSGE model was correctly forecasting a more sluggish recovery.

The last post in the series shows the current FRBNY DSGE forecasts for output growth and inflation and discusses the main economic forces driving the predictions. Bear in mind that these forecasts are not the official New York Fed staff forecasts; the DSGE model is only one of many tools employed for prediction and policy analysis at the Bank.

DSGE models in general and the FRBNY model in particular have huge margins for improvement. The list of flaws is long, ranging from the lack of heterogeneity (the models assume a representative household) to the crude representation of financial markets (the models have no term premia). Nevertheless, we are sticking our necks out and showing our forecasts, not because we think we have a “good” model of the economy, but because we want to have a public record of the model’s successes and failures. In doing so, we can learn from both our past performance and readers’ criticism. The model is a work in progress. Hopefully, it can be improved over time, guided by current economic and policy questions and benefiting from developments in economic theory and econometric tools.

'Stupidest Article Ever Published?'

Via email, I was asked if this is the "stupidest article ever published?":

The Inflation-Debt Scam

If not, it's certainly in the running.

Econometrics Mini-Course (NBER): Theory and Application of Network Models

Summer Institute 2014 Theory and Application of Network Models, July 22, 2014 Daron Acemoglu and Matthew O. Jackson, Organizers:

Matthew O. Jackson, Stanford University Social and Economic Networks: Backgound

Slides accompany this lecture. Click here to view them.

Daron Acemoglu, MIT Networks: Games over Networks and Peer Effects

Slides accompany this lecture. Click here to view them.

Matthew O. Jackson, Stanford University Diffusion, Identification, Network Formation

Slides accompany this lecture. Click here to view them.

Daron Acemoglu, MIT Networks: Propagation of Shocks over Economic Networks

Slides accompany this lecture. Click here to view them.

A Conversation with Lars Hansen

See also conversations with Peter Diamond and Edmund Phelps.

'Data Mining Can be a Productive Activity'

Jennifer Castle and David Hendry on data mining

‘Data mining’ with more variables than observations: While ‘fool’s gold’ (iron pyrites) can be found by mining, most mining is a productive activity. Similarly, when properly conducted, so-called ‘data mining’ is no exception –despite many claims to the contrary. Early criticisms, such as the review of Tinbergen (1940) by Friedman (1940) for selecting his equations “because they yield high coefficients of correlation”, and by Lovell (1983) and Denton (1985) of data mining based on choosing ‘best fitting’ regressions, were clearly correct. It is also possible to undertake what Gilbert (1986) called ‘strong data mining’, whereby an investigator tries hundreds of empirical estimations, and reports the one she or he ‘prefers’ – even when such results are contradicted by others that were found. As Leamer (1983) expressed the matter: “The econometric art as it is practiced at the computer terminal involves fitting many, perhaps thousands, of statistical models. One or several that the researcher finds pleasing are selected for reporting purposes”. That an activity can be done badly does not entail that all approaches are bad, as stressed by Hoover and Perez (1999), Campos and Ericsson (1999), and Spanos (2000) – driving with your eyes closed is a bad idea, but most car journeys are safe.

Why is ‘data mining’ needed?

Econometric models need to handle many complexities if they are to have any hope of approximating the real world. There are many potentially relevant variables, dynamics, outliers, shifts, and non-linearities that characterise the data generating process. All of these must be modelled jointly to build a coherent empirical economic model, necessitating some form of data mining – see the approach described in Castle et al. (2011) and extensively analysed in Hendry and Doornik (2014).

Any omitted substantive feature will result in erroneous conclusions, as other aspects of the model attempt to proxy the missing information. At first sight, allowing for all these aspects jointly seems intractable, especially with more candidate variables (denoted N) than observations (T denotes the sample size). But help is at hand with the power of a computer. ...[gives technical details]...

Conclusions

Appropriately conducted, data mining can be a productive activity even with more candidate variables than observations. Omitting substantively relevant effects leads to mis-specified models, distorting inference, which large initial specifications should mitigate. Automatic model selection algorithms like Autometrics offer a viable approach to tackling more candidate variables than observations, controlling spurious significance.

'On Macroeconomic Forecasting'

Simon Wren-Lewis:

...The rather boring truth is that it is entirely predictable that forecasters will miss major recessions, just as it is equally predictable that each time this happens we get hundreds of articles written asking what has gone wrong with macro forecasting. The answer is always the same - nothing. Macroeconomic model based forecasts are always bad, but probably no worse than intelligent guesses.

More here.

'Presidents and the U.S. Economy: An Econometric Exploration'

Alan Blinder and Mark Watson:

Presidents and the U.S. Economy: An Econometric Exploration, by Alan S. Blinder and Mark W. Watson, NBER Working Paper No. 20324 [open link]: The U.S. economy has grown faster—and scored higher on many other macroeconomic metrics—when the President of the United States is a Democrat rather than a Republican. For many measures, including real GDP growth (on which we concentrate), the performance gap is both large and statistically significant, despite the fact that postwar history includes only 16 complete presidential terms. This paper asks why. The answer is not found in technical time series matters (such as differential trends or mean reversion), nor in systematically more expansionary monetary or fiscal policy under Democrats. Rather, it appears that the Democratic edge stems mainly from more benign oil shocks, superior TFP performance, a more favorable international environment, and perhaps more optimistic consumer expectations about the near-term future. Many other potential explanations are examined but fail to explain the partisan growth gap.

'Further Thoughts on Phillips Curves'

Simon Wren-Lewis:

Further thoughts on Phillips curves: In a post from a few days ago I looked at some recent evidence on Phillips curves, treating the Great Recession as a test case. I cast the discussion as a debate between rational and adaptive expectations. Neither is likely to be 100% right of course, but I suggested the evidence implied rational expectations were more right than adaptive. In this post I want to relate this to some other people’s work and discussion. (See also this post from Mark Thoma.) ...

The first issue is why look at just half a dozen years, in only a few countries. As I noted in the original post, when looking at CPI inflation there are many short term factors that may mislead. Another reason for excluding European countries which I did not mention is the impact of austerity driven higher VAT rates (and other similar taxes or administered prices), nicely documented by Klitgaard and Peck. Surely all this ‘noise’ is an excellent reason to look over a much longer time horizon?

One answer is given in this recent JEL paper by Mavroeidis, Plagborg-Møller and Stock. As Plagborg-Moller notes in an email to Mark Thoma: “Our meta-analysis finds that essentially any desired parameter estimates can be generated by some reasonable-sounding specification. That is, estimation of the NKPC is subject to enormous specification uncertainty. This is consistent with the range of estimates reported in the literature….traditional aggregate time series analysis is just not very informative about the nature of inflation dynamics.” This had been my reading based on work I’d seen.

This is often going to be the case with time series econometrics, particularly when key variables appear in the form of expectations. Faced with this, what economists often look for is some decisive and hopefully large event, where all the issues involving specification uncertainty can be sidelined or become second order. The Great Recession, for countries that did not suffer a second recession, might be just such an event. In earlier, milder recessions it was also much less clear what the monetary authority’s inflation target was (if it had one at all), and how credible it was. ...

I certainly agree with the claim that a "decisive and hopefully large event" is needed to empirically test econometric models since I've made the same point many times in the past. For example, "...the ability to choose one model over the other is not quite as hopeless as I’ve implied. New data and recent events like the Great Recession push these models into unchartered territory and provide a way to assess which model provides better predictions. However, because of our reliance on historical data this is a slow process – we have to wait for data to accumulate – and there’s no guarantee that once we are finally able to pit one model against the other we will be able to crown a winner. Both models could fail..."

Anyway...he goes on to discuss "How does what I did relate to recent discussions by Paul Krugman?," and concludes with:

My interpretation suggests that the New Keynesian Phillips curve is a more sensible place to start from than the adaptive expectations Friedman/Phelps version. As this is the view implicitly taken by most mainstream academic macroeconomics, but using a methodology that does not ensure congruence with the data, I think it is useful to point out when the mainstream does have empirical support. ...

'Empirical Evidence on Inflation Expectations in the New Keynesian Phillips Curve'

Via email, a comment on my comments about the difficulty of settling questions about the Phillips curve empirically:

Dear Professor Thoma,

I saw your recent post on the difficulty of empirically testing the Phillips Curve, and I just wanted to alert you to a survey paper on this topic that I wrote with Sophocles Mavroeidis and Jim Stock: "Empirical Evidence on Inflation Expectations in the New Keynesian Phillips Curve". It was published in the Journal of Economic Literature earlier this year (ungated working paper).

In the paper we estimate a vast number of specifications of the New Keynesian Phillips Curve (NKPC) on a common U.S. data set. The specification choices include the data series, inflation lag length, sample period, estimator, and so on. A subset of the specifications amount to traditional backward-looking (adaptive expectation) Phillips Curves. We are particularly interested in two key parameters: the extent to which price expectations are forward-looking, and the slope of the curve (how responsive inflation is to real economic activity).

Our meta-analysis finds that essentially any desired parameter estimates can be generated by some reasonable-sounding specification. That is, estimation of the NKPC is subject to enormous specification uncertainty. This is consistent with the range of estimates reported in the literature. Even if one were to somehow decide on a given specification, the uncertainty surrounding the parameter estimates is typically large. We give theoretical explanations for these empirical findings in the paper. To be clear: Our results do not reject the validity of the NKPC (or more generally, the presence of a short-run inflation/output trade-off), but traditional aggregate time series analysis is just not very informative about the nature of inflation dynamics.

Kind regards,

Mikkel Plagborg-Moller
PhD candidate in economics, Harvard University

Data Problems in Economics

New column:

Why Economists Can’t Always Trust Data, by Mark Thoma, The Fiscal Times: To make progress in economics, it is essential that theoretical models be subjected to empirical tests that determine how well they can explain actual data. The tests that are used must be able to draw a sharp distinction between competing theoretical models, and one of the most important factors is the quality of the data used in the tests. Unfortunately, the quality of the data that economists employ is less than ideal, and this gets in the way of the ability of economists to improve the models they use. There are several reasons for the poor quality of economic data...

Using Econometrics to Figure Out How the World Really Works

At MoneyWatch, what is econometrics?:

Using Econometrics to Figure Out How the World Really Works, by Mark Thoma: Many people believe there has been no progress in economics, but that isn't true. ...

'Varian: Big Data: New Tricks for Econometrics'

From the Journal of Economic Perspectives' Symposium on Big Data:

"Big Data: New Tricks for Econometrics,"  by Hal R. Varian: Computers are now involved in many economic transactions and can capture data associated with these transactions, which can then be manipulated and analyzed. Conventional statistical and econometric techniques such as regression often work well, but there are issues unique to big datasets that may require different tools. First, the sheer size of the data involved may require more powerful data manipulation tools. Second, we may have more potential predictors than appropriate for estimation, so we need to do some kind of variable selection. Third, large datasets may allow for more flexible relationships than simple linear models. Machine learning techniques such as decision trees, support vector machines, neural nets, deep learning, and so on may allow for more effective ways to model complex relationships. In this essay, I will describe a few of these tools for manipulating and analyzing big data. I believe that these methods have a lot to offer and should be more widely known and used by economists. Full-Text Access | Supplementary Materials

How Well Do Economists Predict Turning Points?

This may be of interest:

“There will be growth in the spring”: How well do economists predict turning points?, by Hites Ahir and Prakash Loungani: Forecasters have a poor reputation for predicting recessions. This column quantifies their ability to do so, and explores several reasons why both official and private forecasters may fail to call a recession before it happens.

'Pseudo-Mathematics and Financial Charlatanism'

"Past performance is not an indicator of future results":

Pseudo-mathematics and financial charlatanism, EurekAlert: Your financial advisor calls you up to suggest a new investment scheme. Drawing on 20 years of data, he has set his computer to work on this question: If you had invested according to this scheme in the past, which portfolio would have been the best? His computer assembled thousands of such simulated portfolios and calculated for each one an industry-standard measure of return on risk. Out of this gargantuan calculation, your advisor has chosen the optimal portfolio. After briefly reminding you of the oft-repeated slogan that "past performance is not an indicator of future results", the advisor enthusiastically recommends the portfolio, noting that it is based on sound mathematical methods. Should you invest?

The somewhat surprising answer is, probably not. Examining a huge number of sample past portfolios---known as "backtesting"---might seem like a good way to zero in on the best future portfolio. But if the number of portfolios in the backtest is so large as to be out of balance with the number of years of data in the backtest, the portfolios that look best are actually just those that target extremes in the dataset. When an investment strategy "overfits" a backtest in this way, the strategy is not capitalizing on any general financial structure but is simply highlighting vagaries in the data. ...

Unfortunately, the overfitting of backtests is commonplace not only in the offerings of financial advisors but also in research papers in mathematical finance. One way to lessen the problems of backtest overfitting is to test how well the investment strategy performs on data outside of the original dataset on which the strategy is based; this is called "out-of-sample" testing. However, few investment companies and researchers do out-of-sample testing. ...

Inequality in Capitalist Systems

New column:

Inequality in Capitalist Systems Is Not Inevitable, by Mark Thoma: Capitalism is the best economic system yet discovered for giving people the goods and services they desire at the lowest possible price, and for producing innovative economic growth. But there is a cost associated with these benefits, the boom and bust cycles inherent in capitalist systems, and those costs hit working class households – who have done nothing to deserve such a fate – very hard. Protecting innocent households from the costs of recessions is an important basis for our social insurance programs.

It is becoming more and more evident that there is another cost of capitalist systems, the inevitable rising inequality documented by Thomas Piketty in “Capital in the Twenty-First Century, that our social insurance system will need to confront. ...

'On R-Squared and Economic Prediction'

Jim Hamilton:

On R-squared and economic prediction: Recently I've heard a number of otherwise intelligent people assess an economic hypothesis based on the R² of an estimated regression. I'd like to point out why that can often be very misleading. ...

Here's what you'd find if you calculated a regression of this month's stock price (p_t) on last month's stock price (p_t-1). Standard errors of the regression coefficients are in parentheses.

The adjusted R-squared for this relation is 0.997. ... On the other hand, another way you could summarize the same relation is by using the change in the stock price (Δp_t = p_t - p_t-1) as the left-hand variable in the regression:

This is in fact the identical model of stock prices as the first regression. The standard errors of the regression coefficients are identical for the two regressions, and the standard error of the estimate ... is identical for the two regressions because indeed the residuals are identical for every observation. ...

Whatever you do, don't say that the first model is good given its high R-squared and the second model is bad given its low R-squared, because equations (1) and (2) represent the identical model. ...

That's not a bad empirical description of stock prices-- nobody can really predict them. ... This is actually a feature of a broad class of dynamic economic models, which posit that ... the deviation between what actually happens and what the decision-maker intended ... should be impossible to predict if the decision-maker is behaving rationally. For example, if everybody knew that a recession is coming 6 months down the road, the Fed should be more expansionary today... The implication is that when recessions do occur, they should catch the Fed and everyone else by surprise.

It's very helpful to look critically at which magnitudes we can predict and which we can't, and at whether that predictability or lack of predictability is consistent with our economic understanding of what is going on. But if what you think you learned in your statistics class was that you should always judge how good a model is by looking at the R-squared of a regression, then I hope that today you learned something new.

[There's an additional example and more explanation in the original post.]

The Tale of Joe of Metrika

This is semi-related to this post from earlier today. The tale of Joe of Metrika:

... Metrika began as a small village - little more than a coach-stop and a mandatory tavern at a junction in the highway running from the ancient data mines in the South, to the great city of Enlightenment, far to the North. In Metrika, the transporters of data of all types would pause overnight on their long journey; seek refreshment at the tavern; and swap tales of their experiences on the road.

To be fair, the data transporters were more than just humble freight carriers. The raw material that they took from the data mines was largely unprocessed. The vast mountains of raw numbers usually contained valuable gems and nuggets of truth, but typically these were buried from sight. The data transporters used the insights that they gained from their raucous, beer-fired discussions and arguments (known locally as "seminars") with the Metrika yokels locals at the tavern to help them to sift through the data and extract the valuable jewels. With their loads considerably lightened, these "data-miners" then continued on their journey to the City of Enlightenment in a much improved frame of mind, hangovers nothwithstanding!

Over time, the town of Metrika prospered and grew as the talents of its citizens were increasingly recognized and valued by those in the surrounding districts, and by the data miners transporters.

Young Joe grew up happily, supported by his family of econometricians, and he soon developed the skills that were expected of his societal class. He honed his computing skills; developed a good nose for "dodgy" data; and studiously broadened and deepened his understanding of the various tools wielded by the artisans in the neighbouring town of Statsbourg.

In short, he was a model child!

But - he was torn! By the time that he reached the tender age of thirteen, he felt the need to make an important, life-determining, decision.

Should he align his talents with the burly crew who frequented the gym near his home - the macroeconometricians - or should he throw in his lot with the physically challenged bunch of empirical economists known locally as the microeconometricians? ...

Full story here.

Econometrics and 'Big Data'

I tweeted this link, and it's getting far, far more retweets than I would have expected, so I thought I'd note it here:

Econometrics and "Big Data", by Dave Giles: In this age of "big data" there's a whole new language that econometricians need to learn. ... What do you know about such things as:

• Decision trees 
• Support vector machines
• Neural nets 
• Deep learning
• Classification and regression trees
• Random forests
• Penalized regression (e.g., the lasso, lars, and elastic nets)
• Boosting
• Bagging
• Spike and slab regression?

Probably not enough!

If you want some motivation to rectify things, a recent paper by Hal Varian ... titled, "Big Data: New Tricks for Econometrics" ... provides an extremely readable introduction to several of these topics.

He also offers a valuable piece of advice:

"I believe that these methods have a lot to offer and should be more widely known and used by economists. In fact, my standard advice to graduate students these days is 'go to the computer science department and take a class in machine learning'."

'Ten Things for Applied Econometricians to Keep in Mind'

Dave Giles:

Ten Things for Applied Econometricians to Keep in Mind, by Dave Giles: No "must do" list is ever going to be complete, let alone perfect. This is certainly true when it comes to itemizing essential ground-rules for all of us when we embark on applying our knowledge of econometrics.

That said, here's a list of ten things that I like my students to keep in mind:

1. Always, but always, plot your data.
2. Remember that data quality is at least as important as data quantity.
3. Always ask yourself, "Do these results make economic/common sense"?
4. Check whether your "statistically significant" results are also "numerically/economically significant".
5. Be sure that you know exactly what assumptions are used/needed to obtain the results relating to the properties of any estimator or test that you use.
6. Just because someone else has used a particular approach to analyse a problem that looks like, that doesn't mean they were right!
7. "Test, test, test"! (David Hendry). But don't forget that "pre-testing" raises some important issues of its own.
8. Don't assume that the computer code that someone gives to you is relevant for your application, or that it even produces correct results.
9. Keep in mind that published results will represent only a fraction of the results that the author obtained, but is not publishing.
10. Don't forget that "peer-reviewed" does NOT mean "correct results", or even "best practices were followed".

I'm sure you can suggest how this list can be extended!

I'll add two that I heard often in grad school:

Don't take econometric techniques in search of questions. Instead, start with the important questions and then develop the econometrics needed to answer them.

Model the process that generates the data.

Any further suggestions?

'We are Essentially Rewriting Economic History'

There will be a big revision of macroeconomic data in July:

Data shift to lift US economy by 3%, by Robin Harding, FT: The US economy will officially become 3 per cent bigger in July as part of a shake-up that will for the first time see government statistics take into account 21st century components such as film royalties and spending on research and development. ...

In an interview with the Financial Times, Brent Moulton, who manages the national accounts at the Bureau of Economic Analysis, said the update is the biggest since computer software was added to the accounts in 1999.

“We are carrying these major changes all the way back in time – which for us means to 1929 – so we are essentially rewriting economic history,” said Mr Moulton.

The changes will affect everything from the measured GDP of different US states to the stability of the inflation measure targeted by the US Federal Reserve. They will force economists to revisit policy debates about everything from corporate profits to the causes of economic growth. ...

The changes are in addition to a comprehensive revision of the national accounts that takes place every five years... Steve Landefeld, the BEA director, said it was hard to predict the overall outcome given the mixture of new methodology and data updates. ... But while the level of GDP may change,... “I wouldn’t be looking for large changes in trends or cycles,” said Mr Landefeld. ...

When working with macroeconomic data, we don't generally assume that there are large measurement errors in the data when assessing the significance of the results. Maybe we should.

Empirical Methods and Progress in Macroeconomics

The blow-up over the Reinhart-Rogoff results reminds me of a point I’ve been meaning to make about our ability to use empirical methods to make progress in macroeconomics. This isn't about the computational mistakes that Reinhart and Rogoff made, though those are certainly important, especially in small samples, it's about the quantity and quality of the data we use to draw important conclusions in macroeconomics.

Everybody has been highly critical of theoretical macroeconomic models, DSGE models in particular, and for good reason. But the imaginative construction of theoretical models is not the biggest problem in macro – we can build reasonable models to explain just about anything. The biggest problem in macroeconomics is the inability of econometricians of all flavors (classical, Bayesian) to definitively choose one model over another, i.e. to sort between these imaginative constructions. We like to think or ourselves as scientists, but if data can’t settle our theoretical disputes – and it doesn’t appear that it can – then our claim for scientific validity has little or no merit.

There are many reasons for this. For example, the use of historical rather than “all else equal” laboratory/experimental data makes it difficult to figure out if a particular relationship we find in the data reveals an important truth rather than a chance run that mimics a causal relationship. If we could do repeated experiments or compare data across countries (or other jurisdictions) without worrying about the “all else equal assumption” we’d could perhaps sort this out. It would be like repeated experiments. But, unfortunately, there are too many institutional differences and common shocks across countries to reliably treat each country as an independent, all else equal experiment. Without repeated experiments – with just one set of historical data for the US to rely upon – it is extraordinarily difficult to tell the difference between a spurious correlation and a true, noteworthy relationship in the data.

Even so, if we had a very, very long time-series for a single country, and if certain regularity conditions persisted over time (e.g. no structural change), we might be able to answer important theoretical and policy questions (if the same policy is tried again and again over time within a country, we can sort out the random and the systematic effects). Unfortunately, the time period covered by a typical data set in macroeconomics is relatively short (so that very few useful policy experiments are contained in the available data, e.g. there are very few data points telling us how the economy reacts to fiscal policy in deep recessions).

There is another problem with using historical as opposed to experimental data, testing theoretical models against data the researcher knows about when the model is built. In this regard, when I was a new assistant professor Milton Friedman presented some work at a conference that impressed me quite a bit. He resurrected a theoretical paper he had written 25 years earlier (it was his plucking model of aggregate fluctuations), and tested it against the data that had accumulated in the time since he had published his work. It’s not really fair to test a theory against historical macroeconomic data, we all know what the data say and it would be foolish to build a model that is inconsistent with the historical data it was built to explain – of course the model will fit the data, who would be impressed by that? But a test against data that the investigator could not have known about when the theory was formulated is a different story – those tests are meaningful (Friedman’s model passed the test using only the newer data).

As a young time-series econometrician struggling with data/degrees of freedom issues I found this encouraging. So what if in 1986 – when I finished graduate school – there were only 28 quarterly observations for macro variables (112 total observations, reliable data on money, which I almost always needed, doesn’t begin until 1959). By, say, the end of 2012 there would be almost double that amount (216 versus 112!!!). Asymptotic (plim-type) results here we come! (Switching to monthly data doesn’t help much since it’s the span of the data – the distance between the beginning and the end of the sample – rather than the frequency the data are sampled that determines many of the “large-sample results”).

By today, I thought, I would have almost double the data I had back then and that would improve the precision of tests quite a bit. I could also do what Friedman did, take really important older papers that give us results “everyone knows” and see if they hold up when tested against newer data.

It didn’t work out that way. There was a big change in the Fed’s operating procedure in the early 1980s, and because of this structural break today 1984 is a common starting point for empirical investigations (start dates can be anywhere in the 79-84 range though later dates are more common). Data before this time-period are discarded.

So, here we are 25 years or so later and macroeconomists don’t have any more data at our disposal than we did when I was in graduate school. And if the structure of the economy keeps changing – as it will – the same will probably be true 25 years from now. We will either have to model the structural change explicitly (which isn’t easy, and attempts to model structural beaks often induce as much uncertainty as clarity), or continually discard historical data as time goes on (maybe big data, digital technology, theoretical advances, etc. will help?).

The point is that for a variety of reasons – the lack of experimental data, small data sets, and important structural change foremost among them – empirical macroeconomics is not able to definitively say which competing model of the economy best explains the data. There are some questions we’ve been able to address successfully with empirical methods, e.g., there has been a big change in views about the effectiveness of monetary policy over the last few decades driven by empirical work. But for the most part empirical macro has not been able to settle important policy questions. The debate over government spending multipliers is a good example. Theoretically the multiplier can take a range of values from small to large, and even though most theoretical models in use today say that the multiplier is large in deep recessions, ultimately this is an empirical issue. I think the preponderance of the empirical evidence shows that multipliers are, in fact, relatively large in deep recessions – but you can find whatever result you like and none of the results are sufficiently definitive to make this a fully settled issue.

I used to think that the accumulation of data along with ever improving empirical techniques would eventually allow us to answer important theoretical and policy questions. I haven’t completely lost faith, but it’s hard to be satisfied with our progress to date. It’s even more disappointing to see researchers overlooking these well-known, obvious problems – for example the lack pf precision and sensitivity to data errors that come with the reliance on just a few observations – to oversell their results.

Live feed for INET Hong Kong Conference

I should have noted this when I posted the conference schedule. If you want to watch a live feed of the sessions, it's at:

Live feed for INET Hong Kong Conference

Remember that all times listed are Hong Kong Time (15 hours ahead of PST, 12 hours ahead of EST). Videos of each session will also be posted (same address as the link above).

'Is 'Intellectual Property' a Misnomer?'

Tim Taylor:

Is "Intellectual Property" a Misnomer?, by Tim Taylor: The terminology of "intellectual property" goes back to the eighteenth century. But some modern critics of how the patent and copyright law have evolved have come to view the term as a tendentious choice. One you have used the "property" label, after all, you are implicitly making a claim about rights that should be enforced by the broader society. But "intellectual property"  is a much squishier subject than more basic applications of property, like whether someone can move into your house or drive away in your car or empty your bank account. ...

Is it really true that using someone else's invention is the actually the same thing as stealing their sheep? If I steal your sheep, you don't have them any more. If I use your idea, you still have the idea, but are less able to profit from using it. The two concepts may be cousins, but they not identical. 

Those who believe that patent protection has in some cases gone overboard, and is now in many industries acting more to protect established firms than to encourage new innovators, thus refer to "intellectual property as a "propaganda term." For a vivid example of these arguments, see "The Case Against Patents," by Michele Boldrin and David K. Levine, in the Winter 2013 issue of my own Journal of Economic Perspectives. (Like all articles in JEP back to the first issue in 1987, it is freely available on-line courtesy of the American Economic Association.)

Mark Lemley offers a more detailed unpacking of the concept of "intellectual  property" in a 2005 article he wrote for the Texas Law Review called "Property, Intellectual Property, and Free Riding" Lemley writes: ""My worry is that the rhetoric of property has a clear meaning in the minds of courts, lawyers and commentators as “things that are owned by persons,” and that fixed meaning will make all too tempting to fall into the trap of treating intellectual property just like “other” forms of property. Further, it is all too common to assume that because something is property, only private and not public rights are implicated. Given the fundamental differences in the economics of real property and intellectual property, the use of the property label is simply too likely to mislead."

As Lemley emphasizes, intellectual property is better thought of as a kind of subsidy to encourage innovation--although the subsidy is paid in the form of higher prices by consumers rather than as tax collected from consumers and then spent by the government. A firm with a patent is able to charge more to consumers, because of the lack of competition, and thus earn higher profits. There is reasonably broad agreement among economists that it makes sense for society to subsidize innovation in certain ways, because innovators have a hard time capturing the social benefits they provide in terms of greater economic growth and a higher standard of living, so without some subsidy to innovation, it may well be underprovided.

But even if you buy that argument, there is room for considerable discussion of the most appropriate ways to subsidize innovation. How long should a patent be? Should the length or type of patent protection differ by industry? How fiercely or broadly should it be enforced by courts? In what ways might U.S. patent law be adapted based on experiences and practices in other major innovating nations like Japan or Germany? What is the role of direct government subsidies for innovation in the form of government-sponsored research and development? What about the role of indirect government subsidies for innovation in the form of tax breaks for firms that do research and development, or in the form of support for science, technology, and engineering education? Should trade secret protection be stronger, and patent protection be weaker, or vice versa? 

These are all legitimate questions about the specific form and size of the subsidy that we provide to innovation. None of the questions about "intellectual property" can be answered yelling "it's my property." 

The phrase "intellectual property" has been around a few hundred years, so it clearly has real staying power and widespread usage  I don't expect the term to disappear. But perhaps we can can start referring to intellectual "property" in quotation marks, as a gentle reminder that an overly literal interpretation of the term would be imprudent as a basis for reasoning about economics and public policy.