Why Did Risk Models Fail?

I've been meaning to write about this, but haven't had the chance to think it through thoroughly. The topic is why risk models failed to inform regulators, and failed to protect investors during the recent crisis. Many of these strategies rely upon mining data for correlations that can be used to construct portfolios that (supposedly) have low variance returns.

I've wondered if the problems these models encountered can be explained using standard criticisms concerning the use of reduced form approaches to look for correlations in the data, correlations that can breakdown when there is any change in the underlying structure of the model, or a change in how policy reacts to macroeconomic variables (i.e. a standard Lucas-type critique of relying on reduced form estimates rather than deeper structural parameters and shocks). There's a difference between mining data for correlations and understanding why those correlations occur. The former can be accomplished through a variety of approaches, some borrowed from other disciplines. The goal isn't to understand how things work, the models generally make no attempt to explain why particular correlations are in the data, they simply look for the existence of correlations and attempt to exploit them. That may work fine as a money-making strategy in the short-run when the structure of the economy is fairly constant, but for understanding why things work the way they do, and to understand how things might suddenly change, theoretical models are necessary. There may also be a problem with the difference in responses to systemic and more localized shocks, particular since large economy-wide shocks are relatively rare and hence hard to evaluate using data-based methods.

But as I said, I haven't given this enough thought, so here are some views on why risk models failed just when they were most needed from people who have spent more time thinking about this than I have. Let's start with Alan Greenspan since the explanation that comes after this references this article:

How did we go so wrong?

The essential problem is that our models – both risk models and econometric models – as complex as they have become, are still too simple... A model, of necessity, is an abstraction from the full detail of the real world. In line with the time-honoured observation that diversification lowers risk, computers crunched reams of historical data in quest of negative correlations between prices of tradeable assets; correlations that could help insulate investment portfolios from the broad swings in an economy. When such asset prices, rather than offsetting each other’s movements, fell in unison on and following August 9 last year, huge losses across virtually all risk-asset classes ensued.

The most credible explanation of why risk management based on state-of-the-art statistical models can perform so poorly is that the underlying data used to estimate a model’s structure are drawn ... from ... periods of euphoria and periods of fear, that is, from regimes with importantly different dynamics.

The contraction phase of credit and business cycles, driven by fear, have historically been far shorter and far more abrupt than the expansion phase... Over the past half-century, the American economy was in contraction only one-seventh of the time. But it is the onset of that one-seventh for which risk management must be most prepared. Negative correlations among asset classes, so evident during an expansion, can collapse as all asset prices fall together, undermining the strategy of improving risk/reward trade-offs through diversification. ...

I do not say that the current systems of risk management or econometric forecasting are not in large measure soundly rooted in the real world. ... But these models do not fully capture what I believe has been, to date, only a peripheral addendum to business-cycle and financial modelling – the innate human responses that result in swings between euphoria and fear that repeat themselves generation after generation with little evidence of a learning curve. Asset-price bubbles build and burst today as they have since the early 18th century, when modern competitive markets evolved. To be sure, we tend to label such behavioural responses as non-rational. But forecasters’ concerns should be not whether human response is rational or irrational, only that it is observable and systematic.

This, to me, is the large missing “explanatory variable” in both risk-management and macroeconometric models. ... We will never be able to anticipate all discontinuities in financial markets. Discontinuities are, of necessity, a surprise. ...

Here is Avinash Persaud, writing on Willem Buiter's blog, with his take on the problem:

Why Bank Risk Models Failed and the Implications for what Policy Makers Have to Do Now, by Avinash D. Persaud: Sir Alan Greenspan, and others have questioned why risk models, which are at the centre of financial supervision, failed to avoid or mitigate today’s financial turmoil. There are two answers to this, one technical and the other philosophical. Neither is complex, but many regulators and central bankers chose to ignore them both.

The technical explanation is that market-sensitive risk models used by thousands of market participants work on the assumption that each user is the only person using them. This was not a bad approximation in 1952, when the intellectual underpinnings of these models were being developed ... by Harry Markovitz and George Dantzig. ...

In today’s flat world, market participants from Argentina to New Zealand have the same data on the risk, returns and correlation of financial instruments and use standard optimization models, which throw up the same portfolios to be favoured and those not to be. Market participants don’t stare helplessly at these results. They move into the favoured markets and out of the unfavoured. Enormous cross-border capital flows are unleashed. But under the weight of the herd, favoured instruments cannot remain undervalued, uncorrelated and low risk. ...

When a market participant’s risk model detects a rise in risk in his portfolio, perhaps because of some random rise in volatility, and he tries to reduce his exposure, many others are trying to do the same thing at the same time with the same assets. A vicious cycle ensues of vertical price falls prompting further selling. Liquidity vanishes down a black hole. ...

Policy makers cannot claim to be surprised by all of this. The observation that market-sensitive risk models ... were going to send the herd off the cliff edge was made soon after the last round of crises*. Many policy officials in charge today, responded then that these warnings were too extreme to be considered realistic.

This brings us to the philosophical problem of the reliance of supervisors on bank risk models. The reason we regulate markets over and above normal corporate law is that from time to time markets fail and these failings have devastating consequences. If the purpose of regulation is to avoid market failures, we cannot use ... risk-models that rely on market prices, or any other instrument derived from market prices such as mark-to-market accounting. Market prices cannot save us from market failures. Yet, this is the thrust of modern financial regulation, which calls for more transparency on prices, more price-sensitive risk models and more price-sensitive prudential controls. These tools are like seat belts that stop working whenever you press hard on the accelerator.

In terms of solutions, there is only space to observe that if we rely on market prices in our risk models and in value accounting, we must do so on the understanding that in rowdy times central banks will have to become buyers of last resort of distressed assets to avoid systemic collapse. This is the approach we have stumbled upon. Central bankers now consider mortgage-backed securities as collateral for their loans to banks. But the asymmetry of being a buyer of last resort without also being a seller of last resort during the unsustainable boom will only condemn us to cycles of instability.

The alternative is to try and avoid booms and crashes through regulatory and fiscal mechanisms designed to work against the incentives ... for traders and investors to double up or more into something that the markets currently believe is a sure bet. This sounds fraught and policy makers are not as ambitious as they once were. ...

Regulatory ambition should be set now, while the fear of the current crisis is fresh and not when the crisis is over and the seat belts are working again.