Risk Management

I probably should have done more to highlight the article on risk management by Joe Nocera that appeared in the NY Times Magazine this weekend. Fortunately, James Kwak and others have it covered:

Risk Management for Beginners, by James Kwak: Joe Nocera has an article ... about Value at Risk (VaR), a risk management technique used by financial institutions to measure the risk of individual trading desks or aggregate portfolios. ...

VaR is a way of measuring the likelihood that a portfolio will suffer a large loss in some period of time, or the maximum amount that you are likely to lose with some probability (say, 99%). It does this by: (1) looking at historical data about asset price changes and correlations; (2) using that data to estimate the probability distributions of those asset prices and correlations; and (3) using those estimated distributions to calculate the maximum amount you will lose 99% of the time. At a high level, Nocera’s conclusion is that VaR is a useful tool even though it doesn’t tell you what happens the other 1% of the time.

naked capitalism already has one withering critique of the article out. There, Yves Smith focuses on the assumption, mentioned but not explored by Nocera, that the ... changes in asset prices ... are normally distributed. To summarize, for decades people have known that financial events are not normally distributed.... Yet ... VaR modelers continue to assume normal distributions..., which leads to results that are simply incorrect. It’s a good article, and you’ll probably learn something.

While Smith focuses on the problem of using the wrong mathematical tools, and Nocera mentions the problem of not using enough historical data - “...VaR didn’t see the risk because it generally relied on a two-year data history” - I want to focus on another weakness of VaR: the fact that the real world changes.

Even leaving aside the question of which distribution (normal or otherwise) to use, VaR assumes the likelihood of future events is dictated by some distribution, and that that distribution can be estimated using past data. A simple example is a weighted coin that you find on the street. You flip it 1,000 times and it comes up heads 600 times, tails 400 times. You infer that it has a 60% likelihood of coming up heads; from that, you can calculate the probability distribution for how many heads will come up if you flip it 10 more times, and if you want to bet on those coin flips you can calculate your VaR. Your 60% is just an estimate - you don’t know that the true probability is 60% - but you can safely assume that the physical properties of the coin are not going to change, and you can use statistics to estimate how accurate your estimate is. ...

By contrast, imagine you have two basketball teams, the Bulls and the Knicks, who have played 1,000 games, and the Knicks have won 600. You follow the same methodology, bet a lot of money that the Knicks will win at least 5 of the next 10 games - and then the Bulls draft Michael Jordan. See the problem?

Now, are asset prices more like coin flips or like basketball times? On an empirical level, they may be more like coin flips; their probability distributions aren’t likely to change as dramatically as when the Bulls draft Jordan.... But on a fundamental level, they are more like basketball teams. The outcome of a coin flip is dictated by physical processes, governed by the laws of mechanics, that we know are going to operate the same way time after time. Asset prices, by contrast, are the product of individual decisions by thousands, millions, or even billions of people... We have little idea what underlying mechanisms produce those prices, and all the simplifying assumptions we make (like rational profit-maximizing agents) are pure fiction.

Whatever the underlying function for price changes is,... importantly, no one tells us when the function changes. Going back to asset prices: To estimate the probability distribution of price changes, you need a sample that reflects your population of interest as closely as possible. Unfortunately, your sample can only be drawn from the past, and your population of interest is the future. So you really face two different risks. You face the risk that, in the current state of the world (assuming you can estimate that perfectly), an unlikely event will occur. You also face the risk that the state of the world will change. VaR, at best (assuming solutions to Smith’s criticisms), can quantify the first risk, not the second. ...

There was one part of Nocera’s article that I liked a lot:

At the height of the bubble, there was so much money to be made that any firm that pulled back because it was nervous about risk would forsake huge short-term gains and lose out to less cautious rivals. The fact that VaR didn’t measure the possibility of an extreme event was a blessing to the executives. It made black swans [unlikely events] all the easier to ignore. All the incentives — profits, compensation, glory, even job security — went in the direction of taking on more and more risk, even if you half suspected it would end badly. After all, it would end badly for everyone else too. As the former Citigroup chief executive Charles Prince famously put it, “As long as the music is playing, you’ve got to get up and dance.” Or, as John Maynard Keynes once wrote, a “sound banker” is one who, “when he is ruined, is ruined in a conventional and orthodox way.”

This, I think, is an accurate picture of what was going on. If you were a senior executive at an investment bank, even if you knew you were in a bubble that was going to collapse, it was still in your interests to play along, for at least two reasons: the enormity of the short-term compensation to be made outweighed the relatively paltry financial risk of being fired in a bust (given severance packages, and the fact that in a downturn all CEO compensation would plummet); and bucking the trend incurs resume risk in a way that playing along doesn’t. ... Or, in the brilliant words of John Dizard (cited in the naked capitalism article):

A once-in-10-years-comet-wiping-out-the-dinosaurs disaster is a problem for the investor, not the manager-mammal who collects his compensation annually, in cash, thank you. He has what they call a “résumé put”, not a term you will find in offering memoranda, and nine years of bonuses.

Comments

[Mark]

Prior to the collapse, the literature was chock full of articles about model uncertainty, short data time series, correlation problems, etc. The money was just too good.

[bakho]

All the more reason to have competent regulation. The pruning shear/chain saw approach to the SEC did not work very well. Maybe it is time for a public sponsored ratings agency that is not in cahoots with the ratees?

[Rusty]

A few comments from someone who's worked with VaR and VaR models for a while:

1. Taleb's made a living off of his Black Swan theory but in fact the fund he ran off that that principle had closed down in 2004 because of losses (low volatility). His principles are sound in theory but because he operates so far out in the tails of the distribution, no one, including himself can properly price the instruments. He's making money now but in the long run, the odds are against him and he'll eventually give back all his gains. Mathematically he's just lucky right now; the same way a gambler hits a "winning streak" (there's really no such thing). He is just playing the other end of the distribution.

2. The VaR models I'm acquainted with used lognormal distributions (pricing doesn't go negative) and we also used extreme value theorem to examine tail events. This is at a few small ($1 B or less) trading shops. I know a lot of the Wall Street quants at the bigger shops(went to Grad school with more than a few) and they all know this stuff better than I did. Great care and attention go into pricing models to forecast future prices. We're less certain of the far future but in my experience, in the short term (say a month out), the pricing models are eerily good and again, this is at small shops with no Ph.D. mathematicians or statisticians; just models built by regular Joes with a minimum amount of sophistication.

I think it's far too simplistic to assume that somehow all this is related to VaR going wacky. Companies set VaR limits to contain losses - all the models do are compute what those losses might be. Every company has different criteria, however, on what the parameters are (99% confidence over 5 days, 95% over 1 day, etc.). It's up to the company's management to set the criteria and the limit. The limits might have been increased too much - I don't know; usually the limits are highly proprietary and I doubt if the author knows what the limits were. VaR is a useful internal tool but it's just a model and a data point; it was never intended to a drive business.

I agree with the comment on rating agencies - as someone in the trenches (and not in academia), I think they have far too big of impact on a company. The bigger issue, however, is accounting rules. In the wake of Enron, we've enacted a lot of well intentioned rules that just don't work. Fix that and the loopholes and you can fix a lot of the problems that we've seen.

[bakho]

Garbage in Garbage out. If they were depending on the ratings agencies for the input to the VaR model, then they were putting garbage in.

It seems like this administration and the regulators were and are trying to keep the finance bubble inflated rather than using the regulations to prevent the bubble from forming. Bush finally admitted this month that we are in recession after spending the last year denying that the economy had problems.

[Zipf]

I would venture that VaR isn't a good metric. Try expected shortfall instead.

Given that VaR with the normal distribution is taught in the intro textbooks, I would like to think something more advanced is used in practice. (e.g., Student's with low DOF.)

[evagrius]

You follow the same methodology, bet a lot of money that the "market" will win at least 5 of the next 10 years - and then Madoff happens. See the problem?

[Russell]

Taleb limits his losses - his strategy can never (barring a collapse of the civilized world, which is possible) lose more than 10 - 20%, but his gains can be 100% or more (as is the case right now). He knows he can't price the instruments properly, but also knows no one else can either. Thus, he admits his own ignorance/limitations while taking advantage of others' arrogance. It's true genius (even if he is not a great writer or interview and understandably rubs a lot of people the wrong way).

Rusty, I think you are assuming a Gaussian distribution. Don't do that.

[Not Mark T]

A random variable is lognormally distributed if the logarithm of the values follows a Guassian distribution (is normally distributed). However, although the lognormal can't take negative values, as the coefficient of variation (stdev/mean) becomes smaller, the lognormal approximates a normal distribution with skewness = 0 and kurtosis = 3. Consequently, if the data fit to the lognormal exhibit little variation, the estimated distribution may not be very heavy tailed (relative to the normal).

[ECONOMISTA NON GRATA]

".... Risk Management for Beginners....." ??????????

LOL..... Ha-ha-ha-ha-ha....! How about... "Risk Management for Dummies, Nincompoops, Masters, Pros, Comemierdas.....? Let me tell you about our "risk management team" madam.... Sounds soooo soothing.... he's the guy in the cage throwing his feces at the customers.... Not sooo soothing now.....!

Best regards,

Econolicious

[Eric Dewey, Portland, Oregon]

Great post, Mark - and James. Keynes' definition of "sound banking" is indeed what has been going on in the financial institutions that have brought us to this pass - I've seen it happen firsthand.

More importantly, James' article points out the fundamental truth that the world will change, a truth that simply cannot be effectively modeled mathematically.

Every model is to some degree a simplification, and if those who make decisions do not have a deep understanding of the data points ignored by the model, eventually their decisions will become toxic. And if the model works long enough for the first decisionmakers using the model to be replaced by a new set, the likelihood of toxic decisionmaking expands significantly (perhaps even exponentially)...

Those who wish to defend VaR as a model aren't necessarily wrong to do so - but unless they are willing to acknowledge its limitations, they are foolish, and I would not spend much time listening to their predictions, unless I knew I only had about 6 months to live...

[don]

"A once-in-10-years-comet-wiping-out-the-dinosaurs disaster is a problem for the investor, not the manager-mammal who collects his compensation annually, in cash, thank you. He has what they call a “résumé put”, not a term you will find in offering memoranda, and nine years of bonuses."
Bears repeating. And points directly to a major culprit - excessive executive compensation.

[Sonic Charmer]

I have a slightly different critique than most here: when popular accounts of Taleb's 'brilliant' insights describe VaR as a "risk management technique", I think they are out of their minds. VaR is among a batch of numbers that must be spit out usually for regulatory or related purposes. (Right?) Everyone knows it's GIGO and no one with half a brain seriously believes otherwise. So I do not believe - and my experience bears this out - that VaR is seriously "used" in practice to actually manage risk in the first place. It could be described as being among a basket of tools and flag-setting filters, of course, but the idea of anyone actually using VaR to manage their risk makes me laugh. Trust me, business heads do not think about VaR unless they have to (i.e. someone bugs them that they're up against some limit, which is often due to some spurious artifact of the calculation or of the organizational structure). Traders? Don't make me laugh - they think about VaR even less, if they even know what it is.

VaR is simply not what these accounts say it is.

But even if VaR were what the Time Magazine version of Taleb said it were (mind you I have not read Taleb and, God willing, never will), and it were somehow used (by whom exactly? the group charged with calculating it? hahaha) as the primary risk management technique, the criticisms still get it wrong. There is nothing inherent about VaR that assumes anything has a normal distribution. Nor lognormal. Banks don't ever, necessarily, make any normality assumption about anything whatsoever. Maybe they use time series. Monte Carlo.

The broader criticism that to compute VaR one must make *some* assumption about distribution is of course absolutely correct (and this is part and parcel of why people don't actually "use" VaR the way these accounts would have you believe), but toss out the bogus implication that virtually all VaR calculations assume normality in everything and you lose the power of the whole 'Black Swan'/underweighted-tail-event charge. Tail events will, if anything, be overweighted if your time series happens to include a huge market blowup....it all depends on the distribution. And the critique above is completely correct about that. But everyone already knows this about VaR...or should.

If I never have to read any more of these self-congratulatory pop-econ accounts of Taleb, VaR, or "black swans" I'll be happy. None of this has very much to do with reality, except in the sense that Taleb becoming a bestselling sage author of Gladwellian proportions is now, regrettably, part of our reality.