I probably should have done more to highlight the article on risk management by Joe Nocera that appeared in the NY Times Magazine this weekend. Fortunately, James Kwak and others have it covered:
Risk Management for Beginners, by James Kwak: Joe Nocera has an article ... about Value at Risk (VaR), a risk management technique used by financial institutions to measure the risk of individual trading desks or aggregate portfolios. ...
VaR is a way of measuring the likelihood that a portfolio will suffer a large loss in some period of time, or the maximum amount that you are likely to lose with some probability (say, 99%). It does this by: (1) looking at historical data about asset price changes and correlations; (2) using that data to estimate the probability distributions of those asset prices and correlations; and (3) using those estimated distributions to calculate the maximum amount you will lose 99% of the time. At a high level, Nocera’s conclusion is that VaR is a useful tool even though it doesn’t tell you what happens the other 1% of the time.
naked capitalism already has one withering critique of the article out. There, Yves Smith focuses on the assumption, mentioned but not explored by Nocera, that the ... changes in asset prices ... are normally distributed. To summarize, for decades people have known that financial events are not normally distributed.... Yet ... VaR modelers continue to assume normal distributions..., which leads to results that are simply incorrect. It’s a good article, and you’ll probably learn something.
While Smith focuses on the problem of using the wrong mathematical tools, and Nocera mentions the problem of not using enough historical data - “...VaR didn’t see the risk because it generally relied on a two-year data history” - I want to focus on another weakness of VaR: the fact that the real world changes.
Even leaving aside the question of which distribution (normal or otherwise) to use, VaR assumes the likelihood of future events is dictated by some distribution, and that that distribution can be estimated using past data. A simple example is a weighted coin that you find on the street. You flip it 1,000 times and it comes up heads 600 times, tails 400 times. You infer that it has a 60% likelihood of coming up heads; from that, you can calculate the probability distribution for how many heads will come up if you flip it 10 more times, and if you want to bet on those coin flips you can calculate your VaR. Your 60% is just an estimate - you don’t know that the true probability is 60% - but you can safely assume that the physical properties of the coin are not going to change, and you can use statistics to estimate how accurate your estimate is. ...
By contrast, imagine you have two basketball teams, the Bulls and the Knicks, who have played 1,000 games, and the Knicks have won 600. You follow the same methodology, bet a lot of money that the Knicks will win at least 5 of the next 10 games - and then the Bulls draft Michael Jordan. See the problem?
Now, are asset prices more like coin flips or like basketball times? On an empirical level, they may be more like coin flips; their probability distributions aren’t likely to change as dramatically as when the Bulls draft Jordan.... But on a fundamental level, they are more like basketball teams. The outcome of a coin flip is dictated by physical processes, governed by the laws of mechanics, that we know are going to operate the same way time after time. Asset prices, by contrast, are the product of individual decisions by thousands, millions, or even billions of people... We have little idea what underlying mechanisms produce those prices, and all the simplifying assumptions we make (like rational profit-maximizing agents) are pure fiction.
Whatever the underlying function for price changes is,... importantly, no one tells us when the function changes. Going back to asset prices: To estimate the probability distribution of price changes, you need a sample that reflects your population of interest as closely as possible. Unfortunately, your sample can only be drawn from the past, and your population of interest is the future. So you really face two different risks. You face the risk that, in the current state of the world (assuming you can estimate that perfectly), an unlikely event will occur. You also face the risk that the state of the world will change. VaR, at best (assuming solutions to Smith’s criticisms), can quantify the first risk, not the second. ...
There was one part of Nocera’s article that I liked a lot:
At the height of the bubble, there was so much money to be made that any firm that pulled back because it was nervous about risk would forsake huge short-term gains and lose out to less cautious rivals. The fact that VaR didn’t measure the possibility of an extreme event was a blessing to the executives. It made black swans [unlikely events] all the easier to ignore. All the incentives — profits, compensation, glory, even job security — went in the direction of taking on more and more risk, even if you half suspected it would end badly. After all, it would end badly for everyone else too. As the former Citigroup chief executive Charles Prince famously put it, “As long as the music is playing, you’ve got to get up and dance.” Or, as John Maynard Keynes once wrote, a “sound banker” is one who, “when he is ruined, is ruined in a conventional and orthodox way.”
This, I think, is an accurate picture of what was going on. If you were a senior executive at an investment bank, even if you knew you were in a bubble that was going to collapse, it was still in your interests to play along, for at least two reasons: the enormity of the short-term compensation to be made outweighed the relatively paltry financial risk of being fired in a bust (given severance packages, and the fact that in a downturn all CEO compensation would plummet); and bucking the trend incurs resume risk in a way that playing along doesn’t. ... Or, in the brilliant words of John Dizard (cited in the naked capitalism article):
A once-in-10-years-comet-wiping-out-the-dinosaurs disaster is a problem for the investor, not the manager-mammal who collects his compensation annually, in cash, thank you. He has what they call a “résumé put”, not a term you will find in offering memoranda, and nine years of bonuses.