Quants Did It?

Scientific American says quants need to learn more economics and history before deploying their mathematical tools:

After the Crash, SciAm: If Hollywood makes a movie about the worst financial crisis since the Great Depression, a basement room in a government building in Washington will serve as the setting for a key scene. There investment bankers from the largest institutions pleaded successfully with Securities and Exchange Commission (SEC) officials during a short meeting in 2004 to lift a rule specifying debt limits and capital reserves needed for a rainy day. This decision, a real event described in the New York Times, freed billions to invest in complex mortgage-backed securities and derivatives that helped to bring about the financial meltdown in September.

In the script, the next scene will be the one in which number savvy specialists that Wall Street has come to know as quants consult with their superiors about implementing the regulatory change. These lapsed physicists and mathematical virtuosos were the ones who both invented these oblique securities and created software models that supposedly measured the risk a firm would incur by holding them... Without the formal requirement to maintain debt ceilings and capital reserves, the commission had freed these firms to police themselves using risk tools crafted by cadres of quants.

The software models in question estimate the level of financial risk of a portfolio for a set period at a certain confidence level. As Benoit Mandelbrot, the fractal pioneer who is a longtime critic of mainstream financial theory, wrote in Scientific American in 1999, established modeling techniques presume falsely that radically large market shifts are unlikely and that all price changes are statistically independent... Here is where reality and rocket science diverge. Try Googling “financial meltdown,” “contagion” and “2008,” a search that reveals just how wrongheaded these assumptions were. ...

In aviation, controlled flight into terrain describes the actions of a pilot who, through inattention or incompetence, directs a well-functioning airplane into the side of a mountain. Wall Street’s version stems from the SEC’s decision to allow over reliance on risk software in the middle of a historic housing bubble. The heady environment permitted traders to enter overoptimistic assumptions and faulty data into their models, jiggering the software to avoid setting off alarm bells.

The causes of this fiasco are multifold ... but the rocket scientists and geeks also bear their share of the blame. ... The regulators must ensure that the many lessons of this debacle are not forgotten by the institutions that trade these securities. One important take-home message: capital safety nets (now restored) should never be slashed again, even if a crisis is not looming.

For its part, the quant community needs to undertake a search for better models—perhaps seeking help from behavioral economics, which studies irrationality of investors’ decision making... These number wizards and their superiors need to study lessons that were never learned during previous market smashups involving intricate financial engineering...

Systems with agents who can respond to changes in their economic environment are very different from the types of physical systems quants are used to working with. Starting with reduced form techniques rather than strucural/behavioral models and attempting to exploit trends that are uncovered from backward looking procedures runs the risk of going very wrong if the response of agents in the model drives the economy away from its historical precedent. Much of what was done amounted to this approach no matter how dressed up it was mathematically. When the economy did, in fact, diverge from past trends as agents responded to the changed economic environment brought about by those attempting to exploit trends in the data, the models, unsurprisingly, failed.

Comments

[Stephen Gordon]

It seems hard to believe that after 30 years, people have to learn the Lucas Critique the hard way.

[Richard H. Serlin]

You might say, a model is only as good as the person interpreting it.

One of the weakness of academic economics and finance is that you can get tenure at Harvard by just acting as though the models are exactly the real world, and working 100 hours per week (from age 5), with strong mechanical skills, but not strong high level intelligence, or not using your high level intelligence if it is strong.

[Bruce Wilder]

"In aviation, controlled flight into terrain describes the actions of a pilot who, through inattention or incompetence, directs a well-functioning airplane into the side of a mountain."

MT: "Systems with agents who can respond to changes in their economic environment are very different from the types of physical systems quants are used to working with."

Physical systems don't have principals, who can take billions of dollars home in their lunch pails. The most important safety feature in aircraft is the requirement that the pilot be on board.

The "technical" errors in risk management are, no doubt, important to note, but this catastrophe is not the consequence of some innocent technical error, nor even the collective hubris of the post-digital geek squad.

Capital requirements were a constraint on take-home pay. This aspect of the situation was not lost on those pressing the SEC for deregulation. And, it should not be lost in the remembering.

[teasrini]

Structural behavioral models of the economy that are useful in forecasting and making money? Can you name one? I don't mean to be snarky, Prescott and some other similar dude (or was it Lucas and Rapping) built one which explained the Great Depression unemployment as rational agents indulging in intertemporal substitution.

I work in the fund of hedge fund industry and have a pretty good idea of the landscape of hedge fund strategies. In fact, the only strategies that have worked consistently and successfully since August 2007 are technically oriented (i.e. mainly price driven) systematic strategies. These are what would broadly be called backward-looking, pattern matching strategies (BTW, these strategies have long history of uncorrelated excess returns). Discretionary macro strategies are a distant second. Every other strategy (other than dedicated short) has performed poorly. Contrary to what most economist think, even if agents are forward looking, price behavior contains information. The reasons are myriad. In that sense, markets are NOT efficient. However, no model is universal and omniscient because the world is non-ergodic--path dependent and evolutionary. Good model builders have several key characteristics--first, is that their models have some kind of Bayesian adaptation either built in, or the model builder keeps updating by using rolling windows. They also have several models and use all or some of them and switch some on/off in different regimes. Last but not least, they stop the model when the volatility of prediction error rises. This is essentially a heuristic methodology to deal with uncertainty, fat tails, and imperfect knowledge.

Lucas critique is hardly relevant for the current predicament. Lucas critique dealt with the use of reduced form model for running government policy. In Lucas's world rational private agents (which presumably would include Wall Street bankers, mortgage brokers, homebuyers, homebuilders) would not make such stupid mistakes, only stupid policymakers would use reduced form models to make policy (I leave it to your judgment to decide who is more stupid, policymakers or rational agents or Lucas). Lucas critique and time inconsistency are both critiques of optimal control using engineering type approach by policymakers. What they are pointing to is the feedback loop from future expected policy action to the agent's current behavior. Needless to say, it is trivially correct, but useless insofar as actual policymaking is concerned because most agents (not brainwashed by these critiques) recognize that the world is far too complex and future policy far-too contingent on unknown unknowns (or at least known unknowns).

says...

The "only strategies that have worked consistently and successfully since August 2007" ...

Wow - time tested!!! No chance those models will go wrong at any point in the future.

Rolling regressions, etc. is precisely what is needed when the Lucas critique is operable, so that actually supports the claim. [And the critique is not limited to policy.]

Lucas point was that rational policymakers had, in fact, made exactly that mistake (see the St. Louis equation) and he was warning them of the consequences. So he hardly assumed rationality ruled out such an outcome as you try to say...

[robertdfeinman]

I take exception to the analysis. The models did what was asked of them. The ignored variable was the allowable level of risk. There is nothing new in this. The Bushies decided to use a similar analysis when they turned down the Army Corp of Engineers request for $175 million to strengthen the levees.

The gamble was that the once-in-a-century event wouldn't happen during their watch. New Orleans lost the bet. The same thing is true in the financial field. All those who have cleaned up over the past 15-20 years were correct in their bets. What happened was that the low probability event occurred. Notice that many at the top even during this crash still have walked away with a bundle.

Part of the willingness to ignore highly unlikely events can be traced back to us, the investing public. We have demanded 10-15% yields year after year. This is something which has never been sustainable, but each new generation has to learn this lesson again. This pressure for high returns forced all but the most prudent firms into taking on riskier endeavors. If your firm refused then your customers went elsewhere.

The real lesson to be learned from this is that essential investments like pensions should not be left the whims of the investors. They don't have the expertise and can't afford the risk. The newspapers are littered with stories of retirees who have lost most or all of their nest eggs and now don't know how they will survive financially.

The models underestimated risk because that's what they were required to do.

This isn't the only foolishness from Wall Street. A few people have been questioning the wisdom of firms buying back their own shares (Floyd Norris has been one of the most consistent). Now it turns out that they bought high just as the most naive investors. Municipalities have been unwise as well. Instead of selling fixed rate securities they also engaged in taking on unpredictable risk and many are now in deep trouble.

What ever happened to the concept of the prudent investor?

[Gorobei]

The whole "blame the quants" thing is fundamentally absurd. Wall St does what it wants, and hires a lot of people to make sure it does it efficiently. Quants were just the latest class of support staff.

The main, and probably only, reason this mess happened was the transition of Wall St firms from partnerships to public firms. As soon as you can do 30-year deals, but get paid on a 5-year basis, the game is over. Quants are just window-dressing at that point.

[calmo]

Somehow Richard, this passage in your post...with strong mechanical skills, but not strong high level intelligence, or not using your high level intelligence if it is strong. has etched Arnold on my brain (actually the recent ad inviting All to spend their holiday in California where he is lounging on the beach with his female companion) [Out damn spot. Out...]

It is noteworthy that Arnold, too, had his model (maybe itwas his wife...don't distract me, with my strong effort to erase...). In fact if you don't have a model, you are likely to be dismissed as a person without a brain...and certainly at the beach it goes without saying, (not like this) without a model, you are a loser.
Which 'splains Arnold from the getgo.

Ok, my model (so transparent) is to make this as entertaining as possible...so that I don't burst into tears before I am done (apart from this, I don't do hostage taking).
It is a very loose model...almost rigorously loose. Specifications for identifying elements of the model, for distinguishing operations ascribed to these elements, for limiting the exceptions to those operations, and more...are as optional as clothes at a nudist beach.
Regressive curve fitting is not only about pining for your youthful days of agility, but about impressing others that you know what the hell happened then and ergo, what's about to happen next.
For some models, this is a hard sell.

rdf asks What ever happened to the concept of the prudent investor? JDH, has a recent thread where he illustrates that this specimen is not extinct...so valiant.
How can you be prudent with so many insiders? Big insiders...well, you just imagine them away...if you have the strong high level whatevaitwas.

[Greg Ransom]

In the 1990s the AEA committee on graduate education was calling these guys "idiot savant" -- finding that these guys knew almost nothing of the real world, or of the economic literature more than one or two years. But they were very, very, very good at mathematics, many of them coming out of physics programs.

David Colander has written many very good articles on the serious intellectual limitations of these "rocket scientists" -- and on how the institution of academic economics has come to be overrun with "math jocks" who essentially don't know any economics.

Ask one of the "rocket scientists" who Adam Smith was and what his ideas were, and they couldn't tell you. Ditto Hayek. Ditto Friedman. Ditto economist worth reading over the last 300 years, who hasn't published in the one of the top 5 economic journals in the last 2 years.

As one of the "math jocks" any "are you smarter than a 5th grader" type questions about how businesses work, or about the American economy, or about the American regulatory regime, and studies who the "rocket scientists" can tell you a lot about, well the latest math they've been working on.

The economics profession is a disgrace. And they have no one to blame but themselves.

[teasrini]

says,

Let me address your comments. First of all, thirteen of the most tumultuous months in financial markets is indeed a long time and more importantly testing time. BTW, you obviously don't read the whole comment, where i clearly noted that these strategies have been making uncorrelated excess returns since the early 1980s.

Lucas critique says nothing about forecasting. It only says that using models to make policy would invalidate the model. Forecasters, insofar as they are private actors and have limited market power (price takers), will and should have no influence on rational actors that populate the Lucas world.

yes, i am well aware that the Lucas critique was made in the context of models such as the St Louis model. I never said that it was incorrect. please read my comment carefully. BTW, have you ever tried to check monetary policy minutes during that period? How closely did the Fed ever hew to the St Louis model? When faced with the complex world, policymakers take all kinds of inputs in making "subjective" decisions. And as I said, rational private actors would recognize that and also recognize that in an uncertain world where no true parameters are known and can be reliably estimated (and are constantly evolving in any case), the Lucas critique is minor irritant, a theoretical curiosum.

However, there is way in which the quantitative modeling sowed its own demise and that was foreseen not by the Lucas critique but by Minsky (although he did not talk about quantitaive modeling per se). If you estimate the statistics using a history of a series (says default probability on home loans) and then figure that default probabilities implied in the mortgage rates are excessive, you would go and make more aggressive loans). In a competitive market, others would follow suit. This process itself has changed the distribution of defaults. This is what Minsky termed as stability lead to instability. Last time I checked, there was no widespread agreement about this argument circa 2007, let alone 2005. Show me a lucas model (or a Prescott model) where he thinks actions of private actors in a competitive world leads to such large scale failure.

[Greg Ransom]

Cornell economist Robert Frank has written and talked about how a modern mathematical education taken in the guise of "economics" has left many students unable to think economically.

Listen to Frank's discussion of this topic here:

http://www.econtalk.org/archives/_featuring/robert_frank/

[Bruce Wilder]

rdf: "What happened was that the low probability event occurred."

What happened is that the inevitable event occurred.

To be less cute about it -- what "event" do you think occurred? It seems to me that it was a case of self-poisoning. I wouldn't even say that it was "accidental" self-poisoning, as many of those introducing the toxic financial claims into the system, must have known that the consequences would come with certainty, but just not right away.

And, although we hear this blaming of the quantitative modeling thesis from time-to-time, one rarely sees any hard evidence that the models were not flashing red, as they say. Maybe the quants actually got it right.

G: "The main, and probably only, reason this mess happened was the transition of Wall St firms from partnerships to public firms. As soon as you can do 30-year deals, but get paid on a 5-year basis, the game is over. Quants are just window-dressing at that point."

William D. Cohan: ". . .the inevitable consequences of encouraging smart people to take risks, free of accountability, with other people’s money are easy to fathom. As innovative products emerged on Wall Street, the compensation system pushed bankers and traders to sell them, to inevitable and disastrous extremes.

"It was the gorging on high-yield bonds that led to the Crash of 1987 and to the credit crunch that followed. The feast of Internet public offerings led to the equity bubble that exploded in March 2000. Excessive issuing of debt for emerging telecommunications companies resulted in the frauds at WorldCom and Enron, among others. And, of course, the huge rewards given to bankers, traders and Wall Street executives for manufacturing mortgage-backed securities have led to our current predicament."

". . . where is it written in stone that bankers and traders have to be paid millions of dollars for their services? The gibberish about needing to pay that much just to keep superstars from fleeing to private-equity firms or hedge funds is just another Wall Street myth. The truth is most of them are lucky to have a job at all and they know it."

[Greg Ransom]

The deepest problem of all in economics: The false model of "science" behind the explanatory strategy which uses a fully survey-able mathematical model constructed out of "givens" as the explanans.

Hayek invented the word "scientism" to describe this false explanatory approach to economic science.

The "rocket scientists" are bewitched by a false understanding of science, falsely applied to economic phenomena.

They will keep getting the economics wrong until the give up their false understanding of how social science is done in the domain of economics.

[chris]

Buffett summed it up best (with hat tip to Virgil): "Beware geeks bearing formulas."

[ekzept]

I heartily agree with Gorobei: the idea of somehow blaming this upon "quants" or financial technicians, in the style and spirit of Warren Buffet's interview on *Charlie Rose*, is just silly and absurd. It is a witch hunt. Buffet is trying to deflect hatred away from himself, embracing the "I'm just an ordinary guy" image that he wants everyone to believe. Should the geeks using formulas to create new drugs and new engineering designs and Google be similarly chastised? Buffet and chris are simply engaging in fashionable anti-intellectualism. I doubt many of these critics understand how CDSes were formulated or how they worked. (I'm sure Buffet does, so I don't understand his puzzlement, assuming it's genuine.) They were simply insurance, after all, and can be seen as technical extrapolation of other insurance models (tropical storms anyone?) to financial environments.

In fact, few major politicos or financial influencers want to say what the true causes of the current situation were: American financial success over decades, and the ideological embrace by nearly all folks that home ownership ought to be supported at any cost. The long time success and self-image of exceptionalism meant that investors of all kinds refused to price in risks they should have. This meant they signed on to deals they should have walked away from. The willingness to extend credit, with loose money provided by the Fed, meant that production and commerce ran in overdrive, exceeding natural capacity. Roubini has estimated 7% of GDP is in that category. When credit was withdrawn -- pricked and collapsed by the unsustainable oscillation consisting of CDSes and the like -- this (current) deflationary spiral began. It won't finish until inventories and goods and services and jobs go back to the natural level of economic activity prior to hyperstimulation by excessive credit.

The reason for the economic collapse is more like the inexorable motion of a major hurricane, a system of partial differential equations solving its way into reality, and not at all to be laid at the feet of any particular group. We all own it. Heck, Greenspan said he goofed.

The "credit crisis" and present financial collapse is our own collective system operating in ways beyond our control. To pretend some subgroup wrecked it because they failed to play by some obscure rules is not only misguided, it is silly, dumb, and will make things worse, not better.

Indeed, solutions to the present mess ought to use more market devices not fewer. CDSes were not publicly traded, and weren't subject to the reporting rules that such markets demand. They should be brought into that, giving the private sector a profit motive to reveal, price, and trade them. Only the quants that people so vilify are capable of constructing such an ediface.

But now, given the crumbling sandcastles built on these clouds of credit, we need to hang on while they collapse. We can only build on solid ground. Like the dynamics of the electrical grid, the dynamics of markets are inscrutable.

[Florin]

teasrini:

I am curious why do you (along with so many others, but you in particular seem to be rather well informed on the subject) call these models "statistics"? Looking back at any particular finite series, which is not a representation of any theoretical model, is not statistics, it is faulty logic. Or, in more mathematical terms, incomplete induction - a pretty common mistake, but not one that a mathematician would do (not even an engineer like myself)

[Daniil]

Quants only did what they were told to, and I'm pretty sure they could've ran a bunch of statistical tests some of which would've told them that their model's assumptions are going south and predictions aren't worth a dime, but I'm not sure anyone would've paid attention to that until those models started loosing money.

Having said that, I agree, statistical analysis based on a buttload of data from tranquil times has zero predictive power at the time of financial stress. And using the data from past stresses is not straighforward excersise and the one that requires knowledge of financial history and economics. So Quants had their run, but I'd say they are victims of the crisis, not the cause.

[Barkley Rosser]

I am in general agreement with teasarini. The problem here is associated with the sorts of things that Minsky wrote about, how finanical success undermines itself by changing expectations and reducing the guarding against risk. This can happen even in a world where financial instruments are very simple and markets are not all deeply interconnected. This process described by Minsky is associated with the emergance of spculative bubbles, and Minsky's ally, Charles Kindleberger, made it clear in his _Manias, Panics, and Crashes_, that speculative bubbles have been going on since at least the crash of the Holy Roman Empire currency in 1618 at the beginning of the Thiry Years War. Not a lot of quants around then.

Now, Greg Ransom does have a point in that the degree of damage that can occur when a speculative bubble gets going can be amplified by the degree to which markets are interconnected and the degree to which those in the markets do not understand what is going on in them, which is relevant to teasarini's critique of the relevance of the Lucas Critique for this. Here one could argue that the quants might be as at fault as are the inventors of weapons for how many more people get killed when people are stupid (or evil) enough to engage in wars. The global speculative bubble in housing is in some sense at fault ultimately for the current collapse, although the amount of damage its collapse has caused has been amplified by the degree of entanglement among highly complicated markets, piled on top of each other in Minsky-style Ponzi pyramids, as the layers of derivatives markets have been.

Even here, focusing too much on the quants may be misleading to some extent. Are they responsible for having invented or demanded the development of these higher level derivatives, such as credit-default swaps or the reinsuring of them by AIG? Or are they merely the "technicians" who designed the way they these things were handled? They may deserve some blame here, although I think that it is the owners and managers of the firms who are at least as to blame for buying into using these markets, whose complicated instruments were then developed by the quants.

I would remind everyone that experiments by Vernon Smith and others have shown that people have a strong propensity to engage in speculative bubbles, even in very simple and clear markets. This is the real source of the problem, even if the damage arising from this may be made worse by the nature of the instruments they are using for their speculative frenzies.

[reason]

Look these "quants" were taking home pay in a year they would have earnt in a lifetime as a physisist. Why blame them for doing precisely what somebody paid them to do? To think that any individual should feel responsible for systemic risk is just idiotic. That is what the Fed and the various regulatory bodies are for. I take it the guy who wrote this thinks unregulated markets work and wants a fall guy. That just won't do, the whole idea of an economic system is compartmentalisation. Yes somebody needs an overview, but he is looking in the wrong place.

[ken melvin]

I wonder if Padilla of the Orange County Register has sold the movie rights to his book. I assume so and look forward to seeing it. The one showing the inter workings of the hedge funds should be very good, too. So far, strangely, seems the buck doesn't stop with the financial managers. Maybe they could dig up Flip for the part of a manager and Jack Nicholson could play the devil (Quant).

[ken melvin]

Outstanding comments, all. I am humbled afore ye.

[ken melvin]

Calmo - I once saw a group calling themselves the LA Times Book Review attribute Arnold great intellect and leadership, so I'm sure that it is so. These modern mediums are truly our greatest source of knowledge. One we should all appreciate.

[paine]

i think articles like this give quants both a bum rap
and us a bum steer

the math models were pure fraud

the profiteers drew in suckers with
a framus wignickler
a double talk flim flam
built out of greek letters
and ivy covered bare asses

these were looters mom

kold blodded looters

not punch drunk rain dancers

who fooled who here ???

surely not the quants

and check out the insiders take home over the last decade or two ..
they got fooled ???

we the nugget sized investor-cules
we of the house lot fool's gold mines
and the 401k mushroom farms

we were the punch drunk rain dancers

they were the guys stealing the collection plate

[calmo]

They were outstanding...and she was outstanding in her field too...
Such an intro, ken, but I am pre-distracted by teas' comment that I just know I am standing outside of: And as I said, [repeatedly overanover before, but without emphasis like this here] rational private actors would recognize that and also recognize that in an uncertain world where no true parameters are known and can be reliably estimated (and are constantly evolving in any case)[oh, you cannot fool me, we all know some actors who do not respond to evolution. They don't.], the Lucas critique is minor irritant, a theoretical curiosum. Curioswhat? A small very squashable bug of insignificant proportions no matter the power of the lens you use...beneath further inquiry save it's mysterious attackability...very damn curious, you know? Here it comes again: bloody curiosumisms...B gone!

[D_rumsfeld]

As many other posters have noted, the blaming of the physicists and mathematicians is foolish. That is like blaming the MPs at Abu Ghraib.

Physicists developing these models are smart people doing what they were asked to do. And I can assure you that in their documentation they noted that these models will fail when x, y, or z assumptions do not hold.

However, physicists were not put in charge of the trillions of dollars by themselves, nor were they likely the supervisors in these operations who were ultimately responsible. Look to the people who were paid most handsomely during the boom and you find the people whose job it was to manage the direction of the company. If they weren't prudent in their operations and use of their technical modelers, then it was a failure of their leadership and understanding of markets.

[kthomas]

D_rumsfeld, you are right.

However, accountability is not something we hold dear in America anymore.

[dirtyal]

Three fundamental precepts of "risk management":
1) Don't risk more than you can afford to lose;
2) Don't risk a lot for a little;
3) Consider the odds.

The big problem with the "melt down" was that unregulated issuers of CDS's wrote them with great abandon. If there had been a means test, the people who wrote them would actually have had the assets to back them up. In my view, this is a classic bubble akin to the great speculative bubbles of the past. People betting their entire fortunes by buying stocks on margin with 10% down.

If you think about what the writers of the CDS's were getting for their risks, it was not nearly enough. (in retrospect) They were, again in retrospect, risking quite a lot for a little.

And, getting back to the quants, they certainly did not correlate the risk with the pay back.

I'm tending to agree with those who say that the quants were just justifying the case for the salesmen. But who let the guys writing the CDS's write so many of them in the first place? And why didn't people like S&P and Moody's figure this out--talk about this not being "rocket science".

says...

Financial markets? Git what you can git, then git.

Grandpa's banker remembered the great depression and enforced strong banking regulation, like capital requirements, strict lending practices, low leverage, central counterparty clearing.

Prudent banking. Reduced systemic risk. It is not novel, it is not rocket science.

says...

Look I'm absolutely sure that the "quants" knew that what they were being paid vastly exceedly the social value of what they were doing. And that could only mean either it was a zero sum game (and somebody was losing big) or it was a ponzi scheme and somebody was going to lose big. But they got families to feed! (Some of them anyway - and the others might have in the future, and hell git when the gittins good!)

[reason]

oops that was me.

[anne]

"That is like blaming the MPs at Abu Ghraib."

Quite a bizarre and especially offensive analogy.

[Joen]

Funny how I've read many times in this blog that economists are the bad guys because our models suck at reproducing the real world, while physicists are the good guys because their models rock! Well now we have physicists trying to build economic models and turns out they suck even more because they don't even know where decades of economic modeling has failed and succeeded. What people should know about modeling economics and physic. All particles of a kind behave in the same way, their behavior might be random but it is generated by the same distribution. All agents in the economy are unique in their own way. People can behave "irrationally" if they choose to just to prove you wrong.

About the quality of the quant's models ... well any portfolio with a lot of leverage will do great when things go north and will suck when things go south.

[a]

It strikes me as just a little ironic that economists are claiming that quants hew too closely to their models.

The Wall Street Journal had a decent article a couple of weeks ago on what went wrong at AIG, called "Behind AIG's Fall, Risk Models Failed to Pass Real-World Test". I can't find it any more but what it said was that the quant who constructed the model, built it to AIG's specifications. The problem was that AIG (I'm not sure *who* at AIG) didn't ask him to model the risks which proved to be the ones which broke the bank. "... AIG didn't anticipate how market forces and contract terms not weighted by the models would turn the swaps, over the short term, into huge financial liabilities." This isn't to say that the models were correct; maybe they were also bad. But, if you believe the WSJ, the immediate problem was that the models were specified in such a way as to not include the relevant risks.

All models - financial, economic, climatology - can suffer from the problem of missing parameters. In order to simplify, only some parameters are put in; many have to be left out. It's a general problem of modelling, and I wish everyone who used models (and that includes economists) would just be a little skeptical about their model's applicability. E.g. before offering a solution of spending 600 billion dollars as a Keynesian stimulus to a country which is already too much in debt.

[Greg Ransom]

My understanding is that most of the "bosses" didn't understand the models that the "rocket scientists" were using.

That's what the reporters are telling us. Who knows if its true.

[BR]

teasrini's comments are interesting.

"If you estimate the statistics using a history of a series (says default probability on home loans) and then figure that default probabilities implied in the mortgage rates are excessive, you would go and make more aggressive loans). This process itself has changed the distribution of defaults."

I'm not an expert -- but here is the problem I see with this. There are no non-linear cutoffs (such as houses are not affordable relative to incomes in aggregate) and there is no imagination of all possible histories instead of just those which occurred in the past. Contrast this with the catastrophe bond modeling approach that Mark Thoma referenced in one of his earlier posts:
-----
The models these companies created differed from peril to peril, but they all had one thing in common: they accepted that the past was an imperfect guide to the future. No hurricane has hit the coast of Georgia, for instance, since detailed records have been kept. And so if you relied solely on the past, you would predict that no hurricane ever will hit the Georgia coast. But that makes no sense: the coastline above, in South Carolina, and below, in Florida, has been ravaged by storms. “You are dealing with a physical process,” says Robert Muir-Wood, the chief scientist for R.M.S. “There is no physical reason why Georgia has not been hit. Georgia’s just been lucky.” To evaluate the threat to a Georgia beach house, you need to see through Georgia’s luck. To do this, the R.M.S. modeler creates a history that never happened: he uses what he knows about actual hurricanes, plus what he knows about the forces that create and fuel hurricanes, to invent a 100,000-year history of hurricanes. Real history serves as a guide — it enables him to see, for instance, that the odds of big hurricanes making landfall north of Cape Hatteras are far below the odds of them striking south of Cape Hatteras. It allows him to assign different odds to different stretches of coastline without making the random distinctions that actual hurricanes have made in the last 100 years. Generate a few hundred thousand hurricanes, and you generate not only dozens of massive hurricanes that hit Georgia but also a few that hit, say, Rhode Island.
http://www.nytimes.com/2007/08/26/magazine/26neworleans-t.html?pagewanted=1

[swells]

It is posts and threads like this that keep me coming back here. So many fruitful avenues for further exploration. And, to those decrying a "scientific" approach, it is always the case, in all fields and at all thimes, that if the only tool one knows how to use is a hammer, every problem starts to look like a nail.

[Barry]

"As soon as you can do 30-year deals, but get paid on a 5-year basis, the game is over. Quants are just window-dressing at that point."

Posted by: Gorobei

5-year basis? We wish. Those bonus checks came out (IIRC) at the end of each January, and promptly cleared the bank.

[teasrini]

Joen,

Last time I checked it was the economists who were busy making all "particles" the same--representative agent model, representative firm model, etc. Last time I checked, the theoretical foundations for each of these aggregations was non-existant--so much for micro foundation. So, it sounds kind of rich if you are complaining about physicists doing economics.

In any case, I dont carry a brief for quants. All those quants who drank from the well of modern portfolio theory and quantitative finance, I have only contempt for. The quant that used cupolas to price CDOs does not understand that concept of correlation in financial markets is dynamic.

However, there is a section of quants--call them engineers if you will--that are practical people. They dont get hung up in high falutin talk about micro foundations and critiques. When faced with complex problems they rely locally on heuristics, knowing fully well that the global maxima is a mirage. That is the quant that I respect.

[Barkley Rosser]

Greg,

And to the extent that the quants themselves did not fully understand all the assumptions being made for all the models they were using to hold (which proved not to hold), they did not necessarily know much better than their bosses. Even if they did not know the details of the instruments, the bosses should have known, more or less, the levels of leverage that they were engaging in, and for this they should be held responsible. Perhaps some of them really believed that credit-default swaps provided credible insurance, but if so, well, they should have learned from the "programmed trading insurance" episode of the 1987 crash that such vehicles can come undone when there is a correlated crash and the counterparties can suddenly disappear.

[Cam Hui]

What is less risky (this not a trick question)?
1 To jump out of an airplane at 30,000 feet without a parachute
2 To stay in the plane while someone else jumps out (yes, there is a pilot flying the plane)

The correct quant answer is 1. Jumping out of the airplane is less risky because the standard deviation of the outcome is zero.

See http://humblestudentofthemarkets.blogspot.com/2007/12/surviving-and-prospering-as-quant.html

[Barkley Rosser]

Well, heck, there is that old joke about the ratex economist who reports having gone deer hunting with his buddy. He claims they killed a deer, but do not have one to show for it. How come? Well one of them shot two feet to the right of the deer while the other one shot two feet to the left of the deer, which means that on average they hit it.

[Keith]

"They dont get hung up in high falutin talk about micro foundations and critiques."

And those guys who don't get hung up on high-falutin talk are the ones who blew it. It's those "practical" people who didn't model the behavioral implications of no-money down, stated income, no-income no-assets loans, neg-am loans, how they affected buyer incentives, and how that fed back into price volatility.

"Structural behavioral models of the economy that are useful in forecasting and making money? Can you name one?"

Structural behavioral models are the models that would have saved people a lot of grief if their models had been used.

If you use a simple forecasting model against my structural model, you will beat me, month after month after month...until one month, you will be catastrophically wrong and I will still perform okay. You will be catastrophically wrong because your model did not incorporate structural behavioral features, so when cetain fundamentals changed, you failed to see their implications, because your model does not have behavioral implications.

So-called "black swans" are often endogenous. They occur when people use "heuristics" and "what works" and "what's practical" day in and day out and fail to incorporate theory. Then, one day, disaster strikes, and they have no idea why.

[calmo]

Why do I think Barkley hunts alone?

[Eric Dewey, Portland, Oregon]

As noted by others above, this is an outstanding group of comments, on a critical topic.

There's at least one more problem here though, and one that at least partially absolves the quants, while placing more responsibility on their bosses.

If we grant that, in spite of their incredible successes, and for whatever reason, the models missed something that can be said to have "caused" this meltdown, the next question is why is it taking so long to get clarity on who's holding the bag? Because 18 months into this thing we STILL DON'T KNOW enough about how the error multiplied so quickly.

Having a central, regulated market would have helped - but why aren't we blaming the lawyers who advised the bosses to sign off on these contracts when you couldn't even tell who your counterparty was or perform adequate due diligence on their true financial condition?

(After all, lawyers are always a good target to shoot at...)

[Patricia Shannon]

I thought the reason CEO's & other executives are making so much is because they know what they're doing.

[Jrossi]

I find it hard to believe most Quants were so stupid that they actually believed their models. These are bright people and must be aware that models are not reality. They did what they were paid to do. Says said it right. Git what you can git, then git. When it's other people's money, what the hell.

[Dipper]

this is just wrong.

1. Quants are well aware of tail risk, and possess the capability to model it.

2. If one deal explains why Lehman's went bust, it was a straightforward property deal, not a set of derivative deals.

3. Banking crises are as old as banking itself, and existed well before derivatives were invented or traded. All have a common root - lending too much money to people who can't pay it back.

[zerobeta]

I think it's important to differentiate between a quant presence and a quant obsession on Wall Street. Like any other "edge" a trader (or trading firm) may have, it's profitable until its not. When the quants came to Wall Street, they brought a unique outlook and skill set that they applied brilliantly (and successfully) for some time. As more quants moved in, they unfortunately found out that the Uncertainty Principle and the Observer Effect could also be observed in both quantum mechanics and financial markets.

Quants were what sold financial products, so Wall Street hired more of them. The more Wall Street hired, the more they devalued them all.

Imagine if the scene weren't Wall Street, but Fermilab - where instead of a group of physicists, there are hundreds or even thousands of them consecutively performing a similar experiment with the idea of publishing their results. They all have the same laws, basically similar data, and most have similar hypotheses they wish to test. At what point would adding the next marginal physicist to the queue detract from the experiment and quantum physics as a whole? Does the Nth physicist take into account the fact that N-1 physicists have used the particle accelerator before him? Does the queue that is visibly forming behind the first physicist cause him to act with too must haste, resulting in poor data? Do they all rush to be the first to publish - substituting a desire not to perish with scientific rigor?

When you put enough brilliant quantitative minds with little to no financial market experience (which was actively sought by many shops), basically the same information and skills and then give them enormous financial rewards to take concepts you don't fully understand and apply them to concepts they don't fully understand, you are looking for trouble.

The result was way to much adherence to the models. By throwing obscene sums of money at these people before they even make a trade you are telling them that their knowledge and ability to apply the laws of quantum physics makes them valuable and will make them successful. Why stray from those laws and think "outside the box"? When the shit hits the fan who better to blame then Einstein?

The result - too much money on the same trade. If you have a group of quants with similar models and data as well as very little incentive to deviate, they are bound to get similar results. The war will be won based on the precision of their tools (i.e. the huge technology expansion on Wall Street), and clever forms of leverage to counteract dwindling risk spreads.

Thus adding more quants is a self reinforcing cycle. The more you add, the better it makes the early ones look, thus causing you to hire even more - until you have what you have an outsized and what you believe to be a market neutral, or risk-free position, but you only "know" its risk-free because the quants tell you it is. All along you are spending tons of money on technology and leveraging yourself to make quicker estimates so that you can front run the other quants orders with the most capital possible - both have diminishing marginal returns.

The quants used leverage and technology to turn the market into their own video game and the models the quants used essentially become the rules or the program of the game. For a while, the way to win the game was to play by the rules, and play the fastest and hardest. The mistake that they all made was to ignore the possibility that the rules could suddenly change.

[Barkley Rosserr]

JRossi,

Lots of people get used to fooling with formuli that are ultimately based on some assumptions and forget about going back and looking at the assumptions or worrying about them. If the formuli have been working so far, that is what counts, and most quants, especially those with a physics background, are strongly empirically oriented. What works is what matters, and theory is for abstract losers.

Dipper,

There are many ways to model tail risk, with many being insufficient. How much tail risk and how to price it? Linear Gaussian binomial copulas? Student t distributions? ARCH/GARCH models? There are many poisons in the pot that do not fully deal with the most extreme of these events.

[Isabel]

Wondering around, as usual, in places where the conversation goes way above my head, I've just read in Brad Setser's blog (waaaaay above my head), these comments by commenter Twofish that seem strangely appropriate. Or maybe totally off topic:

# November 11th, 2008 at 7:28 pm Twofish responds:

Patrick writes: (CDO papers) I really wonder if a clever grad student could handle the math, never mind a clever high school student. But, I’ll admit that I might be missing something.

The basic question that a CDO model tries to answer is “assuming one company defaults, what is the likelihood that a lot of them defaults.” So all those models ask “given this sort of corrrelation between defaults and given this recovery rate how much money is this CDO worth?”

That’s it. If I assume that people won’t default and I can get lots of money from them when they do, CDO’s are worth a lot. If I assume that when one subprime borrower goes bad, they all will, and that I won’t be able to get any money from them, then CDO’s aren’t worth very much.

The silly thing about these papers is that if you go to page 8 of the paper on nested Archimedian couplas, you see “In what follows we assume an identical deterministic recovery rate R for all companies…”

On page 12 “The constant recovery rate is chosen as R=40% for all firms, a commonly accepted assumption.”

So what you build into the model is the assumption that you can get R back if a company goes under. Suppose R is 40%, this means that with this model, you will never lose money on a super senior CDO. If you assume that you are guaranteed to get back 40%, then it’s perfectly safe to hold CDO’s that are funded by the first 10% of the money that you are getting, because you’ve assumed that you will never lose more than 40%.

At this point you may ask the important question “what happens if I can’t get back 40%?” and the even more important question “do I lose my job if I ask that question?”

# November 11th, 2008 at 7:40 pm Twofish responds:

Rickstersherpa responds: how hard it is for a man to accept a fact when it is his or her interest to pretend it does not exist.

It’s actually pretty easy for individuals to accept facts, the hard part is trying to figure out what to do about things.

Put yourself in the role of someone in 2005 that figures out that CDO models are bogus. What do you do? If you have a well run bank with competent senior management then you point this out and the bank doesn’t invest in CDO’s. Suppose you will get a much smaller bonus for asking questions?

What do you do? Go to the regulators, who believed in 2005 that markets can do no wrong? What about the media? Yeah right. Pointing out that the real estate bubble is a bubble isn’t going to make you popular with *anyone*.

You probably do what most people will do, stop asking questions and maybe start looking for another job.

[Keith]

"Put yourself in the role of someone in 2005 that figures out that CDO models are bogus. What do you do? If you have a well run bank with competent senior management then you point this out and the bank doesn’t invest in CDO’s. Suppose you will get a much smaller bonus for asking questions?

What do you do? Go to the regulators, who believed in 2005 that markets can do no wrong? What about the media? Yeah right. Pointing out that the real estate bubble is a bubble isn’t going to make you popular with *anyone*."

You got to work for a hedge fund, short these guys who are invested heavily in CDOs, make a ton of money off of that, and get dragged in fron of a Congressional Committee who blames you for everything.

[anne]

"You go to work for a hedge fund, short these guys who are invested heavily in CDOs, make a ton of money off of that, and get dragged in front of a Congressional Committee who blames you for everything."

Where is the precise reference to what I take as simple meanness?

[anne]

There was open and substantial hedge fund investing anticipating the collapse of the housing-mortgage bubble, and there has been not the slightest criticism to my knowledge.

[calmo]

Consider the isolation (the separation of shareholders generally, who are at the mercy of a few insiders) of the traders of CDOs from the companies whose credit is the source of their bets. These traders are not investing their leveraged resources in anything but their own bet.
It is not a business investment (in anything other than the HF's bets). No additional funds are available to the non-financial company that is the host of the leeching trader.
Ok, that's enough considering.
Contrast the model of handing real dough over to participate in the growth of that particular company's stock price and stock dividends, based on P/E projections --and held for more than a few minutes/hours/days...depending on co-bets, volatility, etc. [Cromagnons of the 60s with their monthly charts of bows and arrows, yes?]
The HFs, not real pedestrians (--men not afraid to be seen walking in public), are in a much faster lane than this, yes? And seriously, you people, (kudos to Kasriel), this is a very private view of capitalism.
Ok, that's enough contrasting.

So Larry, the banks "facilitate" these trades levering gobs for the HFs and prolly participating (when it's not regulated, it's not regulated)...shrinking the amounts available to the non-financial businesses. Business investment further impacted by the Lehman example of what happens when you don't have sufficient cap reserves to survive the "consolidation". (The possibility of HFs actively contributing to the default of those companies whose credit is less than assured, I leave to real savants professionals.)
Ok, enough of Larry and failure to get "lending" basics...which way did Isabel go, people?

[ken melvin]

I remember what happens to well run banks. They get bought out by those more leveraged.

[calmo]

ken...bud...I meant no harm in misidentifying you as 'no professional'.
...anyone can slip, see?
Ok, back to specializing...

[Eric Dewey, Portland, Oregon]

This seems to be pertinent to the discussion - if anyone's still here...

Paul Wilmott's Blog
Actuaries Versus Quants
Posted At : November 17, 2008 9:59 AM | Posted By : Paul Wilmott http://www.wilmott.com/blogs/paul/index.cfm

The following article was written in August 2008 for The Actuary magazine. I was reminded of it by the responses to our Name and Shame Blame Game.
*******************

Those working in the two fields of actuarial science and quantitative finance have not always been totally appreciative of each others’ skills. Actuaries have been dealing with randomness and risk in finance for centuries. Quants are the relative newcomers, with all their fancy stochastic mathematics. Rather annoyingly for actuaries, quants come along late in the game and thanks to one piece of insight in the early ‘70s completely change the face of the valuation of risk. The insight I refer to is the concept of dynamic hedging, first published by Black, Scholes and Merton in 1973. Before 1973 derivatives were being valued using the “actuarial method,” i.e. in a sense relying, as actuaries always have, on the Central Limit Theorem. Since 1973 and the publication of the famous papers, all that has been made redundant. Quants have ruled the financial roost.

But this might just be the time for actuaries to fight back.

I am putting the finishing touches to this article a few days after the first anniversary of the “day that quant died.” In early August 2007 a number of high-profile and previously successful quantitative hedge funds suffered large losses. People said that their models “just stopped working.” The year since has been occupied with a lot of soul searching by quants, how could this happen when they’ve got such incredible models?

In my view the main reason why quantitative finance is in a mess is because of complexity and obscurity. Quants are making their models increasingly complicated, in the belief that they are making improvements. This is not the case. More often than not each ‘improvement’ is a step backwards. If this were a proper hard science then there would be a reason for trying to perfect models. But finance is not a hard science, one in which you can conduct experiments for which the results are repeatable. Finance, thanks to it being underpinned by human beings and their wonderfully irrational behaviour, is forever changing. It is therefore much better to focus your attention on making the models robust and transparent rather than ever more intricate. As I mentioned in a recent wilmott.com blog, there is a maths sweet spot in quant finance. The models should not be too elementary so as to make it impossible to invent new structured products, but nor should they be so abstract as to be easily misunderstood by all except their inventor (and sometimes even by him), with the obvious and financially dangerous consequences. I teach on the Certificate in Quantitative Finance and in that our goal is to make quant finance practical, understandable and, above all, safe.

When banks sell a contract they do so assuming that it is going to make a profit. They use their complex models, with sophisticated numerical solutions, to come up with the perfect value. Having gone to all that effort for that contract they then throw it into the same pot as all the others and risk manage en masse. The funny thing is that they never know whether each individual contract has “washed its own face.” Sure they know whether the pot has made money, their bonus is tied to it. But each contract? It makes good sense to risk manage all contracts together but it doesn’t make sense to go to such obsessive detail in valuation when ultimately it’s the portfolio that makes money, especially when the basic models are so dodgy. The theory of quant finance and the practice diverge. Money is made by portfolios, not by individual contracts.

In other words, quants make money from the Central Limit Theorem, just like actuaries, it’s just that quants are loath to admit it! Ironic.

It’s about time that actuaries got more involved in quantitative finance. They could bring some common sense back into this field. We need models which people can understand and a greater respect for risk. Actuaries and quants have complementary skill sets. What high finance needs now are precisely those skills that actuaries have, a deep understanding of statistics, an historical perspective, and a willingness to work with data.

[Barkley Rosser]

Eric,

So, who says the Central Limit Theorem holds in this bizarro world of current day finance? Sorry, but the actuaries are just dragging in even more dead statistics than were assumed by Black and Scholes. Not a solution at all.

BTW, I just read the original piece in Sci Am, where it is pointed out that the real fault is the SEC's. Apparently in 2004 they removed previously existing limits on debt and capitalization (since reinstituted), which allowed the quants to go crazy. Oh well.

[MotorCityMan]

Some people say it was not so much a software problem, but rather a data problem: the models were (inevitably) based on "historic" data; data about 'normal' (low risk) mortgages, not the high-risk mortgages they were dealing with then.

It may be that the software itself was build on wrong presumptions, but even if the software would have been (or is) perfect, it would have yielded wrong results because they put in the wrong data.

[Walt]

With all due respect, models that price mortgage securities by using economic first principles do terribly. They come nowhere close to the correct rates of default and prepayment. Investors live in a reduced-form world.

[Saionce162]

The Quants are not the responsible party members in this exchange. Pointing fingers does not do any good but if one were so inclined ... its the sales force that undertook our positions and orchestrated request after request which lead to our over leveraged positions.

Scenario: SalesPerson1 wants to harness new_client1 with a "product". New_Client1 wants lower price, higher yield and feels they can get it from a competitor. SalesPerson1 wants the "business" and provides requirements to Quant1 to "construct" a "product" to appeal to new_client1... (time lapses here, very short being as time is $$) ... Quant1 returns with a "product" to SalesPerson1 and explains that there may be risks that are opaque due to the time constraints in which to "construct" this "product". SalesPerson1 wants the kill and pushes the product mercilessly onto new_client1, new_client2, new_client3, ...new_clientX and everyone lives happily ever after ...