## Monday, November 27, 2006

### Varian: Observe, Theorize, Measure, and Test

Here's one last post, for now, related to methodology. This is Hal Varian from a 2002 New York Times article:

Observe, theorize, measure, test and don't overlook what goes wrong. Nobel experiments, by Hal Varian, New York Times, October, 2002: You can learn a lot when an experiment goes wrong.

Edward Chamberlain, a professor of economics at Harvard in the 1950's, pioneered the concept of ''monopolistic competition,'' a hybrid of the pure monopoly and pure competition models that were then the staple of economics courses.

To demonstrate the problems with those theories, he would run an experiment in his classroom in which students were given hypothetical values of consuming or costs of supplying a good, and they would wander around the room trying to make deals with one another.

Mr. Chamberlain would tally up the results on the blackboard and point out that they were far from the predictions of received theory.

One student, Vernon L. Smith, was much impressed with the experiment. In 1955, while teaching at Purdue, he decided to run a classroom experiment of his own. He couldn't remember exactly how Mr. Chamberlain had set up his experiment, so he used the rules governing trade on the New York Stock Exchange.

In the simplest version of this sort of game, the class is divided into two groups, buyers and sellers. Each buyer is given a ''value'' for the good being sold, and each seller is given a ''cost.'' Suppose, for example, that there are three buyers with values (1, 2, 3) and three sellers with costs (1, 2, 3).

Theory predicts an equilibrium price of 2, and either one or two units of the good being sold. (At a price of 2, one buyer and one seller are indifferent about transacting, which is why the quantity is indeterminate.)

The actual experiment Mr. Smith ran was similar, but had more buyers and sellers with different prices, so the outcome was not easily predictable.

Much to Mr. Smith's surprise, the classroom experiment quickly converged on the price predicted by simple demand and supply. He thought there must be something wrong, so he tried again. The same thing happened.

Mr. Smith continued to do these in-class experiments for several years and found that he almost always got the price that theory predicted.

His colleagues weren't impressed: games in the classroom? What could the students learn from games? What could economists learn from experimental markets?

But Mr. Smith persisted in his research and published his first paper on experimental economics six years later. Two weeks ago, he was awarded the Nobel in economic science for his pioneering work.

Experimental economics has had a huge impact on economics, in both theory and practice. Researchers have gone on to examine a variety of social institutions using experimental methods, and today it is firmly established as a legitimate subject.

The discipline of economics, like other sciences, rests on observation, theory, measurement and experimentation. Theory is, necessarily, a simplification of reality. But how do you know you've chosen the right simplification? Experimentation is an antidote to misspecification: one often finds that effects that the theory neglects turn out to be important in practice.

The wording of the instructions or minor variations in the rules of interaction can be critical to the outcome. The wise economist will be on the lookout for this sort of anomaly, just as Mr. Smith was back in 1955, for those anomalies are often the source of a deeper understanding.

Daniel Kahneman, who also received a 2002 Nobel in economic science, is a psychologist by training, but much of his research has focused on how people make decisions. Utility maximization, the workhorse of economics, is a particularly simple theory: people make the choices they most prefer.

Most psychologists would argue this is far too simple. If you define ''most preferred'' too broadly, the theory is virtually tautological. If you define it too narrowly, simple experiments show the theory is literally false.

As with experimental economics, the challenge is to understand the anomalous behavior. Random departures from the theory could just be error, but systematic departures are worth studying.

Mr. Kahneman did much of his work with Amos Tversky, another Israeli psychologist, who died in 1996. Their work has also found belated recognition in the economics profession, under the name ''behavioral economics,'' which uses both theory and experimentation to study decision making as it relates to market phenomena.

Consider the concept of ''loss aversion.'' This refers to the fact that you have to pay people much more to get them to part with something than they would pay to acquire it. Look at some of those losers in your 401(k). Would you rush to buy them at their current price? No? Then cold-blooded economic analysis says you should sell them. But as Mr. Kahneman and Mr. Tversky point out, few people actually do so.

Mr. Kahneman and Mr. Smith are something of an odd couple intellectually. Mr. Smith is, for the most part, a believer in conventional economic analysis. Much of his recent work has been concerned with market design: how should we design electricity markets or markets for airline landing slots to achieve efficient outcomes?

Mr. Kahneman, on the other hand, sees economics as working with models of human behavior that are much too simplistic. He argues that to understand such strange behavior as stock market bubbles, it is necessary to have better models of how people make investment decisions.

Mr. Smith has generated bubbles in economics experiments in the lab and would no doubt agree that better models of behavior would be useful. On the other hand, his inclination would be to tinker with the market design to see if such bubbles could be dampened or even eliminated, rather than to seek a new theory of economic behavior.

But despite their different approaches, both laureates agree on their approach to economic science: observe, theorize, measure and test -- and watch out for experiments that go wrong. They might just lead to a Nobel prize.

Posted by on Monday, November 27, 2006 at 01:08 AM in Economics, Methodology | Permalink  TrackBack (0)  Comments (7)