Is the Wide, Wide World of Economics Too Wide?

Is there anything economists won't study? Should there be?:

Is an Economist Qualified To Solve Puzzle of Autism?, by Mark Whitehouse, WSJ: In the spring of 2005, Cornell University economist Michael Waldman noticed a strange correlation in Washington, Oregon and California. The more it rained or snowed, the more likely children were to be diagnosed with autism. ...

[This] soon led Prof. Waldman to conclude that something children do more during rain or snow -- perhaps watching television -- must influence autism. Last October, Cornell announced the resulting paper in a news release headlined, "Early childhood TV viewing may trigger autism, data analysis suggests."

Prof. Waldman's willingness to hazard an opinion on a delicate matter of science reflects the growing ambition of economists -- and also their growing hubris, in the view of critics. Academic economists are increasingly venturing beyond their traditional stomping ground, a wanderlust that has produced some powerful results but also has raised concerns about whether they're sometimes going too far. ...

Such debates are likely to grow as economists delve into issues in education, politics, history and even epidemiology. Prof. Waldman's use of precipitation illustrates one of the tools that has emboldened them: the instrumental variable, a statistical method that, by introducing some random or natural influence, helps economists sort out questions of cause and effect. Using the technique, they can create "natural experiments" that seek to approximate the rigor of randomized trials -- the traditional gold standard of ... research. ...

But as enthusiasm for the approach has grown, so too have questions. One concern: When economists use one variable as a proxy for another -- rainfall patterns instead of TV viewing, for example -- it's not always clear what the results actually measure. Also, the experiments on their own offer little insight into why one thing affects another.

"There's a saying that ignorance is bliss," says James Heckman ... at the University of Chicago who won a Nobel Prize in 2000... "I think that characterizes a lot of the enthusiasm for these instruments." Says MIT economist Jerry Hausman, "If your instruments aren't perfect, you could go seriously wrong." ...

In principle, the best way to figure out whether television triggers autism would be to do what medical researchers do: randomly select a group of susceptible babies at birth to refrain from television, then compare their autism rate to a similar control group that watched normal amounts of TV. If the abstaining group proved less likely to develop autism, that would point to TV as a culprit.

Economists usually ...[cannot] perform that kind of experiment. ... Instead, economists look for instruments -- natural forces or government policies that do the random selection for them. First developed in the 1920s, the technique helps them separate cause and effect. Establishing whether A causes B can be difficult, because often it could go either way. If television watching were shown to be unusually prevalent among autistic children, it could mean either that television makes them autistic or that something about being autistic makes them more interested in TV. ...

Prof. Waldman and his colleagues had such [techniques]... in mind when they approached autism and TV. By putting together weather data and government time-use studies, they found that children tended to spend more time in front of the television when it rained or snowed. Precipitation became the group's instrumental variable, because it randomly selected some children to watch more TV than others.

The researchers looked at detailed precipitation and autism data from Washington, Oregon and California -- states where rain and snowfall tend to vary a lot. They found that children who grew up during periods of unusually high precipitation proved more likely to be diagnosed with autism. A second instrument for TV-watching, the percentage of households that subscribe to cable, produced a similar result. Prof. Waldman's group concluded that TV-watching could be a cause of autism.

Criticism quickly arose, illustrating some of the perils of the economists' approach. For one, instruments are often too blunt. As Prof. Waldman concedes, precipitation could be linked to a lot of factors other than TV-watching -- such as household mold -- that could be imagined to trigger autism. ... "It is just too much of a stretch to tie this to television-watching," says Joseph Piven, director of the Neurodevelopmental Disorders Research Center at the University of North Carolina. "Why not tie it to carrying umbrellas?"

Also, Prof. Waldman's findings do nothing to explain the mechanism by which television would influence autism, a gap that instrumental variables are inherently unable to fill. That's one reason many autism researchers think he shouldn't have publicized his results or made recommendations to parents. "I think this is irresponsible," says Dr. Klin of Yale. "We should not provide clinical advice unless there is scientific evidence to substantiate it." ...

David Card, a professor at the University of California, Berkeley, who has done influential work on the minimum wage, fears that the fascination with the instrumental-variables technique "leads to interest in topics that economists are not particularly well-trained to study."

Those who favor the method say it's just one tool among many -- all of which have flaws -- and is intended to help fill in the picture. ... Prof. Waldman welcomes the scrutiny, saying he hopes his work will also provoke autism researchers to conduct clinical trials. "Obviously this is an unusual thing for an economist to be looking at," says Prof. Waldman. "Maybe I was overconfident. We'll see."

Comments

[sa]

i don't see what the big deal. waldman saw a problem and is trying to solve it using whatever tools and understadning he has at his disposal. on one hand we fret about a lack of interdisciplinary research and on the other we are always ready for turf wars. if waldman's research leads to a new link between autism and TV watching, humanity will be better off for it and waldman might have a shot at the medicine nobel, if not append one more science experiment failure in a long list.

[alphie]

In times of economic uncertainty, when the chances of making bad economic predictions that might damage the careers of the economists who make them rise...economists turn to making predictions in other fields to protect their reputations.

I invite all jazz dancers, pastry chefs and nuclear technicians to scrutinize my theory.

[anne]

"Early childhood TV viewing may trigger autism, data analysis suggests."

Ah, Sasame Street, Sesame Street is the answer; Elmo in particular but Ernie is not nearly as innocent as we might think. Elmo, purply purply Elmo. Me, I was raised with Monty Python and look how I turned out.

[anne]

Irony anyone, anywhere (not to be studied at Cornell)?

http://www.nytimes.com/2007/02/27/washington/27govs.html

February 27, 2007

Child Health Care Splits White House and States
By ROBERT PEAR

WASHINGTON — Governors clashed with the White House on Monday over the future of the popular Children’s Health Insurance Program, an issue that some members of both parties said was as important as money for the Iraq war.

In the session at the White House, when President Bush reported on progress of the war, governors pressed him to provide more money so they could guarantee health insurance for children. In response, administration officials said states should make better use of the money they already had.

Gov. Sonny Perdue of Georgia, a Republican, said afterward, “Health care for children ought to be a priority, irrespective of anyone’s views on the war.” ...

[Ken Houghton]

Expecting the Bush Administrationthat is, the administration of George W. Bush, who as Governor of Texas successfully pushed to limit eligibility of children to CHIPsto help the States is lunacy.

[tom s.]

One thing you can't tell from the WSJ article is how thoroughly Waldman read the literature from other disciplines on autism. Clearly he wants people to read and act on his research - it is to be hoped he read others first.

For me, that's the key because it shows whether or not you respect people studying the subject you are looking at. If you do your homework and still find you have something to say, good for you whether you are an economist or not. If you don't, then you're an academic imperialist.

[Jason Voorhees]

I also don't see the big deal. This does seem to me like turf war mentality. At the absolute worst, the authors have found partial correlations between precipitation and autism. At best, they've discovered a causal link. Even under the worst case scenario, there's something new that could potentially help other researchers understand the causes of autism - especially if it leads to fruitful research. This is how science progresses - experimentation. Even if the one researcher interviewed is correct, and there's an omitted variable linked to precipitation separate from television viewing (like growing mold), isn't this useful? That they can't even give the paper an inch is weird.

Also, regarding Heckman's comments. There was an interview he did with the Fed a couple of years ago in which he opened up by criticizing applied microeconomists (particularly some of his colleagues at Chicago, and though he didn't name names, it was clear he whose work he had in mind) for their fascination with instrumental variables. It's not clear to me, reading this article, that he is criticizing the Cornell study, or if he's simply speaking more broadly about trends in the profession.

[anne]

"Prof. Waldman welcomes the scrutiny, saying he hopes his work will also provoke autism researchers to conduct clinical trials."

Now, then, we will divide up toddlers in homes with and without rain, and with and without Sesame Street or at least with or without Elmo or Ernie or Cookie Monster (eek) and watch which toddlers toddle properly. The new new new clinical trials, coming to a home near you. Please read the above sentence severals time to understand just how nutty this cat must be.

[marcel]

I've not read the original paper (though I pretend to on TV), but it strikes me that all we can be certain of so far is what Waldmann began with: a correlation between autism and precipitation. There are also correlations between TV viewing and both autism and precipitation. Unless the second instrument, cable subscriptions, is not much correlated with precipitation, it does not really provide any support for the TV-autism link. Rather than focus solely on the TV-autism link, it seems to me that the next step is to list variables correlated with precipitation in these areas (WA, OR, CA), and then:

a) explore actual bio-chemical links between autism and variables on the list

b) gather data from other areas where variables on the list are not correlated with either precipitation or each other, and look for statistical connections between them and autism.

Based on this account, the TV-autism link is weak, while the precipitation-autism link is interesting.

[don]

Of course, correlation is not causation, which makes Waldman's conclusions about television watching a bit curious. Similarly, rising planetary temperatures correlated with rising levels of carbon dioxide does not equal causation. In both instances, the correlation may be evidence of causation, but many, many more variables are at play than just the ones under investigation. Waldman's hypothesis is further weakened by the implied, but not proved, relationship between rainy weather and television watching.

Rainy, northern climates also have greater incidences of childhood leukemia than do sunny southern ones. Correlation? Yes. Causation? Perhaps, and perhaps not.

It is known that massive (otherwise lethal) doses of vitamin D3 (the sunshine vitamin) will cause leukemic white blood cells in a petri dish to mature into normal cells (leukemia being the failure of white blood cells to differentiate into mature, disease fighting cells--while at the same time proliferating.) Is the relative lack of sunshine in rainy northern climes therefore causing childhood leukemia (or autism, for that matter)? At best, it is only evidence of a possible causal factor.

That's why Waldman's leap to television viewing as the cause of autism is bad science--he has no direct evidence of causation at all, and no means to test his hypothesis, if you discount his sample size of one with his own child's experience.

There is nothing wrong with Waldman, as an economist, positing and testing a hypothesis that television viewing is correlated to autism, if the investigation is properly done. However, he, like many, many scientists today, refuses to limit his conclusions to the data tested. That is why his study conclusion is bad science and why he is irresponsible for having proposed it.

[just the messenger]

don - Do you know how instrumental variables work? One of their functions is to differentiate causal and merely correlational arguments. There are a ton of potential problems with IV (e.g. the ubiquitous possibility of omitted variables), but I don't think that what you point out applies to the analysis in question.

[bakho]

Leave the epidemiology to trained epidemiologists.

says...

Just a comment: Who are we to judge who should be conducting scientific research?

Waldman's work should be critized based on the quality of the results and the methodolgy used and not on his credentials.

[piglet]

So the particular qualification of economists consists in the ability to calculate a correlation coefficient?

[billyshears]

It really amazes me how people is usually building medieval guilds on different issues. Whether Waldman is a cocky economist, that has never even read a word on autism, doesn't really matter. He is just suggesting a new approach to a subject many other investigators (from different fields) are interested in, and if his data supports him, maybe it deserves a second thought... I don't see hubris on this.

[piglet]

"There's a saying that ignorance is bliss," says James Heckman ... at the University of Chicago who won a Nobel Prize in 2000... "I think that characterizes a lot of the enthusiasm for these instruments."

I get it. Apart from the ability to calculate a correlation coefficient, economists also have that other crucial qualifaction: ignorance.

Please, I am not being mean here. It's the WSJ author that is being mean. Adding to the embarassment is the author's apparent assumption that nobody before Waldman ever had the idea that weather, or watching TV, might influence people. Finally, notice that while the paper in question can be downloaded from the web, it doesn't appear to have been submitted to any peer-reviewed journal.

[James Killus]

Increased precipitation has all sorts of impacts on a lot of environmental variables. There might, for example, be a leaf mold that is more prevalent after rainfall, and that leaf mold might trigger some biochemical response which is linked to autism. Or alterations in the level of groundwater might affect the chemical composition of well water, to a similar effect.

Each of these is unlikely, of course, but so is the televison thing. The premature selection of an explanation is what makes this bad science. What makes it attrocious science is that someone at Cornell (not necessarily Waldman, but whoever okayed the press release) is a publicity hound.

[DRR]

There is a limit to what economics can teach us & help us understand ourselves & the world. There shouldn't be a limit to where that knowedge is apllied, if it's application might prove useful.

[Norma]

This upsets the genes and vaccines camp of causes for autism, but I see the same mind numbing affect on my husband when he watches too much TV. My theory is he's much more susceptible because he watched it as a child. I was in college before my parents got a TV so I've never watched it that much. Blogging on the other hand. . .

[piglet]

"There is a limit to what economics can teach us & help us understand ourselves & the world. There shouldn't be a limit to where that knowedge is apllied, if it's application might prove useful."

Please, what does this have to do with "applying economic knowledge"? He's just calculating a correlation. It is disheartening to see every stupid claim made in WSJ taken at face value. Use your own brains.

[Bernard Yomtov]

I agree with what I take to be piglet's point: that this research has nothing to do with economics. It's simply an attempt to apply a particular statistical technique to autism, and the same thing might just as well have been done by anyone familiar with instrumental variables, economist or not.

So judge it on that basis. My own grasp of IV's is limited, but precipitation doesn't strike me as a good one, for reasons Waldman himself suggests. Cable TV might be better.

[donna]

I suspect it's more that when kids are kept inside by rain and snow they drive their parents crazy, leading the parents to seek medical relief for their children...

otherwise you just throw the kid outside...

[JRossi]

Correlation studies are not generally accepted in medicine because they are unreliable. Of course they can point the way to proper trials. Wikipedia has an accurate article on evidence based medicine for those that are interested.

[anne]

JRossi; my sister and I are sure it's Elmo, purply Elmo, who we are sure did away with Kermit just about the time we were getting into wearing tiaras and kissing frogs. Elmo, it's Elmo.

[bakho]

I could post an online survey asking people if they are unemployed and then report a monthy unemployment rate. This would be unacceptable to economists (NRO excepted) because there are ways to collect information on unemployment that are meaningful and then there are ways that are AFU.

Waldman is trying his hand at epidemiology, not "applying economic methods to medicine". There are methods used to collect valid epidemiological data that require a lot of training and there are methods that any neophyte can dream up that will get results that are AFU. Is this that hard for people with college degrees to understand?

[anne]

Bakho:

Waldman is trying his hand at epidemiology, not "applying economic methods to medicine".

[Precisely, and precisely what several economists have been about recently coming to equally meaningless or absurd conclusions.]

[mcewen]

Causation v correlation! I am so glad that I read 'How to Lie with Statistics' by Darrell Huff several life times ago, or I could go down completely the wrong path.
Best wishes

[reason]

I'm with Donna. Just because there a more diagnoses does not mere there are more instances.

I thought most of the evidence regarding Autism was that it is real brain damage. Three possible causes:
1. Genetic
2. Development problems before birth
3. Disease

Which of these is caused by television watching? Perhaps wet weather could be correlated with deseases of course.

[piglet]

"I agree with what I take to be piglet's point"

I agree with Bernard, and also with bakho and anne | Feb 27, 2007 7:56:33 PM. Thanks you guys.

[Ron]

Epidemiology is far too important to be left only to epidemiologists. Emily Oster's work on the relationship between Hep B and sex ratios is taken very seriously by many public-health researchers and demographers.

[CA alamo Joe]

Autism and "Rules of Research"

Certain “Rules of Autism Research” have been implied to be definitive. Perhaps they are misunderstood or incorrect. “Refrigerator mothers” could be a misnomer with an unfortunate parallel which could be inferred to be valid.

If you designated women who bottle fed their children as ‘refrigerator mothers’ you might be correct. The literature reports that colitis, reflux, ear infection, bed wetting and sleep problems are a risk for those who were bottle fed. In addition early weaning is thought to be an element of infantile autism. Bottle-feeding underlies the ailments associated with regressive autism suggesting that this an area for research which has been neglected. The phrase “refrigerator mothers” should refer to the storing the baby bottles in the refrigerator and not to the attitude of the mothers.

Until the cause of autism is known, no hypothesis should be taken off of the table. In addition, they should be taken scrutinized as though they are not proved. I include “vaccinations are not to be blamed for autism” and “measles does not cause Crohn’s disease or ulcerative colitis” in the list of concepts not proven.

In my opinion, researchers have not searched everywhere. From the internet, “treating sleep apnea also treated my colitis” suggests that finding the cause of sleep apnea may help find the cause of autism.