Ed Glaeser says that if people were as smart as he is, they would have realized housing price increases were unsustainable and there wouldn't have been a housing bubble:
In Housing, Even Hindsight Isn’t 20-20, by Edward L. Glaeser: ...[Is] the housing market ... starting to hit bottom? ... One major point of economics is that predicting asset prices is extremely hard... Moreover, the last seven years should make everyone wary about predicting housing price changes. ...
The housing price volatility of the last six years has been so extreme that it confounds conventional economic explanations. Over a four-year period — from February 2002 to February 2006 — the Case-Shiller index increased ... about 50 percent in constant dollars.
Certainly, those price increases cannot be explained by increases in average income. Income growth was quite modest from 2002 to 2006. Nor can the boom be explained by a dearth of new housing supply. Construction rose dramatically during the boom...
A number of pundits place the blame for the bubble on ... Alan Greenspan. They argue that loose monetary policy caused housing prices to rise. While lower interest rates are correlated with higher prices, the relationship is far too weak to explain the price explosion that America experienced. ... To get a 50 percent real increase in housing prices, real interest rates would have had to decline by more than ...10 percentage points..., which is not what happened. ... Real rates actually rose slightly between 2002 and 2006.
While low interest rates, on their own, cannot make sense of the bubble, perhaps the increased availability of credit to subprime borrowers has more explanatory power. ... Yet the correlation between housing price growth and subprime lending across markets is as likely to indicate that lenders took more risks in booming markets as that those risks caused markets to boom. ...
The most plausible explanations of the bubble require levels of irrationality that are difficult for economists either to accept or explain.
For many years, the creators of the housing index, Chip Case and Robert Shiller, have argued that housing bubbles were fueled by irrationally optimistic beliefs about future housing price appreciation. More recently, Monika Piazzesi and Martin Schneider have documented the rise in optimistic beliefs about housing price appreciation over the recent boom. Using some elegant algebra, they suggest that overly optimistic beliefs could cause a boom even if those beliefs were held by only a small share of the population.
It is hard to argue with this view. The only way that anyone could justify spending bubble-level prices in Las Vegas was by having the incorrect belief that those prices would increase.
I once thought that the Las Vegas housing market was so straightforward (vast amounts of land, no significant regulation) that no one could be deluded into thinking that prices could long diverge from construction costs, but I was wrong. I underestimated the human capacity to think rosy thoughts about the value of a house.
Yet even if ridiculously rosy beliefs are a major part of bubbles, we cannot say that we understand those bubbles until we understand the sources of such beliefs. Economists like to link beliefs to reality, but these views weren’t grounded in sound statistics. The housing boom was a great wildfire that spread from market to market, but it is hard to make sense of its flames. ...
I don't think people believed that housing prices would never, ever go down, what they thought is that housing prices would go up in real terms, on average, over time - that housing was a good long-run investment. They knew there would be variation around that trend, but they expected the variation to be relatively mild, they didn't expect the severe variation in prices and associated problems that actually occurred.
But as Shiller argues, the belief that real housing prices rise over time is false, the evidence suggests that real housing prices are relatively flat over the long-run. Because people expected prices to rise on average when they should have expected them to remain flat, the correction - the variation in prices - was far larger than anticipated and many homeowners weren't able to simply ride out the short-run variation like they thought they would be able to do.
But this still leaves a question unanswered. Why did people have this false belief about the long-run trajectory of prices? Shiller explains that this happened because people believed that both land and building materials were becoming relatively more scarce over time, a belief he says is false, but that just pushes the "but why did they believe that" question back one step from housing prices to the prices of land and raw materials.
So let me take a quick stab at an explanation (I'm not pushing this, it's just a quick thought). People are told (or were at that time) that stock markets are a great long-run investment. If you have the time to ride out the short-run fluctuations you can earn 8% per year. Just dump your money in an index fund that duplicates the market portfolio, and forget about it until many, many years later and you will do fine. Risk adjusted real returns on assets ought to equalize across markets through arbitrage, so shouldn't housing yield a real return similar to stocks (adjusting for risk)? Shouldn't there be a real return on housing just like in stock and other asset markets, and if so, doesn't that mean real prices will rise on average over time? This still requires beliefs about long-run prices at odds with (Shiller's) evidence though.
One more note. I may be wrong to assert that people thought that housing prices would rise forever. If you know that there is a bubble in an asset market, but you believe you can sell fast enough once the market hits a turning point to still make a profit, or at least not lose much in any case, then you may be willing to make an investment that tries to exploit the short-term surge in prices. But while I think that may apply to stock markets, or other markets where assets can be sold quickly (the belief that is, the reality is quite different when everybody tries to sell at once), I'm not sure this applies to housing where sales can be notoriously slow. But it's still possible that people would know there is a bubble in housing prices, but still be willing to make an investment because they believe that housing prices would fall so slowly that, if necessary, they could sell their house before taking a loss. It just doesn't seem to me that this explanation works as well in housing as it does in stock markets.