As I continue to gather thoughts on the speculation question... [i.e. the question of whether speculation is driving up oil and other commodity prices, previous entries: Speculative Nonsense, Once Again, More Speculation, Even More Speculation, Speculation Continued..., An Answer?] [Update: New version of model here]:
Steve Waldman added another piece to the speculation model, so let me show how it fits into the graphical model that is being used to illustrate ideas.
First, let's review the model of speculation and inventory storage. I'll use Brad DeLong's pictures to illustrate. In the left-hand panel of the first picture shown below, the interest plus storage line (which I'll denote as i+s) represents the marginal cost of holding inventory, and this is always positive since interest rates and storage costs cannot be negative (I show how to add interest rate determination to the model here). The expected appreciation line, E=(expected future price - spot price)/(spot price), represents the expected marginal benefit from holding speculative inventory. If benefits exceed costs (E>s+i), inventories will increase, if costs are greater than benefits (E<s+i), inventories will decrease, and when they are equal (E=s+i), inventories will stabilize. Note, however, that inventories cannot be negative.
The spot price, in this case, is determined in the right-hand panel by the flow supply and demand curves for the good (say oil). However, at this spot price, and given the expected future price, expected appreciation (E) is negative. Since the benefit is expected to be negastive, and the costs (i+s) are positive, no inventory is held. [Note for now that, since i+s (interest plus storage) is always positive, if E is negative no inventory will ever be stored in this version of the model. Even if E is positive it has to be large enough to cover the interest plus storage costs before any inventory will be held for speculative purposes.
Okay, now let's generate inventory storage. We need to do (at least) one of three things. First, we could let the expected future price go up shifting E to the right. Second, demand could shift in. Third, supply could shift out.
Taking the first case, an increase in the expected future price, this shifts the E curve to the right. Looking at the diagram above, as E shifts out the intersection of the blue lines in the left-panel moves up and, when it crosses the dotted line showing the spot price, speculators are at the indifference point on storing inventory. This is shown in the next diagram (the orange line is the new E curve after the increase if the expected future price):
Now, if expectations rise even further, then the E curve will shift out even more. As it does, and speculators begin demanding some of the good for storage, the spot price is driven upward (and is hence determined by speculators as the label in the graph indicates). The result, after E is done shifting (and it turns from orange to blue), is the equilibrium below:
Thus, this shows how an increase in the expected future price can cause the equilibrium to move from one where there is no storage to one where there is. So, when higher future prices are expected, we should see both inventories and the spot price rising.
How else could we go from the first to the third diagrams, i.e. from no storage to storage? Go back to the first diagram above. Now, imagine either the demand curve shifting in (demand falling) or the supply curve shifting out (supply increasing). All we need is for the spot price to fall (holding the expected future price constant), and either change will accomplish that. As the spot price falls, the horizontal dotted line in the first diagram showing the spot price will also fall, obviously, and you can see that it will eventually hit the intersection of the two blue lines in the left-hand panel. i.e. it will hit the point of indifference over speculative inventories. Here's how the graph would look for an increase in supply:
If supply goes up even further so that the price keeps falling (increasing the spread between the future and spot price and increasing the benefit), we will again be at the situation depicted in the third diagram where it will pay to hold inventories.
It's easy to see that a decrease in demand works roughly the same.
Okay, now let's add in Steve Waldman's twist. Notice that in every case above where there was inventory storage, E= (future-spot)/spot, was positive. It has to be since the expected gain has to cover both the interest expense (the opportunity cost of the value of the inventory, it could be earning interest if converted to a financial asset) and the actual costs of physical storage. When the future price is greater than the spot price, and E is positive, it is called Contango. When the spot price is above the future price, and E is negative, it is called backwardization. In the diagram above, there can never be storage when there is backwardization since E is negative and will always be less that i+s. It won't pay (in expected terms) to store anything.
Some have then taken an additional step and said that because prices have been in backwardization lately (see the graphs in Steve Waldman's post linked below), for the most part, how can there be any inventory speculation? Steve Waldman shows how this can happen be adding in a "convenience yield," which is basically the option value of having inventory on hand. For example, if you are storing a product and there is a temporary shortage driving the price way, way higher than anyone expected, you might want to sell some inventory now and realize a large profit, and replace it later when the price comes back down. Having inventory on hand allows these kinds of options, and that is called the convenience factor. Let me turn it over to Steve:
The convenience yield, by Steve Waldman: If a commodity is in "backwardization", that is, if futures prices are lower than current prices, does that imply that futures markets are discouraging storage (encouraging disgorgement)? Paul Krugman makes the case, here and here.
I'm going to challenge him with a low-down, dastardly kind of argument. ...I'm going to offer an unfalsifiable hypothesis. Krugman says that futures prices are too low to cause people to withhold physical oil and sell forward, as required to affect spot prices. But whatever forward price curve he shows me, I can posit an invisible "convenience yield" large enough to make hoarding oil worthwhile. I don't even have to be unreasonable about it. Extrapolating from historical data, we see that gently "backwardized" futures prices might be quite sufficient to encourage storage when convenience yields are taken into account.
Despite all of this, I agree with Krugman that futures markets can't explain the recent skyrocketing oil prices. Not unusually, he's been a voice of sanity and reason. ...
What is this "convenience yield"? Is it real? ...
Oil is a "spiky" commodity. Every once in a while, someone really needs it, now, and will pay a premium for immediacy. The market for oil in Cushing, Oklahoma might be reasonably efficient, but what happens when someone in Peoria needs oil today? Opportunity! ...
[S]ometimes oil spikes even on the wider market, so that anybody with physical oil can sell at a high price and while locking in low-priced near future purchases to replenish their stock quickly enough to meet any other contractual obligation to sell. If you estimate the profit you'd to earn from these occasional opportunities, and subtract a bit to come up with a "certainty-equivalent" value for this uncertain income stream, you'll have determined a convenience yield. It shouldn't be surprising that convenience yields are especially high for volatile commodities subject to frequent shortages and price spikes.
When futures markets are well-arbitraged (which might not always be the case!), the future price of a storable commodity is determined by the spot price plus the total cost of storage, defined as foregone interest, plus storage costs, minus any benefit of temporary ownership — the convenience yield! When a storable commodity like oil is in backwardation, that doesn't mean that the markets are predicting that its price will fall. It means there is a convenience yield. And in order to decide whether futures markets are creating incentives to store or to sell physical stuff, you have to estimate the convenience yield.
What Steve is saying, if I interpret correctly, is that the cost of storage is no longer just i+s, i.e. interest plus physical storage costs, it is now i+s-c, where c is the convenience yield. Thus, it's easy to see that if c is large enough, i+s-c can be negative, so that it's now possible for it to be profitable (in expected value terms) to store the good even though the expected appreciation is negative. If i+s-c < E, then there will be an incentive to store the commodity even if the values on both sides of the inequality are negative.
Here's how that would look graphically (the axis labeled "price" is at zero, and i+s-c is to the left of this):
However, as Steve notes, just because backwardization is consistent with inventory storage does not mean that explains what we are currently seeing in commodity markets. As has been noted again and again, there's' no sign of inventories piling up.
On that note, let me end with a question for Paul Krugman from Arnold Kling, and an answer from Jim Hamilton:
A Question for Paul Krugman, by Arnold Kling: Early in 2007, the price of oil was $60 a barrel. Recently, it has been above $130 a barrel. Which of the following does Paul Krugman believe:
(a) market fundamentals justified $60 a barrel then, and they justify $130 a barrel now; or
(b) market fundamentals justified a much higher price in 2007?
...We know that Krugman does not believe that today's oil price is out of line with fundamentals. Krugman's view, in effect, is that if speculators artificially boost the price of oil, then supply will exceed demand, and the excess has to go somewhere. Where are the inventories?
This view ought to hold in reverse. If speculators artificially kept the price of oil too low early in 2007, then demand should have exceeded supply and inventories should have vanished. Yet they did not. So is Krugman forced by his model to conclude that the price of oil of $60 also reflected fundamentals?
My view is that inventories are not a reliable indicator of supply-demand balance vs. speculation. Inventories, and the futures-spot differential, also reflect interest rates and "convenience yield" (or option value).
Steve Randy Waldman writes,
Krugman says that futures prices are too low to cause people to withhold physical oil and sell forward, as required to affect spot prices. But whatever forward price curve he shows me, I can posit an invisible "convenience yield" large enough to make hoarding oil worthwhile.
In a different post, Mark writes,
while it may be possible to store grains and other commodities in non-traditional forms, if the claim is that's what's gong on now, why store grains in (what I presume are) more costly non-traditional methods when, with stocks this low, there is plenty of storage space available?
That is another good question. I think that I have to argue that non-traditional storage methods (keeping oil under sand, storing wheat as crackers) are in fact less costly (at least for large adjustments) than are changes in the quantities held in storage tanks and silos.
On oil, I'll basically repeat myself....
There has been a dramatic decline in interest rates over the past two years, so that we do not necessarily need to observe an increase in the futures-spot premium to say that the incentive to hoard has increased. Moreover, as Waldman points out, we can always appeal to the unobservable "convenience yield" (or perhaps tell my option-value story).
The other folks (Krugman, most notably) seem to want to say about the commodities markets, "Silos and storage tanks are not filling up. The futures prices are not high relative the spot prices. So you can't blame the rise in prices on speculation about future demand and supply."
My reply would be that there are other ways to store commodities. Oil can be stored under sand. Wheat can be stored as crackers. Also, the futures-spot price differential is affected by interest rates and changes in the fudge factor known as "convenience yield."
What I say is that you cannot possibly have prices in these markets that fail to reflect speculation about future demand and supply. I do not think it is reasonable to claim that the oil market was in long-term balance eighteen months ago and is in long-term balance today. Either speculators vastly under-priced oil back then or they are over-pricing it now, or both.
Here's Jim Hamilton's reply:
Kling's question on oil speculation, by Jim Hamilton: Arnold Kling poses a question for Paul Krugman. Here's how I would answer. ...
The "fundamentals" price of oil depends on a number of factors that cannot be perfectly foreseen. Among these are (1) will the world enter a deep and prolonged recession in 2007, and (2) will global oil production in 2007 be higher than it was in 2006? Today, we know that the answer to both questions is no, and conditional on knowing that answer, we can see that $60/barrel was too low a price. But a year ago, no one knew those answers.
Likewise, the price of oil today is very much dependent on the answer to questions such as (1) will the world enter a deep and prolonged recession in 2008, and (2) will global oil production in 2008 be higher than it was in 2007? Today, we do not know the answer to these questions. If the answer is yes, the price of oil today is much too high. If the answer is no, the price could still be too low.
As for the specific question of "where are the inventories", let's be a little more precise about the question being asked. The correct question is, Did the movement along the demand curve that resulted from the increased price show up as an increase in inventories? The correct answer is, no, it was offset by a shift in the demand curve for newly industrialized countries and the oil producing countries. For example, China may have consumed a half million more barrels of oil per day in 2007 compared with 2006.
Where are the inventories? China already burned them.
Update: Arnold Kling responds.