There is a lot of confusion over the Fed's use of core inflation as part of its policy making process. One reason for confusion is that we using a single measure to summarize three different definitions of the term "core inflation" based upon how it is used.
First, core inflation is used to forecast future inflation. For example, this recent paper uses a "bivariate integrated moving average ... model ... that fits the data on inflation very well," and finds that the long-run trend rate of inflation "is best gauged by focusing solely on prices excluding food and energy prices." That is, this paper finds that predictions of future inflation based upon core measures are more accurate than predictions based upon total inflation.
Second, we also use the core inflation rate to measure the current trend inflation rate. Because the inflation rate we observe contains both permanent and transitory components, the precise long-run inflation rate that consumers face going forward is not observed directly, it must be estimated. When food and energy are removed to obtain a core measure, the idea is to strip away the short-run movements thereby giving a better picture of the core or long-run inflation rate faced by households. I should note, however that this is not the only nor the best way to extract the trend and the Fed also looks at other measures of the trend inflation rate that have better statistical properties. Thus while the first use of core inflation was for forecasting future inflation rates, this use of core inflation attempts to find today's trend inflation rate [There is a way to combine the first and second uses into a single conceptual framework that encompasses both, but it seemed more intuitive to keep them separate. In both cases, the idea is to find the inflation rate that consumers are likely to face in the future.]
Let me emphasize one thing. If the question is "what is today's inflation rate," the total inflation rate is the best measure. It's intended to measure the cost of living and there's no reason at all to strip anything out. It's only when we ask different questions that different measures are used.
Third, and this is the function that is ignored most often in discussions of core inflation, but to me it is the most important of the three, it is the inflation target that best stabilizes the economy (i.e. best reduces the variation in output and employment).
In theoretical models used to study monetary policy, the procedure for setting the policy rule is to find the monetary policy rule that maximizes household welfare (by minimizing variation in variables such as output, consumption, and employment). The rule will vary by model, but it usually involves a measure of output and a measure of prices, and those measures can be in levels, rates of change, or both depending upon the particular model being examined, but generally a Taylor rule type framework comes out of this process ( i.e. a rule that links the federal funds rate to measures of output and prices).
However, in the Taylor rule, the best measure of prices is usually something that looks like a core measure of inflation. Essentially, when prices are sticky, which is the most common assumption driving the interaction between policy and movements in real variables in these models, it's best to target an index that gives most of the weight to the stickiest prices (here's an explanation as to why from a post that echoes the themes here). That is, volatile prices such as food and energy are essentially tossed out of the index.
Another feature is that the indexes often include both output and input prices, and occasionally asset prices as well. That is, a core measure of inflation composed of just output prices isn't the best thing for policymakers to target, a more general core inflation rate combining both input and output prices works better.
The core inflation rate you see in the news, the one that strips out food and energy, should be though of as a short-hand, quick measure of all three of these concepts. But in each case the Fed uses measures (formally or informally) designed to best satisfy these three functions. For example, when it forecasts future inflation, it uses a different concept of core inflation than it uses in setting policy.
The Fed paper linked above is one example of how the Fed searches for the optimal way to predict future inflation. All that matters for policy is finding the most accurate way to predict inflation, and they will use whatever definition of inflation is best suite for this job. There is evidence that core inflation is best and that's why it is used, but it is a statistical question and didn't have to come out that way (and the best measure of core inflation may not be simply stripping out food and energy - also, there are different measures of prices to choose from, e.g. the CPI and the PCE).
When it comes to setting policy, the Fed doesn't formally use a Taylor rule, though they certainly have Taylor rule estimates in the information they use when considering policy moves, instead they look at a variety of measures of inflation (both core and total), and they look at the rate of change in input prices such as wages and commodities (e.g. oil) as well. They then weight each of those pieces of information in some way (and hence construct an implicit index of all of this information), and then set the federal funds rate accordingly. While I have no way of knowing if the weights they use are optimal, this is exactly what the theoretical models say they should do. But the point is that it is some implicit combination of all of this information that matters for policy, and hence this core measure is very different from the core measure that is best at predicting future inflation alone.
The core inflation rate you see in the news, the CPI less food and energy, does do a fairly good job of representing the information the Fed uses to forecast future inflation, i.e. it is a measure of the trend rate of inflation (but not the best one), and it also does well at approximating the information the Fed will use to set policy, but the actual process it uses is more complicated than this and the Fed employs measures that are specialized for the job at hand.
Finally, there is also a question of what we mean by inflation conceptually. Does a change in relative prices, e.g. from a large increase in energy costs, that raises the cost of living substantially count as inflation, or do we require the changes to be common across all prices as would occur when the money supply is increased? Which is better for measuring the cost of living? Which is a better target for stabilizing the economy? The answers may not be the same. For a nice discussion of this topic, see this speech given yesterday by Dennis Lockhart, President of the Atlanta Fed:
Inflation Beyond the Headlines, by Dennis P. Lockhart, President, Federal Reserve Bank of Atlanta: ...Let me begin by posing the simple question: What do we mean by "inflation"? The answer to that simple question isn't as simple as it may seem.
The popular treatment of inflation in our sound bite society risks confusing inflation with relative price movements and the cost of living. By cost of living, I'm referring to the costs you and I incur to maintain our level of consumption of various goods and services including essential items such as food, gasoline, and lodging.
Relative price movements occur continuously in an economy as individual prices react to market forces affecting that good or service. Neither relative price movements nor sustained high living costs constitute inflation as economists commonly use the term....
And I think I'll end with this part of his remarks:
Attempts to measure the aggregate rate of price change—no matter how sophisticated—remain imperfect. As a result, when it comes to measuring inflation, judgment is needed to distinguish persistent price movements that underlie overall inflation from the relative price adjustments. Separating the inflation signal from noise involves much uncertainty—especially when making decisions in real time. Discerning accurately the underlying trend is difficult. It is essential for those of us who have responsibility for responding to these trends to use a wide variety of core measures and inflation projections to make the most informed judgment we can.