Larry Summers on whether the stimulus should have and could have been larger (from a longer interview):
Ezra Klein: Overall, could we have done more?
Larry Summers: There are multiple aspects of this question. First, the administration proposed considerably more than Congress was willing to enact. Taking account of the addition of the AMT, the Administration’s Recovery Act proposals were cut back by about 20% in the process of passing Congress by very narrow margins.
Second, the President’s economic team advised that there was essentially no danger of excessive fiscal stimulus in 2009. I joked ... that worrying about overdoing fiscal policy was like my losing too much weight and becoming anorexic — a conceivable possibility, but very far from the dominant risk. The President’s political advisers — rightly in my view — constrained the initial stimulus proposals to avoid sticker shock and rejection or great delay on Congress’s part.
Third, politics aside there were difficulties in moving spending rapidly in 2009. So-called shovel-ready projects often were not in fact ready to go. ... Of course it would have been possible to increase the tax cuts or assistance to state and local governments, but there were severe political limits in both those areas.
Fourth, we believed in the winter of 2009 that if, as seemed likely, more stimulus would ultimately be required, it could be passed through the Congress using the unemployment insurance extension for 2010 as a vehicle. This view proved incorrect. The administration proposed and the House passed in the fall of 2009 a substantial further program... Unfortunately it did not pass the Senate and the focus has shifted very much towards deficit reduction rather than job creation. It is fortunate that the President was able last fall to lead an effort to pair extended tax cuts with payroll tax reductions — without that stimulus we might be looking at a double dip today
This doesn't explain the rosy baseline forecast for the economy that the White House put out, a forecast that has been a thorn in the side of the stimulus package ever since. Given the forecast, it didn't look like the fiscal package did anything. Had the baseline forecast been more realistic (i.e. lower), the stimulus package would have looked better, and the case for more stimulus would have been much stronger.
Note also that it only seemed "likely" they would need more stimulus, they thought it might be enough. They'd wait and see and ask for more if it was needed. But it was nowhere near enough. Thus, politics or not, they appear to have underestimated the degree of the problem, and hence the size of the response that would be required. Summers seems to say they knew how much was needed, but couldn't get it. I'm not so sure that they fully recognized the size of the problem they faced.