This analysis is so bad
Scott Sumner and/or Mark Sadowski.
There are so many issues with the presentation in Sumner's post that it's hard to know where to begin.
I've already gone through the basic logic that the set of countries that should show austerity has an impact in the Keynesian case are those at the ZLB [1] -- call this set ZLB. The countries that should show monetary offset are those with independent central banks -- call this set ICB.
Sumner effectively tests the Keynesian austerity on the set ICB, and ICB ≠ ZLB. Now he could correct this and test austerity on ICB ∩ ZLB (the intersection -- the countries in the middle of the Venn diagram with circles labeled ICB and ZLB), but there are only a few countries in that set (basically only the Eurozone, the US and Japan are in this situation). And there weren't enough points to be conclusive anyway.
I'd like to summarize the many, many issues:
Issue 1: Data manipulation. Throwing out some of the relevant data to the Keynesian hypothesis and keeping some data irrelevant to the Keynesian hypothesis. This will not test the Keynesian hypothesis. Mentioned in previous blog posts. The way to fix this is to include the correct sets of points (ICB and ZLB) and test each hypothesis separately.
Issue 2: Dependent variable bad. Dependent variable includes a large irrelevant signal -- high trend growth countries will appear at higher y-values for reasons that have nothing to do with austerity during a recession. The ∆NGDP (or ∆RGDP) has two components:
∆NGDP = ∆NGDPtrend + Austerity + Recession
For Singapore, South Korea and Israel ∆NGDPtrend compounded over 5 years will lead to a much larger value: 28% rise over 5 years for 5% growth in Israel and a 5% rise over 5 years for 1% growth in Japan. Even if these two countries engaged in the same amount of austerity (as they apparently did on the graph) the ∆NGDPtrend would put Japan at the bottom and Israel at the top. So two countries -- even with the same amount of austerity -- appear at two different places on the y-axis for reasons that have nothing to do with austerity (trend growth). You could correct for this by subtracting trend growth.
Issue 3: Independent variable bad. At least Israel didn't really have a recession (I am too lazy to look up the other countries, but Israel's NGDP growth barely registers the global recession; see Evan Soltas). Now Israel wasn't in a liquidity trap, so monetary offset works according to both models. Nearly half of Iceland's so-called austerity happens after the country exits the recession and real interest rates rise above zero. So the amount of austerity these countries engaged in listed in the data is suspect. The only way to fix this is to go get the data oneself.
Issue 4: Inconclusive result treated as conclusive. The given data is actually inconclusive (R² is roughly zero, from the comments on the post -- H/T Tom Brown -- the p-value is greater than 0.9). After throwing out the Eurozone countries (or more accurately summing them together, giving them a weight of ~ 0.05), there is insufficient data to show anything at all. Yet Sumner decides that it shows the Keynesian picture is wrong and/or the results are consistent with monetarism. What he should have said was nothing. Sumner could fix this by deleting the post or posting a retraction.
Issue 5: Lack of counterfactuals. You need a model to understand the counterfactual situation in order to understand what austerity is and what growth is in the cases with and without austerity. Market monetarism cannot produce a counterfactual at all because there is no way of knowing how the market would have behaved with and without austerity. Even with prediction markets you can't know what people would have predicted given different a situation. Because of the dependence on expectations (which we only see because of existing markets according to the market monetarist model), we'd have to read human's counterfactual minds somehow. Markets (purportedly) read their real world minds, but how do you read the counterfactual state of a human mind? There is actually no way to fix this.
...
So both axes, the data points, the method and the fit are completely wrong. Did I miss anything? This is just garbage analysis. Either Sumner and Sadowski don't know what they are doing or are deliberately trying to mislead their audience. I generally go with ignorance rather than conspiracy since that's nearly always more likely.
My question is: how did Sumner and Sadowski end up so ignorant of the basics of hypothesis testing and "experimental" set-up?
Maintain consistent treatment of your data. These are issues 1 and 3. Make sure the data you use to test a hypothesis actually satisfies the conditions of the hypothesis. Don't throw out data that tests one hypothesis and keep data that tests another.
Make sure your data tests your hypothesis. These are issues 2, 4 and 5. At best, Sumner came up with an inconclusive result about whether austerity turns your country into a low trend growth country or not. That should be inconclusive. No one ever said that -- Keynesians think austerity affects growth while it is in place -- it doesn't permanently impact the trend; monetarists think there is monetary offset so austerity has no impact -- it definitely shouldn't turn your country into a low growth country or a high growth country.
I'm just baffled.
Update 6/20/2015:
This blog being the farthest from the bright center of the econoblogosphere it might take me awhile to get to all the comments resulting from being linked by a real blog, but I will do my best (accounting for the fact that I have just returned from a two week work trip, so spending some time with my wife on a rather beautiful day in Seattle is a bit higher priority). In the meantime, I responded to one "comment" i.e. Sumner's counter-criticism here. Feel free to hit that post up with criticism as well. Maybe it answers some of your comments below.
Footnotes:
[1] In the information transfer model, these countries are the one with IT index ~ 1 (the zero lower bound isn't relevant). My problems here are with the methodology -- I am actually advocating a different model from Krugman or Sumner so I don't really have a horse in this race. My model also works better than either of these models.