A series of polls commissioned by the Democratic group MoveOn.org Political Action Committee has Democrats and some in the media all abuzz about the effects the government shutdown might have on the 2014 midterm elections. The polls, conducted by Public Policy Polling, or PPP, show that control of the U.S. House of Representatives may be in reach of the Democrats.
For Democrats to regain control of the House, 17 seats would need to change to their favor. The PPP polls show that Republican incumbents are behind in 17 of the districts across the country analyzed.
That might well be true, and the most recent PPP polls might accurately reflect the population of the districts. However, the group’s record in Alaska might shed some light on whether or not the numbers should be trusted.
PPP got it way wrong in the last big Senate race in 2010. That race, however, was a difficult one and others also missed the mark. More recently, however, the group polled in Alaska about gun control. It showed that U.S. Sen. Mark Begich lost support in Alaska as a result of his vote against it. But the poll was badly flawed. Democrats were over represented by a whopping 9 percent, women were over-represented by 12 percent, and the firm just couldn’t figure out the nonpartisan/Alaska Independent Party thing.
Other polls that PPP has conducted in Alaska are similarly flawed.
In the wake of the firm withholding a poll about the recall of a Democratic state senator in Colorado, others in the media are beginning to write about PPP’s suspect methodology.
Uber pollster Nate Silver tweeted that the main problem with group’s polls is that “their approach to polling is extremely ad hoc. Ultimately, that ad-hockery stems from a lack of appreciation/understanding for the statistical fundamentals behind polling.”
In The New Republic, Nate Cohen writes the following:
After examining PPP’s polls from 2012 and conducting a lengthy exchange with PPP’s director, I’ve found that PPP withheld controversial elements of its methodology, to the extent it even has one, and treated its data inconsistently. The racial composition of PPP’s surveys was informed by whether respondents voted for Obama or John McCain in 2008, even though it wasn’t stated in its methodology. PPP then deleted the question from detailed releases to avoid criticism. Throughout its seemingly successful run, PPP used amateurish weighting techniques that distorted its samples—embracing a unique, ad hoc philosophy that, time and time again, seemed to save PPP from producing outlying results. The end result is unscientific and unsettling.
Elsewhere in the piece, Cohn indicates that PPP tweaks the methodology to get the answers it wants.
I don’t know the effect that the current slate of polls will have on the shutdown or on the midterm elections. But I do know that if PPP is trying to help Democrats in Republican districts, which appears to be the case, flawed polls will backfire eventually. They will confirm what many suspect: the group, and the party associated with it, are all big liars.
Contact Amanda Coyne at amandamcoyne@yahoo.com
