Frank Luntz tries to excuse the inexcusable
by David Atkins
Frank Luntz was in the New York Times yesterday to defend the wildly, outrageously inaccurate polling that said Cantor would waltz to victory.
From the spinmeister himself:
ERIC CANTOR wasn’t the only person at a loss for words on Tuesday night.
His pollster, McLaughlin & Associates, found itself trying to explain the impossible — how a projected 34 percent lead for the House majority leader 12 days before the election could end up an 11-point loss on Election Day to David Brat of the Tea Party in the Virginia Republican primary.
We’ve all been there. There isn’t a pollster alive — me included — who hasn’t had to take the walk of shame, hat in hand, to explain to an angry client why a predicted outcome simply didn’t happen.
Make no mistake: This was a whopper for the ages. McLaughlin didn’t merely get it wrong; this was quantitative malpractice — a mind-blowing modern-day “Dewey Beats Truman” moment.
That said, polls can’t predict elections. They are essential tools, windows into the minds of a particular audience — but they cannot and should not be used as infallible crystal balls.
Trouble is, pollsters are under ever increasing pressure to feed a voracious media beast and provide the answer to that perennial question, “Who’s gonna win?” And therein lies the problem with polls, pollsters and consumers of both.
Yes, a poll is a useful tool for gaining insight and information, but it is only one arrow in the quiver. Without qualitative insight — talking with voters face to face to judge their mood, emotion, intensity and opinion — polls can be inconsequential, and occasionally wrong.
The Cantor campaign’s catastrophe is not without modern precedent, even if the size and scope were extreme. Anyone remember Al Gore winning Florida, John Kerry winning Ohio and, of course, President Mitt Romney?
he simple truth remains that one in 20 polls — by the simple rules of math — misses the mark. That’s why there is that small but seemingly invisible “health warning” at the end of every poll, about the 95 percent confidence level. Even if every scientific approach is applied perfectly, 5 percent of all polls will end up outside the margin of error. They are electoral exercises in Russian roulette. Live by the poll; die by the poll.
Even for Luntz, this is deeply embarrassing. Dewey/Truman, Gore in Florida, and Kerry in Ohio were all within either the official statistical margin of error, or close enough as to be excusable. Almost no one but the rubes, "unskewers" and deluded professional operatives actually believed that Romney was likely to win in 2012; just about every single pollster, not to mention every polling aggregate, had Obama as the victor.
A 34 percent lead turning into an 11-point loss isn't a "your mileage may vary" black swan. Eric Cantor wasn't arrested for murder in the meantime--and even if he had been, not even that sort of thing swings 45 points in the polls. Heck, even jailed alleged mafia gun running CA Secretary of State candidate Leeland Yee still pulled 10% in a crowded electoral field, dropping a maximum 10-20 points off his probable tally otherwise.
Yes, the occasional poll does wind up missing the mark so badly that it falls outside the margin of error. That's the business, and why a competent campaign with Cantor's resources polls more than once just to be on the safe side. But it's one thing to poll outside the margin. It's another to miss by 45 points. That's not just a miss. It's not on the same planet.
Luntz probably has friends at McLaughlin & Associates and feels a professional obligation to cover for them. But his take on it is embarrassing. McLaughlin either made a catastrophic error, or it cut corners and grifted Cantor's campaign. One or the other. It is practically impossible--certainly not a 1 in 20 chance--for a legitimately and accurately conducted poll to miss by 45 points. It just doesn't happen.
Luntz does know this, of course. His piece ends with a barely disguised pitch for candidates to do a little less quantitative polling, and a little more in-depth qualitative research of the focus group variety. Luntz has a professional interest there and I happen to agree with him. I'm a professional qualitative researcher myself, after all, and I share Luntz' opinion on the relative value of the two methodologies.
But let's not pretend that the McLaughlin poll was legitimately and correctly conducted. It can't have been.
UPDATE: A reader correctly points out that I doubled the actual number of the "swing" by both subtracting from Cantor's line and adding to Brat's. That's true, so it would actually be a 22-point "swing", potentially less if you count all the undecideds going to the challenger. Even so, it's still far, far beyond any reasonable standard for error. Another reader notes that the poll might have been of registered rather than likely GOP voters--I don't know if that's true, but that would be polling malpractice in a super-low turnout election.
.