Man, I thought I trained you guys better than this.
There is a lot of consternation in my e-mail box about two polls. One, from Mason-Dixon, shows John McCain just 4 points down in Pennsylvania. The other, from FOX News, shows McCain down just 3 points nationwide.
Let's
start with the Pennsylvania result. Mason-Dixon is a pretty strong
pollster. So, however, are many others from among the literally dozen
or so agencies that have conducted polling within Pennsylvania over the
past 72 hours. And none of those other pollsters shows the race that
tight.
Mason-Dixon has also had a Republican
"lean" this cycle of perhaps 2-3 points. They are quite frequently the
most favorable number for John McCain in any given state. That
doesn't mean that they are "biased", and it doesn't mean that they are
wrong – there are many different (and legitimate!) ways to think about
this election. But it does mean that their polls need to be
interpreted in that context. Let's say the average poll in Pennsylvania
has Obama ahead by 9.5 points. Mason-Dixon will probably start out
seeing a 9.5-point state at a 7-point state. If they then end up toward
the McCain side of their margin of error -- and they don't use huge
sample sizes – that’s how you get to Obama +4.
Now, look. I
don't think we need to be in the habit of ripping a poll apart every
time that we don't like the result. There is nothing inherently "wrong"
with this poll. It's simply that we need to look at in concert with the
rest of the evidence. In this case, we have an abundance of evidence,
and it suggests on balance that Pennsylvania is neither particularly
close, nor is it particularly "tightening" (Mason-Dixon's prior poll of
the state, in Mid-September, had Obama up by 2).
It might also help to come at this from the other direction. Here is one poll out of many, out of one "must-win" state out of many, that shows that John McCain is sorta kinda close? This is the best news he can muster? On Monday, I laid out specific criteria for what I'd want to see in order to conclude that the race has tightened materially:
John McCain polling within 2 points in 2 or more non-partisan polls ... in at least 2 out of the 3 following states: Colorado, Virginia, Pennsylvania.
We have yet to see any such results in any of these three states.
*-*
As
for the FOX poll, I'm a little bit taken aback at the number of people
who assume that, just because the poll is from FOX, it must somehow
have been cooked. Sixteen times out of 20, an aberrant result (and I'm
not sure you can really call this "aberrant", since a couple of other
pollsters show the race at about 3 points right now) is the result of
statistical noise. Perhaps 3 times out of 20, it might be the result of
a poor sampling procedure. And then there might be that one case in 20
where the pollster feels compelled to his finger on the scale in some
way -- but these cases are extremely rare. And there's no particular
reason to accuse FOX News of this behavior. Their polls haven't had
much of a partisan lean this cycle, and for that matter, they were
among the only pollsters to have John Kerry winning the popular vote in
2004. If there's a problem with FOX News polls, it's not that they're
biased, but that they're simply not all that good.
It's true
that FOX's sample included a materially higher percentage of
Republicans this time around. FOX, however, does not choose its sample;
its sample chooses itself. In this case, when they drew their ping-pong
balls out of the jar, they came up with a slightly higher percentage of
red ones. This kind of thing will happen all the time unless a pollster weights by party ID,
which FOX News and many other pollsters do not. The Pew poll that came
out the other day, for instance, had a big increase in the number of
Democrats in its sample.
Nevertheless, the change in the
partisan ID of their sample does cut against the notion that the race
is tightening. What we are ultimately interested in is whether the same
voters are starting to look at the race in a different way -- making up
their minds, or changing their minds. In this case, however, it appears
mostly that FOX was talking to different
voters -- a more Republican-leaning set of voters -- rather than
reaching the same sorts of voters and finding that they were thinking
about the race differently. John McCain did pick up a few points among
independents, but the numbers among Republicans and Democrats were
essentially unchanged.
Numeros:
Boy,
that's a lot of data. If there's anything that jumps out here, it's
that we probably shouldn't be too quick to conclude that the race comes
down to exactly Virgnia,
Colorado and Pennsylvania. There are two polls out of Virginia today
that show that state a little tighter than most of the other recent
numbers; on the other hand, Ohio is becoming a real problem for John
McCain, and perhaps North Carolina is too.
Lastly, although it's
probably too early to conclude anything much about whether the Obama
infomercial was successful, the Rasmussen and SurveyUSA state polling
that was in the field last night seemed to contain pretty good numbers
for him, slightly better than for much of the past week.
--Nate Silver