You are using an outdated browser.
Please upgrade your browser
and improve your visit to our site.

The Perils of the Polling Average: How A Basic Election Prediction Tool Is Favoring Romney

The polling average is the essential tool for gauging the state of the race. It's remarkably simple and has an excellent record of performing in crunch time. But while the polling averages are likely to nail the final results yet again, there's an important disclaimer: Two of the most frequent polls, Rasmussen and Gallup, will tend to drag the national polling averages in Romney's direction. 

Take the RCP average, which only considers the last two or three weeks of polling. As a result, only a handful of polls are usually represented. The decision to privilege timeliness over comprehensiveness is entirely justifiable, since a shorter window will pick up shifts in the race more quickly than a less sensitive long-term average. But so far in 2012, there haven’t been genuine shifts in the race—so poll-watchers haven’t yet reaped the benefits of responsive, narrow time-frame polling averages (although they will after the conventions). Instead, the polling averages are shifting with the changing composition of the pollsters included in the average—and their house effects.

If all polls were released at similar intervals, this wouldn’t create systemic bias. But Gallup and Rasmussen release polls every day, so they’re always in the RCP average. So, even though Gallup and Rasmussen are just two of 20 or so national pollsters, they usually represent anywhere from one-third to one-sixth of the RCP average. In contrast, Pew Research, which consistently shows Obama leading by 4-to-7 points, generally releases polls once a month. And for good measure, they produce huge sample sizes, accrued over long periods, so they’re kicked out of the polling average sooner than they would if they surveyed fewer individuals over one weekend.

The same factors influence the Pollster.com trendline. Since Gallup and Rasmussen are usually among the most recent polls, the Pollster trendline usually surges toward Romney—especially if you select "less smoothing." Using "more smoothing" helps, but it still has over-represents Gallup and Rasmussen. (Pollster enters Gallup and Rasmussen into their average every week, rather than once a month, which would be similar to the rate that most other pollsters conduct surveys.) It's worth noting that Pollster also includes two additional tracking polls from PPP and YouGov/Economist that aren't included in the RCP average. Those two pollsters diminish Gallup and Rasmussen's importance, but make the pollster average even more sensitive to tracking polls, rather than the less-frequent surveys conducted by other pollsters.

So how could we correct for this? Well, most pollsters are conducting a poll at least once a month, so a monthly polling average that includes one result from each firm should largely correct for the oddities of the time-sensitive averages. Gallup and Rasmussen only represent two of 15 polls conducted since the last cycle of monthly polls began, so a monthly average reduces their weight to something much more appropriate. If you go back through the RCP average until the beginning of the most recent monthly polling cycle, you’d find Obama leading by 2.5 points, slightly more than Obama’s current 1.8 point advantage and much more than his 1.1 point lead from a few days ago.

The monthly average takes a lot of the volatility out of the RCP average. For instance, RCP shows Obama jumping from a 1.1 point advantage to 1.8 points over the last few days, as Democracy Corps replaced The Washington Times, Reuters, and ABC News/Washington Post, but the monthly average would have only moved from 2.4 to 2.5 in a monthly average. Last week's larger gap between the monthly average and the RCP average isn't uncommon either—the monthly average almost always shows Obama leading by a larger margin than the RCP average.

This is a relatively minor issue, but a 1-to-1.5 point difference in the RCP average affects perceptions of the election. Analysts often give the impression of a deadlocked race, but while the race is close, Obama actually has a clear and narrow lead. Obama leads in the swing states, he leads 13-3-1* among national pollsters conducting surveys over the last month, and he even leads by more in surveys conducted by a few especially reputable pollsters. Sure, Obama's advantage might fade—or even vanish—after switching to likely voter models, but the exact magnitude of such a shift remains to be seen.

The averages are likely to move into alignment by November, since nearly every pollster is likely to conduct a survey immediately prior to the election. The surge of polls will dilute the influence of Gallup and Rasmussen, providing a more representative picture of the popular vote, regardless of whether Romney or Obama will benefit at that point. But in the meantime, be sure to consider the big picture. Just because a poll isn't in the most recent RCP average doesn't mean it stopped mattering.

*Includes a four week average of Gallup, Rasmussen, YouGov/Economist, and PPP weekly tracking polls