Monday, October 08, 2012

Gallup's Sample Part 1 - Education Demographics

There have been numerous complaints over the samples used by polling organisations this year. They have been reporting an unlikely (to be generous) number of Democrats in their samples. The response has been more or less uniform, something on the order of, "We don't weight our samples to political orientation, because those have a history of change, we only weight by other characteristics like sex, race, age that aren't subject to change to the same degree.

So far, I haven't seen anyone look into how reasonable the breakdown for those things is. This is my first post doing so. I chose Gallup, because they have been quite reputable for a long time. As it turns out, their samples also show major problems in other demographics.

The first issue I will go over  is in the area of education, because not only did the numbers look obviously wrong, they may be the easiest to demonstrate. Gallup's sample over the dates September 10th-September 30th includes 22% (2162 of 9842) people with a 'Post-Graduate' education level. Nearly a quarter of registered voters hold advanced degrees?! Impossible. According to 2011 Census tables (the top file), the percentage of the adult population with better than a bachelor degrees is less than 10% (22 million out of 231 million). While I'm sure that those with advanced degrees are more likely to be registered than the average American, we had approximately 187 million registered voters in 2008. Even if every single one of those 22 million people with post-graduate degrees were registered, if the number of registered voters is similar this year, they would account for less than 12% of registered voters (22/187).

This hugely over-sampled group goes to Obama by almost 3:2 - 57% prefer Obama to 39% for Romney.

Another major issue with the educational demo is that the 'College Graduate' demographic is under-sampled. There are over 62 million Americans with Bachelor and/or Associate degrees, according to that same Census table - nearly 3 times as many as hold advanced degrees. Yet they only make up about 24% of the sample. According to statistics posted by George Mason University, 87.9% of eligible Americans registered to vote in 2008. If anything, college graduates should be more likely to register than the average American. However, using that percentage would give us about 55 million registered college graduates (.879 times 62.5 million). That would, in turn, indicate that a representative polling sample should include at least 29% college graduates (55/187).

This under-sampled demographic leans toward Romney, 50-44.

Adjust for both of these discrepancies, and Obama's lead goes poof. When you consider that this is a registered voter poll, and actual voters almost always turn out to favor Republicans by several points when compared to registered voter polls (especially this year), and Romney is actually leading.