One possibility we can probably rule out is the difference between live and automated interviewers. Both Survey USA and American Research Group used the automated technology, and their estimates of the gap between Obama and Clinton are 18% apart. It also appears as though all three organizations used random digit dialing rather than working from a registration list (which is good since those registration lists have changed quite a bit in PA over the last few weeks).
Question wording is another possibility. Here is a comparison of each organization's question:
Quinnipiac: "If the 2008 Democratic primary for President were being held today, and the candidates were Hillary Clinton and Barack Obama, for whom would you vote?"
Survey USA: "If the Democratic Primary for President of the United States were today, would you vote for...(names rotated) Hillary Clinton? Barack Obama? Or some other Democrat?"
American Research Group: "If the 2008 Democratic presidential preference primary were being held today between (names rotated) Hillary Clinton and Barack Obama, for whom would you vote - Clinton, Obama, or someone else?"
Nothing obvious stands out from those questions...all are asked in slightly different ways, but none in a way that seems to explain such major differences.
To gain some insight into what might be causing these divergent results, we can take a look at some basic demographic breakdowns from each survey. Each survey tells us how preferences were divided by gender, race, and age. Survey USA and American Research Group also add information about the percentage of their sample that was comprised by each of those groups, I had to do a bit of backwards calculation to get that information for the Quinnipiac survey. The table with this information is here:
The first thing to look at is the composition of each sample. Note that there are not major differences here. Survey USA does have the highest percentage of women; this can account for some of the discrepancy between this poll and the others, but not all of it. Survey USA also differs from American Research Group in its assumption that more than half of the electorate will be older than 50. Again, this can explain some of the discrepancy in that survey, but not all of it.
What may be more notable is that among various subgroups, the samples appear to find very different preferences. For example, support for Clinton among women ranges from 52% in the ARG sample to 61% for Survey USA. Combine that difference with the different percentages of women sampled, and we begin to really account for a lot of the different findings. Note also that support for Clinton is 38% among the 18-45 group in the ARG survey, but 52% with the Survey USA sample. Thus, the differences in the survey results are not just a matter of how the organizations are defining their samples, but also a function of the fact that they are finding very different vote patterns among the same demographic groups. However, what remains a mystery is why this is the case.
I'm largely stumped on this one. One explanation is simply that the organizations are using very different screening to determine who is and isn't a likely voter. It may also be the case that the Survey USA poll is a bad sample. The thing that stands out for me with this sample is the breakdown for the 18-45 age group; it seems a little odd to see Clinton up by 11% with this group. The last Survey USA poll also showed a narrower race than this one. This stands out since almost every other organization polling consistently in the state has showed a race that is consistently narrowing.
Any thoughts?
UPDATE: Mark Blumenthal has some discussion of these Pennsylvania polls as well (here and here).
UPDATE 2: Two new surveys out (PPP and Strategic Vision) both have the race in PA close (3 points in one poll, 5 in the other); this suggests that the Survey USA poll really is the outlier. Question remains...will it be the correct outlier?
UPDATE 3: I've put together some data on the PPP and Insider Advantage surveys released from PA in the last couple of days. Survey USA appears to be giving Clinton a bigger advantage among women and whites than any other survey, and Survey USA is the only poll that gives Clinton an advantage among the 18-45 or 18-49 age group.
10 comments:
I think the polls are actually very consistent with respect to Obama. Obama's polls have generally showed 40-43% support, a 4% variation. The outlyers tend to be a poll here and there showing 38% or 44%. Hillary's polling, on the other hand, is all over the board. She polls up to 56% down to 45%. And this is probably what makes the difference.
In general, the more "undecideds" that appear in the poll, the closer the candidates are. I suspect that the way the question is asked determines the number of undecideds. The Survey USA polls asks "...The President of the United States...", while the other polls ask "...the President...". This slight change in wording may make a difference in focusing the electorate to making a choice, rather than putting off the choice to election day. This would bode well for Hillary, since it would show Obama's support at a ceiling of only 43%. It would also show that Hillary would fetch the lion's share of the undecided votes at this point. Past primarys, with a few exceptions, have shown this to be the case.
Of course, this is all theory. The SurveyUSA poll may be an outlyer. However, the trend between SurveyUSA and other polls have been consistent in PA, with SurveyUSA showing consistently more Hillary support than others. That would point to something in the methodology that is the reason.
Can someone please explain to me where they get the phone numbers they use for these polls? Do they randomly call numbers from a phone book? The reason I ask is, it is clear that a large percentage of Obama supporters are younger, and most younger people don't have home phones. We prefer to stick to cellular phones. Depending on how these numbers are collected, a large percentage of potential voters may never be contacted. And the method that is used may inherently distort the results since the voter base is so dissimilar to begin with.
been tryign to figure this out. looked back at CA an 2004 and SUSA was the outlier, and right.
Did you look at the tied survey. They counted not only registered democrats but what they call independent democrats. That may be the difference.
Oh sorry, the first did as well but the large majority was registered democrats. So a trend seems to be that when registered closed primary democrats vote, Obama seems to fall behind. When a survey is conducted where no question is ask wether or not they are registered or what party they are registered with, they seem to be even. Kinda makes me go hmmm.
Also I looked at the PPP poll and 11% where undecided. How convienient. ON the strategic vison poll, they gave no indication on how they recieved there polling data therefore they shouldn't be trusted.
SUSA does not weight their polls. Basically everyone else does.
Some polling companies are deciding that 36% of the electorate will be black and only 57% white.
75% of the Penn pop is white, 10% black.
In the last primary, which was uncontested blacks made up just under a third ofthe voters.
Most pollsters are making the FALSE assumption that black voter turnout will increase as a percentage of the vote.
This is a false assumption. Due to added interest in this primary, all demographics will increase their voting numbers but not as a percentage. One should assume that the percentage black pop in the previous primary was due to lack of interest and that it will actually go down.
There is ABSOLUTELY NO WAY that 10%of the population will make up 36% of the primary vote. It just wont.
If white interest and black interest are the same for this primary, and i think it is and ALL blacks are democrats, 0% republican, the balck percentage of the vote should be around 20%, not the 36% weighting for some of these polls.
SUSA is the only company that lets the chips fall where they may. They call people randomly, screen for likely voters and let the numbers talk for themselves. No weighting.
They have been acurate more than anyone else this primary season.
Weighting is the problem with these polls and probably sample size within demographic groups. The margin of error is huge when your sample size is small and then you weight it to be worth more.
So, uh, why are only 94% of the respondents in the IA poll 18 and above?
ARG now shows Clinton with a 20 point lead. That is 20 points up on its last poll!
I think it is the first data post the "bitter" gaff. Still I find it implausable that Obama should get hit so much harder for telling an unpleasent truth than Clinton's "sniper" lie.
how reliable is the ARG poll?
Good discussion of the new ARG poll (in the context of the other PA polls) here:
http://www.pollster.com/blogs/comparing_pollsters_in_pennsyl.php
Post a Comment