Three surveys released for the Pennsylvania Democratic primary in the last few days have many political analysts scratching their heads. Quinnipiac's poll (conducted April 3-6) has Clinton up by 6% over Obama. Survey USA's poll (conducted April 5-7) has Clinton up by 18% over Obama. And American Research Group's poll (conducted April 5-6) has the race tied. This is an incredible amount of variation for polls taken within the same few days, so what gives?
One possibility we can probably rule out is the difference between live and automated interviewers. Both Survey USA and American Research Group used the automated technology, and their estimates of the gap between Obama and Clinton are 18% apart. It also appears as though all three organizations used random digit dialing rather than working from a registration list (which is good since those registration lists have changed quite a bit in PA over the last few weeks).
Question wording is another possibility. Here is a comparison of each organization's question:
Quinnipiac: "If the 2008 Democratic primary for President were being held today, and the candidates were Hillary Clinton and Barack Obama, for whom would you vote?"
Survey USA: "If the Democratic Primary for President of the United States were today, would you vote for...(names rotated) Hillary Clinton? Barack Obama? Or some other Democrat?"
American Research Group: "If the 2008 Democratic presidential preference primary were being held today between (names rotated) Hillary Clinton and Barack Obama, for whom would you vote - Clinton, Obama, or someone else?"
Nothing obvious stands out from those questions...all are asked in slightly different ways, but none in a way that seems to explain such major differences.
To gain some insight into what might be causing these divergent results, we can take a look at some basic demographic breakdowns from each survey. Each survey tells us how preferences were divided by gender, race, and age. Survey USA and American Research Group also add information about the percentage of their sample that was comprised by each of those groups, I had to do a bit of backwards calculation to get that information for the Quinnipiac survey. The table with this information is here:
The first thing to look at is the composition of each sample. Note that there are not major differences here. Survey USA does have the highest percentage of women; this can account for some of the discrepancy between this poll and the others, but not all of it. Survey USA also differs from American Research Group in its assumption that more than half of the electorate will be older than 50. Again, this can explain some of the discrepancy in that survey, but not all of it.
What may be more notable is that among various subgroups, the samples appear to find very different preferences. For example, support for Clinton among women ranges from 52% in the ARG sample to 61% for Survey USA. Combine that difference with the different percentages of women sampled, and we begin to really account for a lot of the different findings. Note also that support for Clinton is 38% among the 18-45 group in the ARG survey, but 52% with the Survey USA sample. Thus, the differences in the survey results are not just a matter of how the organizations are defining their samples, but also a function of the fact that they are finding very different vote patterns among the same demographic groups. However, what remains a mystery is why this is the case.
I'm largely stumped on this one. One explanation is simply that the organizations are using very different screening to determine who is and isn't a likely voter. It may also be the case that the Survey USA poll is a bad sample. The thing that stands out for me with this sample is the breakdown for the 18-45 age group; it seems a little odd to see Clinton up by 11% with this group. The last Survey USA poll also showed a narrower race than this one. This stands out since almost every other organization polling consistently in the state has showed a race that is consistently narrowing.
UPDATE: Mark Blumenthal has some discussion of these Pennsylvania polls as well (here and here).
UPDATE 2: Two new surveys out (PPP and Strategic Vision) both have the race in PA close (3 points in one poll, 5 in the other); this suggests that the Survey USA poll really is the outlier. Question remains...will it be the correct outlier?
UPDATE 3: I've put together some data on the PPP and Insider Advantage surveys released from PA in the last couple of days. Survey USA appears to be giving Clinton a bigger advantage among women and whites than any other survey, and Survey USA is the only poll that gives Clinton an advantage among the 18-45 or 18-49 age group.