Last Updated: May 20, 1999
Printer-friendly version
Perspectives
on polls
Pew study finds that media polls don’t understate conservative
views, but may miss racial hostility
In recent presidential election campaigns, critics have charged that
media-sponsored public opinion surveys produce biased and inaccurate results.
These polls, critics claim, are based on skewed samples that do not fully
represent certain kinds of people or points of view.
Most recently, for example, critics argued that polls taken during the
last presidential campaign regularly overstated President Clinton’s lead
over Republican challenger Bob Dole. New York Times columnist William Safire
has suggested that most media polls were "grievously misleading," not only
exaggerating President Clinton’s lead in 1996 but reducing turnout among
dispirited Republicans.
A unique survey research experiment by the Pew Research Center finds
no evidence, however, that public opinion polls understate conservative
opinions or support for the Republican Party. At the same time, the study
does suggest white hostility toward blacks and certain ethnic minorities
may be understated in surveys that are conducted in just a few days — as
most opinion polls are.
The Pew Research Center fielded two national polls that asked exactly
the same questions but differed in several respects in the way they were
conducted. The first — the "standard survey" — used typical polling techniques,
contacting 1,000 adults by phone in a five-day period. The second — the
"rigorous survey" — was conducted over eight weeks, interviewing 1,201
adults. The longer time frame for the second survey made it possible to
undertake an exhaustive effort to interview highly mobile people and to
gain the cooperation of people who were initially reluctant to participate
in the survey — the types of people who are likely to be underrepresented
in more quickly-conducted polls.
Few differences, except on race
A leading criticism of media polls is that they miss some people. If
a survey fails to interview some segments of society, then those people’s
opinions may not be fully reflected in the poll results. But as it turned
out, the standard and rigorous surveys produced strikingly similar results.
Despite the differences in the way the surveys were administered, the findings
of the two polls barely differed. The surveys included more than 85 questions
concerning media use, lifestyle and a range of political and social issues.
Excluding several questions that were affected by news events during the
period the surveys were being conducted, just five out of more than 85
questions showed statistically significant differences between the two
surveys.
These findings suggest that for most topics, typical media polls do
a good job gauging public opinion. But results based on questions about
racial issues may be more problematic. In fact, the experiment suggests
that accurately measuring racial antagonisms may be a problem in all survey
research. This may help explain why pre-election polls have overestimated
white support for black candidates in biracial elections.
On two of four questions involving racial issues, white respondents
in the rigorous sample were noticeably less sympathetic toward blacks.
For example, 64 percent of whites in the rigorous sample said blacks who
can’t get ahead are responsible for their own condition, while just 26
percent blamed racial discrimination. This compares with a narrower 56
percent to 31 percent division on the question in the standard sample.
These differences offer a clue into what may be one of the biggest challenges
facing pollsters who seek to accurately measure public opinion on racial
issues. People who are reluctant to participate in telephone surveys and
those who are willing to participate in polls hold strikingly different
views on racial questions. For example, 22 percent of those who agreed
to participate in the survey the first time they were called held a "very
favorable" opinion of blacks. This compares with just 15 percent of those
who initially refused to take part in the poll, and participated only when
called back a second or third time.
Other questions on racial issues show similar differences. This suggests
that to increase the accuracy of surveys that focus extensively on racial
issues, pollsters need to make an extra effort to obtain interviews with
people who initially refuse to participate.
Kohut is Director of the Pew Research Center for the People &
the Press.
