Designing the Survey and Sample10.2 Describe the methods for conducting and analyzing different types of public opinion polls.
The polling process most often begins when someone says, “Let’s find out about X and Y.” Potential candidates for local office may want to know how many people have heard of them (the device used to find out is called a “name recognition survey”). Better-known candidates contemplating a run for higher office might wish to discover how they might fare against an incumbent. Polls also can help gauge how effective particular ads are or how well (or negatively) the public perceives a candidate. Political scientists have found that public opinion polls are critical to successful presidents and their staffs, who use polls to “create favorable legislative environment(s) to pass the presidential agenda, to win reelection, and to be judged favorably by history.”4
Public opinion polls have become an increasingly common tool for gauging public opinion not only in the United States but also around the world. These global public opinion surveys help us to understand the commonalities and differences across international contexts, and also reflect the social, cultural, and political variations of different states. To think more deeply about these trends, examine the results of four questions from the 2012 Pew Global Attitudes Survey that assess the attitudes toward global powers held by citizens in a sample of countries.
How do citizens’ ratings in these surveys reflect—and not reflect—political and geographic alliances?
Which countries’ citizens are most and least positive across the board? Why do you think this is the case?
What might be some of the challenges in measuring and comparing cross-national public opinion?
Designing the Survey and SampleNo matter the type of poll, serious pollsters or polling firms must make several decisions before undertaking the process. These include determining the content and phrasing of the questions, selecting the sample from the public, and deciding how to go about contacting respondents.
The first matter candidates, political groups, or news organizations must consider when deciding to use a poll concerns what questions they want answered. Determining the content of a survey is critical to obtaining the desired results, and for that reason, candidates, companies, and news organizations generally rely on pollsters. Polls may ask, for example, about job performance, demographics, and specific issue areas.
Special care must be taken in constructing the questions. For example, if your professor asked you, “Do you think my grading procedures are fair?” rather than asking, “In general, how fair do you think the grading is in your American Politics course?” you might give a slightly different answer. The wording of the first question tends to put you on the spot and personalize the grading style; the second question is more neutral. Even more obvious differences appear in the real world of polling. Responses to highly emotional issues, such as abortion, same-sex marriage, and affirmative action, often are skewed depending on the phrasing of a particular question. Even in unbiased polls, how a question is worded can unintentionally skew results.
During a political campaign, strategists will often ask questions that help candidates judge their own strengths and weaknesses as well as those of their opponents. They might, for example, ask if you would be more likely to vote for candidate X if you knew he or she was a strong environmentalist. These kinds of questions are accepted as an essential part of any poll, but concerns arise over where to draw the line. Questions that cross the line are called push polls and often result from ulterior motives.5 The intent of push polls is to give respondents some negative or even untruthful information about a candidate’s opponent so that they will move away from that candidate and toward the one paying for the poll. A typical push poll might ask a question such as “If you knew Candidate X beat his wife, would you vote for him?” Push poll takers do not even bother to record the responses because they are irrelevant; the questions themselves are meant to push away as many voters from a candidate as possible. Although campaign organizations generally deny conducting push polls, research shows that this type of polling has targeted more than three-quarters of political candidates.
After deciding to conduct a poll, pollsters must determine the population, or the entire group of people whose attitudes a researcher wishes to measure. This universe could be all Americans, all voters, all city residents, all Hispanics, or all Republicans. In a perfect world, the poll would ask every individual to give an opinion, but such comprehensive polling is not practical. Consequently, pollsters take a sample of the population that interests them. One way to obtain this sample is by random sampling. This method of selection gives each potential voter or adult approximately the same chance of being selected.
Simple random samples, however, are not very useful for predicting voting because they may undersample or oversample key populations not likely to vote. To avoid such problems, reputable polling organizations use stratified sampling (the most rigorous sampling technique) based on U.S. Census data that provide the number of residences in an area and their location. Researchers divide the population into several sampling regions. They then randomly select subgroups to sample in proportion to the total national population. These selected primary sampling units often are used for many years, because it is cheaper for polling companies to train interviewers to work in fixed areas.
The pollsters choose about twenty respondents from each primary sampling unit for interviewing; the total is 600 to 1,000 respondents. Large, sophisticated surveys such as the American National Election Studies and the University of Chicago’s General Social Survey attempt to sample from lists of persons living in each household in a sampling unit. A key to the success of the stratified sampling method is not to let people volunteer to be interviewed—volunteers often have different opinions from those who do not volunteer.
Contacting RespondentsAfter selecting the poll’s methodology, the next decision is how to contact those to be surveyed. Political polling today takes a variety of forms, including telephone polls, in-person interviews, and Internet polls.
Telephone polls are the most frequently used mechanism by which to gauge the mood of the electorate. The most common form of telephone polls is the random-digit dialing survey, in which a computer randomly selects telephone numbers for dialing. In spite of some problems (such as the fact that many people do not want to be bothered or do not have landline phones), most polls done for newspapers and news magazines operate in this way. Pollsters are exempt from federal and state do-not-call lists because poll taking is a form of constitutionally protected speech.
The most common method of conducting a public opinion poll is via telephone. These polls are often administered by survey researchers working at large phone banks, such as the one shown here.
During the 1992 presidential elections, the introduction of tracking polls, which were taken on a daily basis via phone by some news organizations, allowed presidential candidates to monitor short-term campaign developments and the effects of their campaign strategies. Today, tracking polls involve small samples (usually of registered voters contacted at certain times of the day) and take place every twenty-four hours. Pollsters then combine the results into moving three- to five-day averages (see Figure 10.2).
Day-to-day fluctuations in public opinion on electoral contests are often shown through tracking polls. This figure shows the ups and downs of the 2012 presidential election. President Barack Obama led for much of the race among registered voters, although the polling data generally remained within the margin of error.
Source: USA Today and CNN/Gallup Poll results, http:/
Some polls, such as the American National Election Studies, continue to perform individual, person-to-person interviews. In-person surveys allow surveyors to monitor respondents’ body language and to interact on a more personal basis; thus, they may yield higher rates of completion. However, the unintended influence of the questioner or pollster may lead to interviewer bias. How the pollster dresses, relates to the person being interviewed, and asks the questions can affect responses.
Exit polls, a special form of in-person poll, are conducted as voters leave selected polling places on Election Day. Generally, large news organizations send pollsters to selected precincts to sample every tenth voter as he or she emerges from the polling site. The results of these polls help the media predict the outcome of key races, often just a few minutes after the polls close in a particular state and generally before voters in other areas—sometimes in a later time zone—have cast their ballots. By asking a series of demographic and issue questions, these polls also provide an independent assessment of why voters supported particular candidates.
Well-established pollster John Zogby was among the first to use a scientific Internet survey. Zogby regularly queries over 3,000 representative volunteers (selected using the sampling techniques discussed later in this chapter) on a host of issues. Zogby, Harris Interactive, and other Internet pollsters using scientific sampling strategies have established relatively effective records in predicting election outcomes and gauging opinions on numerous issues of importance to the American public. Political scientists, too, use online polling to collect survey research data. The biannual Cooperative Congressional Election Study conducted by political scientists is one example of this kind of research.
Contrasting sharply with scientific Internet surveys are unscientific Web polls that allow anyone to weigh in on a topic. Such polls are common on many Web sites, such as CNN.com and ESPN.com. These polls resemble a straw poll in terms of sampling and thus produce results that are largely inconclusive and of interest only to a limited number of people.
Analyzing the DataAnalyzing the collected data is a critical step in the polling process. Analysis reveals the implications of the data for public policy and political campaigns. Data are entered into a computer program, where answers to questions are recorded and analyzed. Often, analysts pay special attention to subgroups within the data, such as Democrats versus Republicans, men versus women, age groups, or political ideology, among others. Reporting the results of this analysis can happen in a variety of ways, such as by news organizations or campaigns.
The emergence of a growing number of public opinion research firms means that citizens, the media, and political leaders know more than ever before about how voters perceive political issues and candidates. This information may be useful for promoting a more democratic and representative government. But, it may also be notoriously mercurial, marked by dramatic fluctuations from week to week or even day to day, as illustrated in the political cartoon below.
Why do so many agencies and organizations want to collect public opinion data? How do the data help them advocate for their cause?
How might frequent fluctuations in the results of public opinion polls both help and hurt candidates?
Is there too much public opinion data in modern politics? Why or why not?
Is there too much public opinion data in modern politics? Why or why not?