# A deeper look at polls.

It's political polling season and, for campaign junkies, there is an ample supply of statistics to feed their habits. Here are some tips for sorting out the numbers.

TIP 1: Despite how journalists describe it, the sampling margin of error is a slope, not a cliff. Most media accounts faithfully provide the sampling error. This number (say, 4 percent) means that, if an infinite number of surveys were taken, 95 percent of the results would fall within plus-or-minus 4.0 percent of the proportion (say, 48 percent for Bush) found for the survey being reported. But any number between 48 percent plus-or-minus 4 percent, that is 44 percent-to-52 percent, is not equally likely to occur.

The distribution, your old friend the normal curve, slopes downward steadily on each side of the 48 percent. The next most likely outcome is plus-or-minus 1 percent (49 percent or 47 percent), plus-or-minus 2 percent (50 percent or 46 percent) is considerably less likely, and so forth down both slopes.

Most reporters do not convey this subtlety. When the St. Louis Post-Dispatch/KMOV survey showed Bush leading Kerry 49 percent-to-42 percent in Missouri, the Sept. 17 article cautiously noted that "the president's lead is at the edge of the poll's margin of error of 3.5 percentage points, which means that any individual number could be that much higher or lower."

The message conveyed is that the Bush lead is statistically shaky. But--and this is the central point--any individual number for the Bush estimate between 52.5 percent (plus 3.5 points) and 45.5 percent (minus 3.5 points) is not equally likely. In fact, with a seven-point lead in a survey with a plus-or-minus 3.5 percent sampling error, the odds that Bush is really ahead are about 99 in 100, a safer bet than the 95 percent confidence level.

Here's a rule of thumb. If the sample size is between 500 and 1,000, treat any lead of 5 percentage points or greater as real. For those over 1,000, consider any advantage of 4 points or more to be meaningful.

TIP 2: For an incumbent, there are two equally important poll numbers: one, the difference between the incumbent and the challenger and the other, the difference between the incumbent and 50 percent. Take the Bush 49, Kerry 42 result. The good news for the GOP is that the president has a seven-point lead.

The not-so-good news is that the incumbent president still falls short of a simple majority. After three-and-a-half years in office, he has not sealed the deal with a winning majority (assuming no third-party candidates).

In a some respects, a Bush 52 percent and Kerry 47 percent result--a five point lead having Bush over 50 percent--is better for the Republicans than is Bush's 49 percent and Kerry's 42 percent--a seven point lead but with the incumbent under 50 percent.

TIP 3: Pay attention to whom the results apply: all registered voters or all likely voters? Few political polls include all voting-age adults, screening respondents so that only those reporting that they are registered are included. But, of course, even a substantial fraction of registered voters will not cast a ballot in November.

To adjust for this, many polls use one or more questions to estimate those who are most likely to turn out and then provide estimates both for them ("likely voters") as well as the entire sample ("registered voters").

Where things get tricky, especially more than a month in advance of the election itself, is predicting who is likely to participate. Gallup, for example, has a seven-question sequence that includes such items as how closely the respondent is following the campaign. Some argue that screener is fine within two weeks of balloting but is much shakier anytime earlier.

TIP 4: Weights matter. When initially completed, most telephone surveys overrepresent some demographic groups (e.g., persons over 50) and underrepresent others (e.g., men). The reason is straightforward: older people and women are more likely to answer the phone.

To correct, analysts "weight" the results so that they resemble the population at large. That's relatively straightforward if one is generalizing to all adults since census data provide an acceptably accurate gauge. It is shakier when attempting to project the 2004 electorate's demographic profile.

Pollsters typically use some combination of demographic information from the previous election's (2000 in this case) exit polls and the Census Bureau's surveys of voting participation. This assumes, however, that the 2004 electorate will resemble its 2000 counterpart. The biggest wild card this year are voters 30 and younger, notably absent in 2000 but apparently much more energized in 2004.

According to a Sept. 17 report issued by the Harvard John F. Kennedy School of Government's Vanishing Voter Project, "young adults' election involvement is perhaps higher in 2004 than in any presidential election since 1972" and "50 percent of young adults in our recent survey reported having had an election-related conversation within the past day, as compared with only 25 percent in our survey during the same week of the 2000 campaign."

Instead of their recent presidential turnout rate of about 30 percent, about half of young adults might vote this time. Current polls are probably not weighting them nearly this high.

TIP 5: If you are looking for a shortcut to poring over tens of polls to determine who's ahead, try the Iowa Electronic Market (www.biz.uiowa.edu/iem).

This electronic market, sponsored by the University of Iowa College of Business, allows purchasing Bush or Kerry shares although there is a cap of how much you can invest. If the candidate in which you hold shares wins the popular vote, you receive \$1 per share. If the candidate loses, your shares become worthless.

Near the end of August, there was price parity--both Bush and Kerry shares were selling for about 50 cents each, meaning the market had the race about even. Since then, Bush shares have risen steadily so that, by Sept. 23, they had passed 65 cents. In gambling terms, the market is making Bush a two-to-one favorite.

If you subscribe to the view that markets efficiently incorporate all existing information about a company's or candidate's prospects, the Iowa Electronic Market provides a one-stop prediction.

Terry Jones is professor of political science at UM-St. Louis.
COPYRIGHT 2004 SJR St. Louis Journalism Review
No portion of this article can be reproduced without the express written permission from the copyright holder.