Reading into the polling numbers
MARK MELLMAN
In 2000, too many inaccurate conclusions were reached too quickly on the basis of overinterpreted poll data. Upon sober post-election reflection, analysts seemed to agree that ignoring margins of error by making too much of small day-to-day fluctuations was an error that should not be repeated.
But here we go again.
I should disclose that I have an interest in what follows; Sen. John Kerry (D-Mass.) is my client. But that does not change the facts.
First, a dull excursion into statistics. We all know that each number in a poll has a margin of error, based on the sample size. The margins of error on each number are independent of each other, so the margin of error on the margin between two candidates is twice the margin of error for the poll. Moreover, the margin of error for comparing two polls is about one and a half times the margin of error on each.
Finally, differences in methodology make comparing polls from two different organizations quite dangerous.
With that introduction, let’s evaluate the spate of headlines about the recent Howard Dean “surge” for the Democratic presidential nomination.
An ARG New Hampshire poll in July showed Kerry with 25 percent; Dean, former governor of Vermont, with 19 percent; Sen. Joe Lieberman (Conn.) with 6 percent; and Rep. Richard Gephardt (Mo.) with 10 percent.
Do those data reflect a Dean surge? He has exactly the same percent he garnered in May. Kerry declined by 1 point, Gephardt by 2 and Lieberman by 6. The margin of error for comparison of those two polls is about plus or minus 6 points.
The Franklin Pierce poll was also heavily reported. The margin of error for comparing their April and July polls is about plus or minus 8. Between April and July, the Dean vote registered no change. Kerry declined by 2, Lieberman by 3 and Gephardt by 2. All of that change is well within the margin of error.
But if we reduce the margin of error by averaging those two July polls together with the Boston Herald poll (for which there is no earlier point of comparison), the results are Kerry 24 percent, Dean 23 percent and Lieberman and Gephardt both at 8 percent.
If anything, Kerry has held steady while Gephardt and Lieberman have declined a bit and Dean has improved slightly. That differs dramatically from earlier prognostications. Pundits predicted that Dean’s “rise” would come at Kerry’s expense. In fact, Lieberman and Gephardt are the ones who may have paid the price.
California’s Field Poll reflects a similar dynamic. Field shows Kerry 1 point behind Dean. Since April, Kerry dropped 1 point and Dean rose by 9 while Lieberman fell 8 points and Gephardt shed 9. The 10-point margin of error for comparison means there may have been little movement at all. But if Dean did rise, it was not at Kerry’s expense. Rather Dean seems to have taken voters from Gephardt and Lieberman.
Iowa presents a slightly different case. After spending about a quarter of his total media budget, Dean has moved into first, ahead of Gephardt and Kerry, who have spent nothing on the air. No surprise here.
Four conclusions emerge from those polls:
• Dean may or may not have increased his support in California but probably has not moved at all in New Hampshire.
• Kerry and Dean are tied in New Hampshire and California.
• To whatever extent Dean has risen, it has been at the expense of Lieberman and Gephardt but has not hurt Kerry, as originally predicted.
• If you go on TV alone, your numbers will go up, at least temporarily.
All in all, it is a less exiting story than “Dean surges” but rather more accurate.
Mark S. Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982.
MARK MELLMAN
In 2000, too many inaccurate conclusions were reached too quickly on the basis of overinterpreted poll data. Upon sober post-election reflection, analysts seemed to agree that ignoring margins of error by making too much of small day-to-day fluctuations was an error that should not be repeated.
But here we go again.
I should disclose that I have an interest in what follows; Sen. John Kerry (D-Mass.) is my client. But that does not change the facts.
First, a dull excursion into statistics. We all know that each number in a poll has a margin of error, based on the sample size. The margins of error on each number are independent of each other, so the margin of error on the margin between two candidates is twice the margin of error for the poll. Moreover, the margin of error for comparing two polls is about one and a half times the margin of error on each.
Finally, differences in methodology make comparing polls from two different organizations quite dangerous.
With that introduction, let’s evaluate the spate of headlines about the recent Howard Dean “surge” for the Democratic presidential nomination.
An ARG New Hampshire poll in July showed Kerry with 25 percent; Dean, former governor of Vermont, with 19 percent; Sen. Joe Lieberman (Conn.) with 6 percent; and Rep. Richard Gephardt (Mo.) with 10 percent.
Do those data reflect a Dean surge? He has exactly the same percent he garnered in May. Kerry declined by 1 point, Gephardt by 2 and Lieberman by 6. The margin of error for comparison of those two polls is about plus or minus 6 points.
The Franklin Pierce poll was also heavily reported. The margin of error for comparing their April and July polls is about plus or minus 8. Between April and July, the Dean vote registered no change. Kerry declined by 2, Lieberman by 3 and Gephardt by 2. All of that change is well within the margin of error.
But if we reduce the margin of error by averaging those two July polls together with the Boston Herald poll (for which there is no earlier point of comparison), the results are Kerry 24 percent, Dean 23 percent and Lieberman and Gephardt both at 8 percent.
If anything, Kerry has held steady while Gephardt and Lieberman have declined a bit and Dean has improved slightly. That differs dramatically from earlier prognostications. Pundits predicted that Dean’s “rise” would come at Kerry’s expense. In fact, Lieberman and Gephardt are the ones who may have paid the price.
California’s Field Poll reflects a similar dynamic. Field shows Kerry 1 point behind Dean. Since April, Kerry dropped 1 point and Dean rose by 9 while Lieberman fell 8 points and Gephardt shed 9. The 10-point margin of error for comparison means there may have been little movement at all. But if Dean did rise, it was not at Kerry’s expense. Rather Dean seems to have taken voters from Gephardt and Lieberman.
Iowa presents a slightly different case. After spending about a quarter of his total media budget, Dean has moved into first, ahead of Gephardt and Kerry, who have spent nothing on the air. No surprise here.
Four conclusions emerge from those polls:
• Dean may or may not have increased his support in California but probably has not moved at all in New Hampshire.
• Kerry and Dean are tied in New Hampshire and California.
• To whatever extent Dean has risen, it has been at the expense of Lieberman and Gephardt but has not hurt Kerry, as originally predicted.
• If you go on TV alone, your numbers will go up, at least temporarily.
All in all, it is a less exiting story than “Dean surges” but rather more accurate.
Mark S. Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982.
0 Comments:
Post a Comment
<< Home