Beyond the Horse Race, Chapter 6 Summary
“Beyond the Horse Race, How to Read Polls and Why We Should,” was released recently by John Zogby. Chapter 6 is titled “The Aggregation-Industrial Complex.” Here is a summary of it.
As the world of public opinion research has continued to evolve, there is also a growing critique over aggregating polls, as modeled by Web sites including FiveThirtyEight and RealClearPolitics. It is a trend Zogby calls the “Aggregation-Industrial Complex,” contending that, while this may give the public a good snapshot of overall trends, all too often this leads to simplistic and misleading conclusions. Aggregation masks important differences in individual surveys related to methodology, sample selection, and question phrasing that can result in distorted or inaccurate findings.
Zogby said not all tracking polls are taken the same way, and these differences sometimes can make substantial differences. Aggregators smooth out those differences to give a false sense of where a race really stands. He criticizes the media, too, for relying too much on the aggregated numbers, which reduce complex polling data to simple horse-race narratives devoid of reporting with nuance.
“Polling is equal parts art and science,” Zogby said. “The context in which data is taken can be as critical as the data itself.” As this chapter makes clear, individual polls need to be weighed with great care particularly their crosstabs and demographic breakdowns-before any accurate conclusion can be reached through any aggregated average. It is very easy for aggregators to miss regional or demographic trends that might be vital in interpreting an election.
Key Lessons – Chapter 6:
Beware the ‘Poll Aggregation’: Zogby puts a premium on the individual polls which tend to get lost in the mire of averaged-out results.
Methodology Matters: Every poll is different, and these are differences that can make a difference. According to Zogby, “Pollsters and analysts have to know what methodology is behind every poll if they are to accurately interpret the data.”.
The Media’s Role: Zogby also fingers the media, which focuses basically on aggregate numbers, calling for further digging by reporters into specific polls to get nuance in voter behavior.
