Beyond the Horse Race, Chapter 3 Summary

John Zogby recently released the book “Beyond the Horse Race, How to Read Polls and Why We Should.”  Here is a review of Chapter 3.

Chapter 3 details how polls can be both accurate and interpreted correctly. John Zogby outlines what goes into constructing a poll that is going to be sound; it is not about asking questions but asking the right questions to the right people. He puts particular stress on how samples selected and how questions are framed constitute the base components a poll’s reliability stems from. This means that all the samples taken by pollsters should resemble large populations, and all the questions should be framed to get candid and reflective answers.

He also provides a behind-the-scenes look at various methodologies of his market research company, Zogby Strategies. He speaks to some of the hybrid polling techniques, like the most recent mixture of telephone, online, and even text message-based invitations. He goes even further to note that modern polling can no longer depend on any one method, especially in view of the trends in which people communicate. The chapter also addresses Zogby’s emphasis on weighing data to account for demographic disparities, something imperative in any qualitative research company or opinion survey company desiring to show accurate results.

The important thing to take away from this chapter is the understanding that not all polls are created equal. Zogby argues that, in today’s world, it is farfetched to trust the overall national average simply because regional, age, sex, and ethnic differences could be so distinctively far. According to Zogby, at his company consistency and reliability are built into the polls by a concentration on “crosstabs” – what polling data broken down into the main demographic groups. Such in-depth analysis separates the chaff from the wheat in polling.

Zogby also writes about important wordings in survey research.

He tells of how badly framed questions bring about inaccuracies due to the bias or confusion introduced. Leading a respondent by a question or ambiguous terms within it will lead people to give a certain response and hence yield faulty information. His company John Zogby Strategies, invests much time in redacting balanced and neutral questions to reflect the actual opinion of the public. 

It is a must-read for anyone who works with market research companies or any form of polling services, giving quite a realistic view of how to ask the right questions and read the answers in the right manner.

Key Lessons From Chapter 3:

Sampling: The watchword of any good poll is its sample. According to Zogby, samples should be “truly representative” of the population. Most lousy polls, he says, fall flat because their samples “aren’t remotely representative of the diverse electorate”.

Use Clear and Neutral Questions: The framing of a question in the findings of the polls may create all the difference. According to Zogby, “pollsters should avoid leading or biased questions that distort results”. 

Deep insights can be read from Crosstabs: According to Zogby, “Top-line numbers simply don’t tell the story.” A pollster has to go deep into crosstabs and find how some specific demographic groups are thinking and acting. In his words, herein lays the real value of a poll. 

Go Hybrid: As response rates for traditional telephone surveys continue to decline, Zogby urges his colleagues to go hybrid telephone, online, and text messaging to sample wider and more representative populations.