Why you shouldn’t trust polls…… Shut-up statistics is cool

In the 1936 general election of the USA it was Roosevelt vs Landon. With the USA climbing out of the great depression and the potential of the next world war on the horizon, it was an important election. Many media outlets vied to run the biggest and best survey in order to be the one that predicted the result of the election. To that end, the Literary digest sent out 10 million surveys to its readership and had an astonishing 2.4 million replies. The labor of opening 2.4 million letters and processing the results (without computers) was exceptionally expensive, but it would all be worth it because of the huge number of subscriptions they would sell by reporting the results of the election before the election had taken place. From this survey they reported Landon would win by a convincing 55% to 41% and with a sample of 2.4 million how could they be wrong? You undoubtedly see where this story is going; Roosevelt destroyed Landon by 61% to 37%. But a young statistics enthusiast predicted the election correctly with a survey 2% the size of the Literary digest poll, fixing his name in the lexicon of american politics forever – George Gallup.

Gallup did things differently, he would poll people by interview, not send in letters, and he would stratify and weight the poll so that each sub-group of the voting public would be weighted according the census data. That is, if 5% of America was working class and jewish, then Gallup would weight the responses he got from working class jewish people in his polls at 5%. Follow-up research by Squire et al. 1988 found that the biggest mistake the Literary digest had made was relying on the post (an ancient, pre-internet form of communication). This is a bad idea because people who take the time and effort to fill out a form and post it in tend to have different and/or stronger opinions than those who don’t. This makes the sample you receive by post fundamentally different to those who are voting in the actual election. Gallup got around this by interviewing people in a pseudo-random method. Internet review systems all suffer from same kind of error as the Literary digest poll, people with moderate opinions don’t tend to go to the effort to review a business or a movie, but if you lose my package  (FEDEX YOU D*CKS) then I AM GOING TO WRITE A REVIEW! So in short, don’t trust Stuff.co.nz or Dailymail.co.uk internet polls, in fact don’t trust any poll until you read how it is done. If it looks like it is done well, like the Gallup poll, then you can trust it  – but only a little.   

Squire, P.. (1988). Why the 1936 Literary Digest Poll Failed. The Public Opinion Quarterly, 52(1), 125–133. Retrieved from http://www.jstor.org/stable/2749114

Blackwell J. (2003). 1935: The poll that took America’s pulse
http://www.capitalcentury.com/1935.html

Note image is just a hilarious result from a poorly designed internet poll and has nothing to do with Gallup. 
Art from http://corex13.deviantart.com/art/Mutant-Insect-Monster-515839486