Dodgy use of opinion polls

This graphic has been doing the rounds on Twitter. It is perhaps one of the most misleading use of opinion poll data I’ve ever seen. 

The graphic comes from this Ipsos MORI poll. Here is the link to the charts for the poll and I’ve reproduced the page with the graph on below. 

You can see there are actually two graphs. The one on the left is the one at the start of this blog post and is the one doing the rounds on Twitter. It shows Labour with a 5 point lead. The one on the right is shows a 1 point lead for the Conservatives. 

So what’s the difference? The one on the left gives the voting intention for all the people sampled. The one on the right gives “headline voting intention” which weights the result to take account of the likelihood of people to actually vote. 

This is massively important because if you don’t do this you get a distorted picture. Indeed in the 2015 general election the pollsters overestimated the Labour vote share. 

In their post election analysis the pollsters came to the conclusion that the reason they overestimated the Labour vote share was because they overweighted the likelihood of many Labour voters (particularly younger voters) to vote. 

So by choosing the graph on the left the person who tweeted this has presented a distorted picture that magnifies the errors in the polls done prior to the 2015 general election. No one credible ever quotes opinion poll results without weighting for turnout. 

They have either been guilty of blatant and deliberate cherry picking, or if I’m being generous, an unwitting confirmation bias. Either way it massively distorts the data. 

For the avoidance of doubt this post is not meant to be pro or anti any party, or party leader. It is meant to be pro-clarity and anti-distortion. 

And finally, I will add my six laws of opinion polls to help understand opinion polls more clearly. 

First law: If an opinion poll gives a result you don’t like and your defence is “they can’t be trusted” that’s fair enough.

Second law: If you have ever said “they can’t be trusted” but you have used an opinion poll to back up your preferred issue then you have broken the first law.

Third law: If the poll says what you would like it to say how sure are you that you are not guilty of confirmation bias?

Fourth law: Whilst voting intention polls are not great at precise predictions of vote share, they are broadly right. 

Fifth law: Remember every poll has a margin of error. Find out what it is.

Sixth law: It is dangerous to rely on one poll as the result could be an outlier within the margin of error (see fifth law) and/or you could be guilty of confirmation bias (see third law).