Election Poll Miss Blamed on Unrepresentative Samples

Opinion pollsters in the lead-up to the 2015 UK General election systematically over-represented Labour supporters and under-represented Conservative supporters in their samples, according to the report of the BPC / MRS inquiry.

The inquiry, established by the BPC (British Polling Council) and the MRS the day after the election, declared the 2015 polls 'some of the most inaccurate' since election polling first began in the UK in 1945, and examined the reasons why. Almost all pollsters prior to the day predicted a close-run election and a hung Parliament, but the final result was 331 to the Tories and 232 to Labour - 36.9% and 30.5% of the total vote respectively. A BBC exit poll was the first to get close to the result, forecasting that the Tories would win 316 seats and Labour 239.

The inquiry considered eight different potential causes of the polling miss, and concluded that the primary cause was unrepresentative samples. By replicating all published estimates for the final polls using raw microdata, the inquiry team was able to exclude the possibility of flawed analysis, or use of inaccurate weighting targets on the part of the pollsters. It also found that the procedures used by the pollsters to handle postal voters, overseas voters, and unregistered voters made no detectable contribution to the polling errors, and that there was no difference between online and phone modes in the accuracy of the final polls. While there may have been a very modest 'late swing' to the Conservatives between the final polls and Election Day, this can only have contributed - at most - around one percentage point to the error on the Conservative lead.

On the basis of these findings, the BPC has made twelve recommendations, and has agreed to implement immediately rule changes that will require greater transparency about how polls have been weighted; specify what changes have been made since a company's previous published poll in how the data have been weighted or otherwise adjusted; and place an obligation on members to supply to any inquiry or committee that has been established by the BPC the micro data set for any poll in which that inquiry or committee has an interest. In addition, the BPC advises that The Economic and Social Research Council (ESRC) should fund a pre as well as a post-election random probability survey as part of the British Election Study in the 2020 election campaign.

The BPC has agreed that work should be undertaken to develop an industry-wide method for calculating the confidence limits associated with a poll's estimate of a party's share of the vote, along with an industry-wide approach to calculating the statistical significance of the change in a party's estimated vote share since a company's previously published poll. Professor John Curtice, President of the BPC, commented: 'The inquiry has undertaken what was an important but demanding task in a timely and professional fashion. The council now wishes to ensure that its work is put to best use so that the transparency and accuracy of opinion polls is enhanced in future'.

Web sites: www.britishpollingcouncil.org and www.mrs.org.uk .

All articles 2006-23 written and edited by Mel Crowther and/or Nick Thomas, 2024- by Nick Thomas, unless otherwise stated.