Incorrect polling data scrutinized by academic panel following 2017 civic election

Incorrect polling data published prior to the 2017 Calgary civic election has come under scrutiny from a report from the Marketing Research and Intelligence Association (MRIA).

In the days and weeks leading up to the October 16 election, multiple polls were released by Mainstreet Research that showed Bill Smith leading incumbent Mayor Naheed Nenshi by as much as 17 points.

See also

• Mainstreet Research CEO admits they 'f*cked up' election polling
• Naheed Nenshi re-elected Mayor of Calgary in 2017 General Election
• Nearly 1 million Canadians report spending over $250 on pot in 3 month period

However, when the votes were ultimately counted, Nenshi took the mayorship for a third term with 51.4% of the vote, while Smith only received 43.7%.

Three academics were called on by the MRIA to investigate the divide and to determine how the election was affected by the inaccurate polling data.

The panel also found that, while no poll published leading up to the 2017 election was close to forecasting the actual results that were seen — Nenshi winning by 7.65% margin over Smith — there were at least two other polls conducted that showed Nenshi winning by a large margin.

One was commissioned by the LRT on the Green Foundation, and the other by the Canadian Municipal Election Study.

“These polls tempered what turned out to be the erroneous narrative created by the Mainstreet polls that the mayor was likely to lose but may have added to public and media confusion,” the report states.

Even with the conflicting poll results, the report argued that the three Mainstreet polls were the most widely published as they were commissioned by Postmedia, the publisher of both the Calgary Herald and the Calgary Sun.

The report states that these polls significantly affected the course of the campaign.

“They threw Nenshi’s campaign on the defensive, gave impetus to Smith’s campaign, and possibly doomed the prospects of another candidate, André Chabot, who Mainstreet’s poll suggested was not a close contender.”

The report also called on the news media that covered the election for not being critical enough in its reporting of the polls, and that the coverage of the discrepancies between the Mainstreet polls and the others leading up to the election was not “technically sophisticated” enough to properly equip readers to evaluate the data.

“Moreover, Postmedia did not share with its own readers [the] concerns it had about the polls and the degree to which Mainstreet was altering its methodology to address them,” the report states.

Following the investigation, the MRIA has put forward dozens of recommendations to the media, polling firms, and to the general public/city.

The ones aimed at polling firms include instituting the continuous practice of updating disclosure standards, creating a standardized disclosure checklist that firms publish alongside their polls to act as a tool of accountability, a yearly report in the state of the polling industry, enhanced services to both the public and media that would let them better understand the polls they are presented with, and a code of conduct developed by polling firms that would guide their interactions with media.

When addressing the media, the panel wrote that they would recommend “the application of normal journalistic context and skepticism of polls,” an improvement in the technical skill set journalists have when reporting on polls, and the full disclosure of relationships when publications obtain exclusive polling data.

The report also asked that Mainstreet and MRIA agree upon a neutral academic auditor to evaluate Mainstreet’s current practices and standards.