The Directory News
Article Archive

Did This Poll Ever Happen?

By Nicholas Molinelli

•
PoliticsNationalAnalysis
Did This Poll Ever Happen?

A fake polling firm slipped fabricated numbers into American elections and called it a “social experiment.” The more troubling question is how voters are supposed to know which polls are real and what pollsters and journalists owe the public before asking them to believe one.

A poll can be wrong, sometimes spectacularly wrong. That is not particularly new.

What happened this month is different.

An organization calling itself Median Strategies published what appeared to be ordinary political polling in multiple American elections. In Wisconsin, it released numbers showing Democratic gubernatorial candidate Francesca Hong leading Milwaukee County Executive David Crowley by 23 points.

Crowley won the Aug. 11 Democratic primary by roughly half a percentage point.

At first, Median Strategies responded the way a legitimate pollster might after a terrible miss: it acknowledged that it had gotten Wisconsin badly wrong. Then came the admission.

The poll had not merely been inaccurate. It had never happened.

Median later said its operation was a “short-term social experiment” designed to test how purported polling information could enter and spread through the political information ecosystem without independent verification. The organization withdrew its polling and declined to identify the people behind the project.

That confession changes the question.

The most disturbing thing about Median Strategies is not that a fake poll was wrong.

It is that, had its creators never admitted the poll was fake, how would the public have known?

The Poll That Wasn’t

Wisconsin was not Median Strategies’ only experiment.

In Los Angeles, the organization published another purported survey showing Mayor Karen Bass leading Councilmember Nithya Raman ahead of the city’s November runoff. The poll looked ordinary enough: field dates, a sample size, a margin of error and professional-looking presentation.

It also traveled.

A news outlet reported the numbers. Bass’ campaign amplified them. Former mayoral candidate Spencer Pratt shared coverage of the poll with a large online audience. The information moved far beyond the tiny organization that had created it.

Then Median acknowledged that survey was fabricated too.

The operation had successfully demonstrated the weakness its creators said they were testing: numbers produced by an unknown organization could acquire the appearance of legitimate political information, enter news coverage and spread before anyone had established that the underlying research actually existed.

The Wisconsin poll may be even more revealing because it did not look absurd.

It looked plausible.

Wrong Is Not the Same as Fake

That distinction matters.

There is no evidence that legitimate Wisconsin pollsters fabricated their research simply because they also missed the final result. Some transparent, established polling had Hong well ahead of Crowley shortly before the election.

The race then moved dramatically. Candidates left the field. Crowley consolidated support. Gov. Tony Evers backed him. Late-deciding voters broke heavily enough toward Crowley for him to narrowly win.

Pollsters can conduct real research honestly and still get an election wrong.

A legitimate poll is a measurement, and measurements contain error. Pollsters must determine who is likely to vote, find people willing to respond and weight a comparatively small sample to resemble an electorate consisting of hundreds of thousands of people.

Median Strategies did something fundamentally different.

It did not conduct a survey at all.

That creates an uncomfortable problem: if a legitimate poll can miss an election by 20 points, then being wrong by 20 points cannot tell us whether a poll was legitimate in the first place.

A real poll can be badly wrong.

A fake poll can accidentally be right.

Election results cannot authenticate the research after the fact.

So How Do We Know?

Most voters cannot audit a polling organization, and neither can most political reporters.

When a poll appears online, readers generally rely on familiar signals of legitimacy: a company name, a website, a sample size, a margin of error, charts, demographic tables and a methodology paragraph.

Those signals normally accompany real research.

Median Strategies showed that the appearance of that research can be reproduced without doing the research.

That matters because the most effective fake poll may not be an outrageous one. It may be one that looks almost exactly like everything else being published.

Median’s Wisconsin numbers placed Hong 23 points ahead. Legitimate polling around the same period also showed Hong holding large leads.

The fake number hid among real numbers.

That should worry anyone who assumes fraudulent polling will always expose itself by looking ridiculous.

A malicious operator would have no reason to publish obviously unbelievable numbers. It could publish ordinary results, build a track record and occasionally be right. Then, when a particular election mattered, it could move the numbers.

If the poll missed badly on Election Day?

Polls miss elections.

That gives fraudulent polling a remarkable form of camouflage.

A Poll Is a Claim About Research

Election polls are often treated as predictions.

That is not quite what they are.

A poll is fundamentally a claim about research. The organization publishing it is saying:

We asked these people these questions, using this method, during this period, and this is what they told us.

Whether the election eventually matches those results is a separate matter.

A legitimate poll can show Candidate A ahead and still watch Candidate B win. A fabricated poll does not become legitimate simply because its invented result happens to predict the winner.

Accuracy cannot prove that the research occurred.

Transparency can help.

The American Association for Public Opinion Research already maintains extensive standards for publicly released surveys. Those standards call for disclosure of who sponsored and conducted the research, who funded it, who was surveyed, how respondents were recruited, the method of data collection, field dates, sample size, question wording, weighting procedures and relevant limitations.

The principle is straightforward:

Show your work.

The problem exposed by Median Strategies is that the political information system does not always demand that work before amplifying the result.

Polls Can Change the Race They Measure

This matters because polling does more than satisfy curiosity about who might win.

Campaigns use favorable numbers to raise money. Donors use them to judge viability. Political committees decide where to spend. Journalists decide which races deserve attention. Volunteers choose where to devote their time, and voters hear which candidates supposedly have a chance.

Polling does not merely describe the political environment.

It can help shape it.

There is currently no evidence that Median Strategies changed the outcome of Wisconsin’s primary or caused a measurable number of voters to stay home, donate differently or switch candidates. Claims that its poll drove Crowley from the race also fail on chronology: Crowley suspended his campaign and later reentered it weeks before Median’s Wisconsin operation became public.

But the broader possibility deserves attention.

Imagine a congressional primary, county race or city council election where relatively few people vote. Several apparently professional polls suddenly show one candidate losing badly.

Does a donor decide not to write the check? Does a volunteer move to another campaign? Does a newsroom stop treating the race as competitive? Does a voter conclude that supporting a preferred candidate is pointless?

Those effects would be difficult to measure after the election.

They also do not fit neatly within the traditional definition of voter suppression. No polling place was closed. Nobody was falsely told Election Day had moved. No one was prevented from casting a legal ballot.

But manufactured information about political viability raises a related question:

Can political participation be suppressed by manufacturing the belief that an election is already over?

We do not know that Median accomplished that.

We should still ask whether the tactic could.

What Do Pollsters Owe the Public?

If pollsters want the public to distinguish honest uncertainty from invented data, they have to offer something more than a topline number.

They owe the public verifiability.

Who conducted the poll? Who paid for it? Who was surveyed? How were respondents selected and contacted? When was it conducted? What were they asked? How was the data weighted? What limitations does the pollster acknowledge?

Increasingly, there may be another reasonable question:

Who is willing to put a real name and professional reputation behind the work?

New polling firms should be allowed to establish themselves. But an unknown organization should not automatically receive the same presumption of credibility as an institution with years of transparent methodology and accountable leadership.

Trust should be earned.

What Do Journalists Owe the Public?

Pollsters are only half of this story.

Median Strategies could have published fabricated PDFs forever and accomplished very little if no one amplified them.

Journalism is one of the transmission mechanisms.

Political reporters receive polling from campaigns, PACs, universities, advocacy groups, consultants and independent research firms. Often the headline becomes the number while methodology is pushed toward the bottom.

Median suggests the order should occasionally be reversed.

Before reporting what a poll says, a newsroom should establish a reasonable basis to believe a poll actually occurred.

Who is the organization? Who conducted the work? Who paid for it? Is the questionnaire available? Is the methodology plausible? Are identifiable people willing to answer questions about how the survey was conducted?

If those basic questions cannot be answered, a newsroom may not have polling.

It may have a PDF containing numbers.

Those are not the same thing.

The Most Frightening Part Is the Confession

Median Strategies ultimately exposed itself.

Its creators withdrew their polling and told the public that the surveys had been fabricated. That is why we know this happened.

They have also left substantial questions unanswered.

Who were they? Why did they select these races? Why these candidates? Was anyone associated with an academic institution? Was the experiment reviewed by anyone? Who funded it? What records exist?

Median has offered explanations for its motives, but those explanations are coming from an anonymous organization whose stated project involved intentionally presenting false information as true.

Skepticism is appropriate.

Even if every explanation eventually checks out, however, the experiment leaves behind a more important result.

It worked.

A fabricated survey entered political discourse because it looked enough like legitimate polling to be treated as legitimate polling.

And we know that only because the people who created it eventually told us.

Had they simply disappeared — or continued publishing — its Wisconsin poll might have been remembered as one more miss in an election where legitimate pollsters also got the result badly wrong.

That should bother pollsters, journalists, campaigns and voters alike.

The lesson is not that polling cannot be trusted.

It is that trustworthy polling requires the public to trust the process, not merely the prediction.

A reputable pollster should be allowed to get an election wrong. Research requires room for uncertainty and failure.

But in exchange for that tolerance, the public should be entitled to know that the research actually happened.

Before we argue about whether a poll is right, perhaps there is a more basic question we should start asking:

Did this poll ever happen?

Did This Poll Ever Happen? | The Directory News