Over the 10 years I’ve been a pollster, my field has been collapsing.
It used to be that the industry standard was calling people on their landlines. As response rates went into free fall, pollsters largely switched to web polls, where companies recruit people to serve as survey takers on online panels, usually in exchange for money.
The panels are then weighted to bring the sample in line with the population the pollster wants to represent. Online surveys are much, much easier to field — a lot of expensive labor was required to make those landline calls and record the results — but far harder to get right.
When using a web panel, you have to make decisions about what you believe the electorate will look like, down to exactly how many people of what demographic to sample. This is a vastly different universe than old-school random-digit dialing, where pollsters would call people at random. . Before, when most voters had landlines that they would answer, the sample of people polled was people with phone numbers. Now, the sample of people polled is people who sign up to take polls. You can try to correct for the racial skew or the gender skew or the age skew, but it’s very difficult to correct for unobservables like the interested-enough-in-politics-to-volunteer-for-a-poll skew.
As a result, you now have to aim for highly specific demographic targets when constructing your poll: how many nonwhite people with college degrees, how many men over 65, and so on. Pollsters often create those targets by studying their prevalence in prior elections, which works in a stable environment but leaves them vulnerable to error when the makeup of an electorate shifts in ways they don’t expect. Essentially, every single poll silently bakes in the pollster’s assumptions about who will show up on Election Day.
After pollsters widely overestimated Abdul El-Sayed’s expected margins in the Michigan Democratic Senate primary, and then had a similar error overstating Francesca Hong’s strength in the Wisconsin Democratic gubernatorial primary, there was a lot of hand-wringing about what assumptions were made about how young the electorate would be. But that wasn’t a matter of people missing an obvious answer! It was a matter of them guessing wrong in a world where they had minimal guidance to make choices that could dramatically skew their results.
I used to think of “polling” as a process where you measured something concrete about the world, and “election forecasting” as a process where you took those concrete inputs, put them in a blender with historical information and your best guesses, and extracted a prediction. That line has now significantly blurred.
As a pollster, I think accurate information about public opinion is incredibly important for the functioning of a democracy. It’s the best way for politicians to know what voters care about, since the vast majority of voters will never directly speak to a politician. That’s why it pains me to say that what I’ve described above is what we should expect when contemporary polling is done well. There will continue to be high-profile polling misses: Thoughtful people struggling to figure out how to match their nonrandom sample of polling respondents to a future, unknown voting population are going to make mistakes, even when doing their best.
The problem is, polling has also become a playground for grifters, liars, and frauds.
The state of polling has gotten so bad that we now have multiple categories of “fake poll”: AI-generated synthetic data (you did something, but that thing isn’t polling), skewed or motivated results (you did something, it was polling, but you did a bad job), and ... what the authors of a recent fabricated poll in the Los Angeles mayoral race are calling a “social experiment” (you did literally nothing and just made some numbers up).
Another pollster, The Public Sentiment Institute, recently revealed that while polling in the Florida governor’s race, it actually switched some respondents’ answers from one candidate to the other simply because its results didn’t match its expectations.
I am, frankly, sick of watching all of this. And it’s only going to get worse.
It’s really hard to stop polling frauds
In an ideal world, we’d have some sort of mechanism to prevent trash polls from getting publicity, but we don’t have that. There’s no true central body regulating polling. There isn’t an authority that can stop you from simply making stuff up and calling it a poll. The American Association for Public Opinion Research’s Transparency Initiative is a useful standard, but it only requires disclosure of data collection and sampling methods, rather than dictating best practices. Plus, I doubt very much that people who don’t work in the polling field are aware of it at all.
If everyone posted their full toplines and crosstabs every single time they did a poll, it would be harder to generate a fully fake poll, though not impossible. (You can do a lot with LLMs these days.) The larger issue is that there’s no way to force people to adhere to this standard or prevent people from promoting polls that fail to meet it.
Since many of the media groups that aggregated and wrote about polling have broken up or shrunk dramatically (RIP HuffPost Pollster, RIP FiveThirtyEight), there are fewer and fewer people who are going to check the details.
This isn’t a problem caused by aggregators like Nate Silver, who are mostly pretty careful about what goes into their models and averages. It’s a problem caused by randos on Twitter. You can just tweet or skeet or post whatever, with no link — in fact the platforms sometimes reduce your reach if you include a link! — and people will report on it. For instance, that fake Los Angeles mayoral poll showed Mayor Karen Bass up 12 points over her competitor, Nithya Raman. The California Post and local broadcasters covered these numbers, and Bass’ campaign promoted them on Twitter. Her campaign spokesperson subsequently explained that the poll was seen as legitimate because it was “reported on by multiple news outlets.”
In March, a group called Heartland Forward released a “maternal health poll” claiming that 88% of Americans considered maternal mortality a serious problem. Axios reported the findings uncritically; then, after it was revealed that the “pollster” was actually an AI firm that had simulated thousands of respondents, appended an editor’s note that readers had to sign up for a newsletter to see.
In August, a “synthetic audience” study of attitudes toward data centers made the rounds in the climate change research world before careful readers pointed out that no actual humans were surveyed. Around that same time, a polling aggregator account posted an Arizona poll falsely attributed to Data for Progress.
Well-staffed media platforms like More Perfect Union post unsourced results all the time. It drives me crazy. And some of the most popular sources for disseminating polling are pseudonymous miscellaneous accounts that don’t include links or methodology, or even necessarily check that the polls exist. The platform incentives drive people toward unsourced screenshots and catchy toplines without doing the bare minimum to verify that they’re not popularizing shoddy polls.
Shoddy polling makes it more difficult to have serious conversations about what voters want, makes it impossible to evaluate candidates, and breeds distrust of the electoral system and voters themselves.
More polling, but at what cost?
The lower cost of web-first polling led to an explosion of smaller polling firms. I helped set up one of these, Data for Progress, in 2020. I think this was mostly good, because it made polling available to types of campaigns and organizations that couldn’t afford the white-glove service provided by traditional polling firms. At the time, you could get novel polling into the news fairly easily, which was an enormous boon to people trying to advocate for popular left-leaning ideas that might otherwise be ignored.
The problem was that it also became cheap for advocacy groups to pay for polls that would confirm their priors and comfort their donors.
The subsequent flood of cheap, often low-quality polling has made “having a poll showing your idea is popular” a much weaker signal than it used to be. Even worse, the rise of betting and prediction markets creates a financial incentive to fabricate polls.
Prediction markets are enormously responsive to new polling results, to a degree that makes me seriously doubt their overall predictive power. In the New York City mayoral race, a very late poll led to Zohran Mamdani’s odds dropping, and in 2024, the now-infamous Selzer poll moved Polymarket odds of a Kamala Harris victory in Iowa from 6% to 18%. If you can drop a new poll (or “poll”) into the mix, you can make a pile of money. Doing this with fabricated polls is clearly fraud.
A distorted polling environment can be a response to more benign incentives as well. One is the perennial desire of campaigns to be seen as gaining ground. There’s a reason aggregators discount the results of internal polls. Even if the pollster is doing their absolute best to be accurate, a poll commissioned by a campaign isn’t going to be released if the numbers aren’t favorable. That isn’t the fault of the pollster — and it doesn’t say anything about their accuracy — but it’s a statistical skewing of what polls you’d expect to see in public.
And with the rise of entirely fake polls, I’m concerned that we’re going to begin seeing such polls released to help pump up flagging campaigns.
This terrifies me for 2028. We’re riding a wave of anti-institutional sentiment. We’re about to go into a contested multiway presidential primary. When you add in fake polls, you set voters up to be living in entirely different informational universes. Democratic voters have shown us that they’re responsive to information about who can triumph against Republicans in the general election. If that information is faked, it will be a betrayal of their trust.
A 2028 Democratic primary could easily devolve into a mudslinging mess. It’s going to be the biggest news story in the country for months. The incentive for random people to make things up for clicks will be huge. We’re barely prepared for miscellaneous TikTok personalities to post random lies about the candidates. I think we’re not at all prepared for numbers that look and feel reputable to be utterly fake.
Promoting fake polls should be punished
There are a couple of ways that we can reduce our exposure to this threat. In 2020, the Democratic National Committee had to determine which candidates in a crowded field would be invited to debates. One way it did this was by setting a minimum threshold for candidate support as measured by a set of reputable polls.
Using polling as a qualification threshold in 2028 would, in my opinion, be a mistake, even if reputable pollsters are selected in advance. Even in 2020, some candidates like then-Rep. Tulsi Gabbard claimed that the polls included in the DNC threshold were less accurate than other, excluded polls that would have put her on the debate stage.
Voters who believe their candidate is being unfairly suppressed by the DNC have already shown us that they will never, ever get over their distrust (see many Sanders supporters for examples of this). It could be catastrophic for the primary winner to be seen as illegitimate by a chunk of the Democratic voter base.
On the betting market side, the platforms need to take fake polls seriously as a type of fraud. They should very publicly hand out perma-bans to anyone engaging in bets that seem to be tied to fraudulent polling, particularly when those involve high-profile elections. They need to make this not worth the risk and prove to the public that they’re taking it seriously. When Polymarket is consistently doing tweets about questionable or purely false information to drive clicks, it’s hard to believe that it cares.
Political institutions like the Democratic Senatorial Campaign Committee and DNC should make it clear that lying to the public is a red line. Candidates and consultants who are involved with false polling, leaked or otherwise, should face professional sanctions. Campaigns that fundraise off fabricated polls should correct the record and offer refunds to donors solicited with those claims, even if the campaigns were initially deceived themselves. Campaigns and consultants who knowingly or recklessly promote fabricated polling should also face professional sanctions.
These consequences cannot always be contingent on whether people knew the poll was fake at the time. Campaigns should get far more cautious about what information they promote to their followers and donors. If this leads them to pull back on boosting unvetted polling as a whole, that will be a very good thing.
Recommended reading:
Four things I changed my mind on after a year of polling
When The Argument‘s editor-in-chief Jerusalem Demsas and I first began talking about building a polling program, we were both obsessed with one problem: Issue polling was dominated by interest groups.
The election experts and the models can’t both be right
Since The Argument and Split Ticket released our 2026 midterms model, I keep being asked the same question: “Isn’t that awfully optimistic for Democrats?”
This is the best and most thorough ‘state of polling and how not to make it worse’ piece I’ve read in years. The small part I try to play in this is to discourage philanthropies from commissioning expensive polls designed to rationalize what they already want to do anyway. But basically it’s a war against confirmation bias at every level.
I want to stick up for the pollsters in the Wisconsin governor’s race. There’s not much evidence that they did a bad job—it was just an extremely volatile race where the facts on the ground kept changing. Notably, there were always high numbers of undecideds, and candidates kept dropping out. It’s entirely possible that the polls that were taken provided an accurate snapshot of the electorate as it existed when the poll was taken but undecideds made up their minds faster than they could be measured. Notably, Hong’s numbers in the polls were relatively stable for the last month or so and more or less matched the final outcome, which is what you’d expect if the problem with the polls was that undecideds were making up their minds.