Midterm Elections Live Updates: Trump Visits Ohio as Obama and Harris Lead Georgia Blitz

Are polls overestimating the Democrats’ chances? What are we doing to accurately capture support for candidates? I’ve been thinking about these questions in my current role focusing on reader trust in The Times. For instance, we published a poll this month showing a six-point lead for James Talarico, the Democratic nominee for Senate in Texas. Many other polls also show Talarico leading. Given the base-line partisanship of Texas and voters’ track record of electing Republicans there, regardless of headwinds, I wondered whether readers would trust our polls or question their credibility.

I recently spoke about polling and the midterms with Nate Cohn, our chief political analyst, and William P. Davis, who leads our Election Analytics department. They work on our Times/Siena Poll, which is widely considered to be one of the most accurate polls in U.S. politics, and Nate recently explored the question of polls and Democrats’ chances in his newsletter, The Tilt. Our interview has been edited and condensed.

Let’s start with a point-blank question. Should people trust these polls showing Democrats running ahead or neck-and-neck in red states?

NATE COHN: You can trust them to a degree. You can trust the polls to usually be in the ballpark, but you can’t trust them to be exactly “right.” Polls are inherently imprecise and subject to many sources of error.

So if by “trust” you mean “assume that the leading candidate will win,” then the answer is “no.” The polls have been wrong by this standard before and they will be wrong again. But if trust means “rely on the polls as the best source of information on the electorate, even if an inexact one,” then the answer is yes.

WILLIAM P. DAVIS: There are a lot of indicators pointing in the same direction this year — polls of specific races, President Trump’s approval rating, the fact that Republican candidates in deep-red states are distancing themselves from the president, the expectation that Democrats have a turnout advantage in midterm elections, etc.

The question becomes: How good of a year will it be for Democrats? Polls suggest a pretty good one. But whether the national results swing eight or 10 or 12 points to the left from 2024 will make a material difference in whether or not Democrats are able to take the Senate, and polls can’t guarantee that level of precision.

But some people read a Times poll headline saying so-and-so is ahead and take that to the bank. What do you say to people who see our Senate polls and regard them as predictions for Election Day?

WILL: Polls measure public opinion at a specific point in time. If people are still making up their minds, or late-breaking information upends a race, any prior poll will miss it. Polls can help us understand which races are close, and why; we won’t know until the results are in how they will shake out.

As rigorous as our polls are, I’m a little skeptical that a Democratic Senate candidate will win in Texas, given historical candidate performance there. What’s your pushback on that?

NATE: I’d say that you shouldn’t be quite so skeptical. Over the Trump era, Texas Republicans have become dependent on elevated levels of support from Hispanic voters. Without them, Texas is highly competitive, as Republicans have lost enormous support among white voters in the state’s highly educated suburbs. Every available piece of evidence — from our polls to on-the-ground reporting — suggests that the G.O.P.’s gains among this group have evaporated.

For good measure, the G.O.P. has nominated a candidate — Ken Paxton — who has been investigated, indicted and impeached. Add in a hostile national political environment for Republicans, and I don’t think there would be anything surprising about a Democratic win in Texas. Nearly every nonpartisan poll since at least Labor Day has shown Talarico ahead.

WILL: It’s reasonable, even good, to be skeptical. But people also have a false memory of Texas polls in the past. Most polls did not show Beto O’Rourke ahead in the 2018 Senate race against Ted Cruz. Our Times/Siena poll of that race had O’Rourke down eight points, and he actually only lost by 2.5 points, so we underestimated him a bit in that race. This year’s race is still pretty close, but it’s undeniable that Talarico is doing better there in polls than any Democrat in recent history.

I raised the question earlier of reader trust in polls. Readers have asked me if we’re “sure” that our Texas polling is right and whether we would kill the poll if we weren’t sure. How do you see that? And do we ever not publish polls that we’ve conducted?

NATE: It might seem strange, but it’s pretty important that we publish every survey result. We’ve never withheld a Times/Siena poll result. Many smart election analysts would have never imagined Trump winning in 2016, would have never imagined that Democrats could have avoided a “red wave” in 2022, or would have never believed Trump could surge among young and nonwhite voters in 2024 — if it weren’t for polls raising it as a real possibility.

And if The Times had withheld positive polls for Trump in 2016, we would have been accused of bias.

NATE: Yes. We would absolutely be accused of cherry-picking polls that fit a partisan bias or, even worse, in order to fit or construct a partisan narrative.

Of course, there are sometimes good reasons to wonder whether our poll is in the ballpark. That’s especially true when there’s a lot of other polling or our reporting shows something different. When we get an outlier like that, we can be cautious about the result in our write-up.

Even when the poll does appear to be a true outlier, it doesn’t mean it’s wrong. In 2022, I downplayed the result of a poll in Kansas’s Third District, which showed the Democrat Sharice Davids ahead by 14 points. This seemed totally bonkers. The race was considered a true tossup, with both parties spending millions of dollars. One Democratic pollster called the result “cray” and sent the laughing-while-crying emoji. In a very rare move, the Democratic campaign even disputed the result: “I don’t know what model The Times is using but this is going to be a very tight race. Our polls show it’s a true tossup at this point.”

In the end, Davids won by 12 points. I still get flack for downplaying the result.

Even modest polling errors in the past have underestimated Republican strength. Why is that, and what have we done to account for it?

WILL: Poll error from years when Donald Trump has been on the ballot are top of mind for most people, and with good reason. The chief theory for why polls have missed relates to who polls have a hard time reaching: low-education voters who don’t vote as regularly. If those voters overwhelmingly favor one candidate over another, and if polls don’t reach enough of these voters, that can add up to the sorts of issues many pollsters saw in 2016.

At the Times/Siena Poll, we put a lot of effort into reaching these voters, in part by placing more calls to voters who are less likely to respond. That ends up costing us a lot more money, but it pays off: In 2024, we saw early on that Trump was building a lead with these voters.

And we are continually reviewing how our poll performs and making improvements. In 2024, we made a change to keep in our poll results respondents who hang up mid-survey. This year, we made significant changes to how we weight our polls, with the goal being to better account for the types of people we have a hard time reaching.

Do we know how to poll Democrats better than we know how to poll Republicans?

NATE: Not necessarily. When polls underestimate Republicans, it’s entirely possible that Democrats contribute to the problem. Imagine, for instance, if the “resistance” Democrats are much likelier to take a poll than a working class registered Democrat who sometimes votes for Trump.

What else have we learned from past polls, and what are we doing differently this year?

NATE: The thing that we’ve learned over the last decade — and that the industry as a whole has learned — is that even a perfectly designed survey will probably underrepresent Trump’s supporters.

As Will mentioned, we’ve adopted a new method for “weighting,” the technique pollsters use to give more weight to underrepresented groups. The new approach lets us leverage the rich data we have on our survey respondents while weighting the poll to more thoroughly represent the population. This approach improved our past polls when we took the data from our old surveys and applied the new weighting approach. It would have especially helped in the white working-class states where polls have erred in the past, but that’s no guarantee of better results in this cycle or the future.

WILL: There are serious challenges to making sure you have the right methodology, that you’re reaching a representative group, that your weighting is correcting for the things that matter most. We sweat every one of those decisions.

And there are other ways to mess up beyond the methodology of the poll. The specific wording of questions is difficult to get right. Our colleague Ruth Igielnik leads our questionnaire development process, and we often spend weeks refining the specific language we use. But we sometimes realize after scrutinizing the results, reviewing recordings of calls, calling back voters and comparing our results to other polls that we didn’t quite get it right.

One example was in 2023, when we found that more than 20 percent of voters in the presidential battlegrounds said they would support Robert F. Kennedy Jr. for president. We had a lot of reasons to believe that number was way too high: We compared our numbers to what others were finding, we looked at what respondents supporting Kennedy said in other questions, and we had a theory about what caused the issue.

We still released the results, but we worked hard to place it in the proper context when we wrote our story about the finding. It was uncomfortable for us, to somewhat disavow our own finding. But we thought it revealed something about how unpopular Trump and Biden were, and we tried to explain that. And a few months later, we conducted an experiment to try to prove out our theory of why we got such a flawed result. Our theory was proved correct, and we wrote that up, too, to try to help readers better understand how polling works.

We reported the other day that a fake polling firm had deceived The Times and other polling aggregators with several phony poll results. They said they were trying to “demonstrate the vulnerabilities” in the political polling field. Size this up — what did they prove and what did they not?

WILL: It’s worth noting that the “polls” put out by this group, which went by Wedgewood Polls, had no bearing on our own Times/Siena polls. They appeared in our poll trackers but did not receive prominence in our coverage.

Wedgewood certainly proved that we’re not perfect. But they claimed that The Times and others had included their polls without vetting them, and that’s not true. The Times vets all new pollsters, asking a series of questions designed to determine whether a group is attempting to field representative polls, learn where their funding comes from.

This process has weeded out untrustworthy pollsters. But Wedgewood deceived us — as they appear to have deceived others who asked similar questions or even gave them money. It’s not clear to me that their true goal was to demonstrate vulnerabilities. We are putting a lot of thought into how we could have found these issues out earlier.

One thing the episode highlighted is the way that newer pollsters can attract attention. The group was able to quickly gain traction, in part by claiming to be associated with another polling group and in part by polling some races that had little polling.

It’s gotten evermore expensive to field a high-quality poll and ever-cheaper to field a low-quality one. And A.I. is making it easier than ever to fake things. The Times/Siena Poll is one of a few pollsters left with the resources to both put a focus on quality and poll many of the key races each election. Increasingly, pollsters are having to choose between one or the other.

Last question. If Election Day results are materially different than our polling suggests, is the polling worth the risk to our credibility?

WILL: In the end, I will regard how well we did as journalists by how well our coverage reflected what ended up happening.

If Democrats do really poorly, we should do some soul searching about both our methods and the questions we asked, as we would for any other story we got wrong.

If Democrats do well but fail to take the Senate, I would ask myself how well our coverage prepared readers for that possibility. I hope the answer is that it prepared them well, because we’re constantly talking about how to ensure our coverage prepares people for this scenario. I would also ask how well our poll questions helped us understand and explain why these races were so close.

Similarly, if Democrats do much better than expected, I would ask myself how well we understood the circumstances that caused so many Trump voters to either stay home or switch to Democrats.

NATE: Polling isn’t like a lot of the other journalism at The Times. Most of the report is about facts. If Peter Baker says that Trump’s going to Illinois tomorrow, that’s a fact. A poll is not a fact. It’s a best measurement. There’s no guarantee that a “best” measurement is good enough — whether in terms of The Times’s credibility or whether it improves our understanding of politics. Historically, there have absolutely been elections when it didn’t feel like polling passed that test. The 2016 election is the easiest example; there will be another election like it again.

That said, readers come to The Times to understand American politics, which includes an understanding of public opinion during an election. For better or worse, polling is really the only quantitative measure of the attitudes of the electorate. There just isn’t an alternative. The polls will still drive the political conversation, even if The Times doesn’t conduct any of its own polling. Campaigns run on polls. Our coverage would inevitably be affected by the polls that others are conducting. If that’s going to be the case, it’s probably better that we’re conducting our own.