Who in the World Is Still Answering Pollsters’ Phone Calls?

We’re already in the field with our next New York Times/Siena College national survey, so it’s a good time to go through some of the poll-related reader mail we’ve received recently.

Here’s a version of a question we get a lot:

Given that pollsters are relying on calling people on the phone (per your methodology description at the bottom of the poll), how do you know where they are, and how do you account for the fact that so few people answer their phones at all anymore? I for one have moved twice since I got my current phone number, most recently to a different state, so my phone number has nothing to do with my actual location. Meanwhile, most of my calls are spam, so I almost never answer my phone unless I recognize the phone number — and I am someone who is old enough to have grown up with what is now called a landline. My teenage kids almost never answer their phones at all. The only people I know who still ever use a landline at all are my parents. — Doug Berman, West Jordan, Utah

There are a lot of good points here, so let’s take it bit by bit.

How do we know where they are? Some pollsters (like us) call voters from a list of telephone numbers on a voter registration file, a big data set containing the names and addresses of every registered voter in most states. The addresses tell us “where they are” with a great deal of precision.

How do we deal with people who have moved? The voter file offers a solution to this problem as well. Once you’ve registered to vote in your new state, pollsters can call your phone number if it’s on the voter file — and call it regardless of whether it’s an in-state or out-of-state area code.

How do you account for the fact that few people answer? Before I respond, I want to dwell on just how few people are answering. In the poll we have in the field right now, only 0.4 percent of dials have yielded a completed interview. If you were employed as one of our interviewers at a call center, you would have to dial numbers for two hours to get a single completed interview.

No, it wasn’t nearly this bad six, four or even two years ago. You can see for yourself that around 1.6 percent of dials yielded a completed interview in our 2018 polling.

The Times has more resources than most organizations, but this is getting pretty close to “death of telephone polling” numbers. You start wondering how much more expensive it would be to try even ridiculous options like old-fashioned door-to-door, face-to-face, in-person interviews.

Call screening is definitely part of the problem, but if you screen your calls almost 100 percent of the time, it might be a little less of one than you might think. About one-fifth of our dials still contact a human. But once we do reach a person, we’ve got a number of challenges. Is this the right human? (We talk only to people named on the file, so that we can use their information.) If it is the right person, will he or she participate? Probably not, unfortunately.

OK, back to the question: What do we do to account for this? The main thing is we make sure that the sample of people we do reach is demographically and politically representative, and if not, we adjust it to match the known characteristics of the population. If we poll a state where registered Democrats outnumber Republicans by two percentage points, and our respondents wind up being registered Democrats by a four-point margin, we give a little less weight to the Democratic respondents.

We make similar adjustments for race; age; education; how often people have voted; where they live; marital status; homeownership; and more. As I explained last month, we believe our polls provide valuable election information. Is all of this enough? After 2020, it’s hard not to wonder whether the people who answer the phone might be more likely to back Democrats than those who don’t answer the phone. We’re conducting some expensive multi-method research this fall to help answer this question, to the extent we can. We’ll tell you more at a later time.

What about cellphones? Finally, an easy one: We call cellphones and landlines! About three-quarters of our calls go to cellphones nowadays — including nearly every call to young people.