The Evidence Base

Informing Policy in Health, Economics & Well-Being
A collaboration with
USC Dornsife Center for economic and social research

Why Won’t Santa Answer His Survey?

These are times of rapid change in communications technology. Most of us get our news from electronic devices, communicate online and turn to the Internet to provide and obtain information. We are too mobile, too busy, and constantly connected to some device or another that ties us to the rest of our network and the world. In the olden days, as they were, people read newsprint, sent thank you cards by post, and took surveys in person. Today, in-person surveys can be cumbersome, and are too expensive.

Even if the Internet seems ubiquitous, connectedness is still growing. According to the US Census Bureau, 67% of US Households had a computer in 2005, but 83.8% reported they did in the 2013 Census. Of the 2013 US Population, 74.4% of households reported Internet use. This leaves more than a quarter of all American households with no Internet use at home.

Still, the growing presence of the Internet and electronic devices across the country is providing new opportunities for surveying. At the same, the survey world is very concerned about increasing survey refusal rates, and struggling with how to gather a representative sample via traditional methods, such as random digit dialing, when landline distribution is no longer representative. In this changing, mobile, interconnected world, how can we improve the way in which we reach and understand the population?

When we decided to build the Understanding America Study (UAS) in 2013, all research pointed to Address Based Sampling as the way to go. We used a commercial vendor and received a random set of addresses pulled from zipcodes across the country. Remarkably, our sample is the most representative unweighted sample we have started with in our history of panel building. We are short of perfection in a few key brackets however; younger people (18-34) and those with less than a high school education. Anecdotally, this makes sense. Younger people move around a lot, may not have their names on utility bills, public records, etc. Less educated people may no have access to the Internet or use a computer at work.

In the UAS we provide tablets to respondents who return our paper survey stating they are willing to participate in more surveys but don’t have Internet. Even with this commitment to inclusiveness, we need to send out a consent form and a return envelope, we need them to have some sort of telephone number so we can get the tablet to them via FedEx or another traceable mailing service, and we often need to spend hours with our new respondents helping them get acclimated to using a tablet. People reading online blogs likely will not be aware of how very unintuitive swiping and selecting can be to someone who has not used a computer or a smartphone.

The UAS team is very protective of our sample. Our names and addresses are stored separately from the survey data, we use prepaid cards to pay our respondents, which do not require Social Security numbers, and all of our data is encrypted. We want to hear from everyone living in America when we ask them if they are worried about Ebola, if they are planning for retirement, if they plan on voting, and if they are looking forward to traveling to Cuba. We’re convinced inclusiveness is worth it. If social scientists continue this fight to collect data from everyone willing to give us their opinions, to protect their data like our own, we just may make the world better for everyone.