Household surveys — most notably the U.S. Census as well as the American Community Survey and the Current Population Survey — are important sources of information in the United States. They provide policymakers with data to help them make decisions essential to nearly every government program, including those involving housing, law enforcement and public education. At the most basic level, the U.S. Constitution requires that a count be taken of every American resident once per decade — data that is used to draw federal, state and local legislative districts as well as school district boundaries. Government agencies as well as private organizations and research groups conduct household surveys, some of which have been criticized because they ask for detailed personal information such as how many toilets a home has and whether someone in the family suffers from mental illness. The Republican National Committee has demanded that the U.S. Census Bureau stop being so “nosy.”
Regardless of how U.S. residents feel about these surveys, large and nationally-representative surveys are a relatively fast and inexpensive way to collect this sort of data. When surveys are changed or administered in a different way, researchers and policymakers oftentimes react with concern or frustration. For example, statisticians and demographers have spoken out against attempts to make optional the American Community Survey — a mandatory survey sent to about 3.5 million randomly-selected addresses each year that covers a wide range of topics, including the race and gender of household members as well as their income, educational attainment and disability status. In 2014, the U.S. Census Bureau caused a “mild firestorm” after proposing changes in the way it measures health insurance coverage.
As important as some surveys are, it is more important that the responses given are accurate and thorough. In recent years, decreasing response rates and data errors — for instance, some respondents give inaccurate information about their personal finances — have challenged the usefulness of some surveys and resulted in lower quality data. A 2015 study published in the Journal of Economic Perspectives, “Household Surveys in Crisis,” examines these problems. Bruce D. Meyer of the University of Chicago, Wallace K. C. Mok of the Chinese University of Hong Kong and James X. Sullivan of the University of Notre Dame worked together to better understand these problems and why they occur.
Their study’s findings include:
- U.S. households are increasingly less likely to answer surveys. The nonresponse rate of the National Health Interview Survey, for example, rose from 8 percent in 1997 to 24 percent in 2013.
- More people are leaving some survey questions unanswered, especially questions about income. For most of the years since 2000, more than 20 percent of the people who have taken the federal government’s Current Population Survey or the Survey of Income and Program Participation did not answer whether they receive financial assistance through assistance programs known as Aid to Families with Dependent Children (AFDC) and Temporary Assistance for Needy Families (TANF).
- Survey respondents are underreporting the amount of financial assistance they receive through government programs. “In recent years, more than half of welfare dollars and nearly half of food stamp dollars have been missed in several major surveys,” the authors state.
- Some of the biggest reasons why people are less likely to answer surveys are disinterest, a lack of time and privacy concerns. Poor health and language problems also have prevented individuals from participating.
- The rise in gated communities and decline in land-line phones have made door-to-door and phone surveys more difficult.
The authors suggest that one way to reduce errors in survey data is by linking this information to existing administrative data sets — for instance, data on federal program expenditures. This would allow for an external validation of survey responses. New methods of surveying residents via the Internet as well as through traditional mail, telephone and in-person interviews may reduce costs and have the potential to improve data quality.
Related research: A 2015 study in the Journal of Official Statistics, “Models for Combining Aggregate-Level Administrative Data in the Absence of a Traditional Census,” looks at the limitations of using administrative data sets to estimate population counts by age, gender and geographical area. A 2014 study in the International Journal of Humanities and Arts Computing, “Automatic Record Linkage of Individuals and Households in Historical Census Data,” examines a new method for linking individuals and households across historical census datasets.
Keywords: surveys, census, polls, inequality, welfare, population, household, statistics
Expert Commentary