Evidence from a randomized control trial indicates that when you introduce a question about national citizenship on a United States Census form, respondents — especially Hispanics — answer a smaller percentage of the questions.

In the study, about 9,000 residents of the U.S. were asked to fill out a mock census survey. Half were randomly assigned a survey that included a question about citizenship; the other half received a survey without the citizenship question. In total, the respondents whose survey included the citizenship question skipped 26% of the questions on the form. Those filling out a survey with no citizenship question skipped 23% of the questions, according to a working paper from researchers at the Harvard Kennedy School’s Shorenstein Center.

Among self-identified Hispanics, the number of unanswered questions rose by 4.21 percentage points when the citizenship question appeared. For Hispanics who reported being born in either Mexico or Central America, receiving the citizenship question led to an 11.04 percentage-point increase in the number of questions skipped.

The researchers say their findings suggest that adding a citizenship question to the 2020 Census likely would result in significant undercounts of certain populations, especially Hispanics, with negative downstream “ripple effects,” such as serious financial and political consequences, likely into 2030 (when the next census after the 2020 count is scheduled to take place).

 

“Extrapolating our results to the general population, we estimate that asking about citizenship would reduce the share of Hispanics recorded by the Census by approximately 6.07 million, or around 12.03 percent of the 2010 Hispanic population — a sizable reduction in the share of the U.S. population that would be recorded as Hispanic,” they write in their paper. 

After last appearing on the 1950 census, the proposed reintroduction of the citizenship question –‘Is this person a citizen of the United States?’ — to the 2020 census reignites concerns of “disproportionate effects on Hispanic populations,” according to the paper.

In March 2018, U.S. Secretary of Commerce Wilbur Ross, who is authorized by Congress to determine which questions are on the decennial census, “approved a late request from the Justice Department for a citizenship question to be added to the 2020 census,” according to Hansi Lo Wang of National Public Radio. Wang summarizes the argument: “The Justice Department claims that it needs a better count of voting-age citizens from the census in order to enforce protections against voting discrimination under the Voting Rights Act. But critics of the citizenship question say that they’re worried that adding the question will discourage noncitizens, especially unauthorized immigrants, from participating in the national headcount.”

The Supreme Court is scheduled to hold a hearing on the citizenship question on Tuesday, April 23. (This timeline sheds more light on the backstory of how the question ended up in court.)

The working paper, “Estimating the Effect of Asking About Citizenship on the U.S. Census,” presents research conducted by Matthew Baum, Bryce J. Dietrich, Rebecca Goldstein and Maya Sen that indirectly grew out of Baum and Sen’s work at Harvard’s Kennedy School investigating voter fraud.

Participation in the U.S. Census is important because, among other things, the census decides apportionment — how the 435 seats in the U.S. House of Representatives are split among the 50 states — and the allocation of an estimated $880 billion in federal funding each year (according to FY2016 data). Taken every 10 years, the census determines how political districts are redrawn and affects legislation on issues such as education, health insurance, minimum wage and welfare reform. It provides critical data used by governmental organizations from the Centers for Disease Control and Prevention to the FBI to the U.S. Department of Agriculture.

“For good reason, the citizenship question has sparked a lot of debate,” says Dietrich, a post-doctoral fellow at the Shorenstein Center who is also an assistant professor at the University of Iowa. “It’s definitely an important public policy question, and the Shorenstein Center was excited to support the project.”

Significantly, the paper, the first survey experiment that mimics the census form, posits a causal relationship between asking about citizenship and decreases in census completion — something that has not been done in previously published investigations and provides more conclusive evidence than, “observational data and cross-survey comparisons.”

Discussing the survey’s design, Dietrich identified two types of concerns with the citizenship question. The first, what’s referred to as “unit non-response,” occurs when individuals simply do not return the census form. Transient populations, and those without permanent addresses, contribute in large part to unit non-response, which has been studied extensively by the Census Bureau due to the heavy cost (approximately $1.6 billion in 2010) and continual nature of the problem, known by the Census Bureau as “nonresponse followup.”

The second type of concern, called “item non-response,” refers to not completing certain questions or sections on the census form and results from a variety of complex factors. Though the Census Bureau is concerned about both types of non-response — and hopes to efficiently increase overall response rate by transitioning to the first-ever digital census in 2020 — Dietrich and his colleagues specifically wanted to investigate item non-response, which could ultimately produce underreporting of Hispanics, and therefore compromise census accuracy and quality above and beyond some households simply not returning the Census form.

The researchers worked with Qualtrics, a third-party customer survey software company, to distribute the mock census-survey in two waves. The first wave targeted 4,104 (45% of the sample) people who self-identified as non-Hispanic and the second wave targeted 4,931 (55% of the sample) people who self-identified as Hispanic. The researchers intentionally oversampled Hispanics in the population in order to increase precision. Though Dietrich noted the relevance of the study for other demographic groups, the primary goal was to isolate the effect of the citizenship question on responses from the Hispanic population, which many believe will be disproportionately affected.

The study finds that introducing the citizenship question increased the number of questions skipped by 3.07 percentage points. The researchers found that this general effect was especially pronounced among Hispanics.

Dietrich said that “age, race and ethnicity give context to skipped questions.” He and his colleagues found evidence that as people encountered demographic questions on the form about such things as date of birth, race and ethnicity, they answered fewer questions. One important consequence of that, according to the paper, is the underreporting of Hispanic household members. Among survey takers who were not Hispanic, there was a 1.38 percentage-point difference in the percentage of the household reported as being Hispanic between those who did and did not receive the citizenship question.

For Hispanics, this effect increases to a 5.95 percentage-point difference with, again, stronger effects being found for Hispanics who reported being from Mexico and Central America. When the citizenship question was presented to Hispanics from Mexico or Central America, they indicated 8.32 percentage points fewer household members were Hispanic as compared to the control.

The study also shows that simply filling in the missing values using other data — something commonly done by the Census Bureau — is not enough to address the problem; the citizenship question, Dietrich notes, is a “special class of question” that cannot be accounted for like other data that the census collects.

The researchers stress that their findings are likely conservative. Because the adults who participated in this study were paid a modest hourly wage to complete the survey, they might have completed more questions than they would have if they were actually completing the real census online.

Dietrich estimates that the completion rates could be significantly lower without that financial appeal — and especially compared to receiving an envelope in the mail with “Census Bureau” written large across the front. Further, the people who might be most difficult to reach by mail would also likely not be persuaded by the Census Bureau’s advertisements or efforts to create a sense of social responsibility to complete the census. And, though it is technically illegal for people living in the U.S. or a U.S. territory not to complete the census, violations have rarely been prosecuted.

The most challenging part about conducting this study, Dietrich says, was to determine whether people dropped out because of the citizenship question (and/or related demographic questions), or because of fatigue or random chance. “That’s why we needed 9,000-plus respondents. It’s a much more challenging variable to work with,” he says, explaining that it’s almost as if trying to measure the loudness of silence.

The paper is currently under peer review and, based on comments the authors have received, they hope it will be published soon.

While Dietrich says he hopes the working paper sheds new light on the potential impact of the citizenship question, he wonders whether the heavily polarized political debates about the census have already sparked alarm in potential census respondents. Even further, given that the 2020 Census will be the first-ever digital census, Dietrich finds it “curious” that the method of data collection and content — two substantial changes — are planned to occur simultaneously.

“After years of rigorous research that the Census Bureau has spent in perfecting the process, to disrupt the method of collection as well as introduce a question that has been intentionally left off the census for decades … it’s going to be very chaotic and challenging for the Census to adjust,” he says.