Random Digit Dialing (RDD) is a popular research method in which participants are contacted by the random dialing of telephone numbers. As more U.S. residents move to cell phones, however, they are increasingly out of reach of such traditional phone-based surveys. The Internet can also be used to administer surveys, and it has the advantage of allowing researchers to target specific populations (known as a “probability” survey) or random groups (a “non-probability” survey).

A 2011 study from Stanford University, University of Pennsylvania, University Chicago and LinChiat Chang Consulting published in Public Opinion Quarterly, “Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Sample,” assessed the accuracy of RDD and Internet research methods by analyzing the outcome of surveys in 2004 and 2005 as well as follow-ups in 2008 and 2009. The results were then compared with a series of benchmark studies to assess the accuracy of each approach.

The study’s findings include:

• Targeted surveys were more accurate than random surveys on both the telephone and Internet. Telephone surveys were the most accurate method, followed closely by targeted Internet surveys.
• Survey adjustments improved the accuracy of telephone and targeted Internet surveys. Of the seven random Internet studies, adjustments greatly or marginally increased the accuracy of three, decreased the accuracy of one, and didn’t change the accuracy of three.
• Among the random Internet surveys, one was “unusually inaccurate” while the remaining six were “roughly equivalently inaccurate.” The researchers suggested that little can be done to improve the accuracy of this type of survey, but maintained that they could be useful in specific, limited circumstances.
• For all types of surveys, a higher number of responses produced not more accurate results, but less accurate ones: “Completion rates and response rates of the surveys were negatively correlated with their accuracy.”
• The authors noted that pre-election Internet polling may be a specific genre of random survey that can compete with the accuracy of a targeted survey. They surmised that this may be due to different polling procedures.

The researchers noted several potential limitations to the study. In particular, the benchmarks used for comparisons were biased toward targeted survey methods and were the product of only seven market research firms.

Tags: technology