U.S. flag

An official website of the United States government

Skip Header

Communicating Data Use and Privacy: In-Person Versus Web-Based Methods for Message Testing

Written by:

Communicating messages about privacy, data use and access, and confidentiality is critical to earning and keeping the trust of respondents and to ensuring their willingness to participate in surveys. Informing respondents about their rights and how their data will be used is often required by law. However, there is currently not much data available on how respondents react to these messages or how they understand the meanings we try to convey.

Our research addresses this gap in our knowledge about respondents’ understanding of an intended message. We focused our research on sets of messages that convey different types of information. The sets of messages informed respondents of a range of factors including:

  • Who has access to survey responses.
  • Survey responses are confidential.
  • Data are for statistical use only.
  • Individuals will not be identifiable when statistical data are released.
  • Responses will not be shared with law enforcement or used for eligibility for government benefits.
  • Data are sometimes shared with other federal agencies.

Four additional sets of messages tested statements that the Census Bureau is legally required to provide, such as the mandatory nature of the census, confidentiality protections, burden notifications and other language from the Paperwork Reduction Act.

We used both web-based and in-person methods to test respondent comprehension of messages about privacy and confidentiality. First, we used an open-ended internet instrument to collect qualitative data on respondent comprehension of these messages. Web-based testing was remote and respondents did not interact with an interviewer. We analyzed these data to identify high and low performing messages. We then tested some of those messages in a smaller-scale, in-person cognitive test with 30 respondents.

Combining these two methods allowed for a larger scale data collection than is typical in a qualitative study while retaining the ability to elicit rich description and allow for spontaneous probing. This research not only helps us understand how respondents comprehend our messages, but also facilitates exploration of web-based methods for testing survey questions.

One of the central limitations of in-person interviewing is the difficulty of obtaining respondent diversity. In-person interviewing is also costly in both labor hours and respondent compensation. However, this type of interviewing allows for considerable interviewer flexibility. Through direct comparison with more traditional methods, this research highlights the advantages and limitations of using alternatives to traditional in-person cognitive interviewing. For an update with our findings, come see our presentation at the American Association for Public Opinion Research on Saturday, May 14, 2016, in Austin, Texas.

Is this page helpful?
Thumbs Up Image Yes Thumbs Down Image No
255 characters maximum 255 characters maximum reached
Thank you for your feedback.
Comments or suggestions?


Back to Header