Since 1950, the U.S. Census Bureau has incorporated a testing, evaluation, and experimental program into the overall enumeration effort. The 2020 Census Evaluations and Experiments (EAE) operation is one component of our 2020 Census Data Quality efforts, designed to document and evaluate 2020 Census programs and operations, as well as to test new methods suggested from previous research. Results from the 2020 Census EAE operation serve as the background or basis for designing, testing, and implementing the 2030 Census.
The 2020 Census Evaluations and Experiments (EAE) consists of more than 65 studies. The studies consist primarily of operational assessments, but also include evaluations, experiments, quality control results, and a topic report. The list of planned studies by type, descriptions of each type, and tentatively scheduled completion dates are viewable below. The results from some of these studies will not be released publicly because of the proprietary or restricted nature of the information. The first reports from these studies were released on August 18, 2022.
These studies aim to describe the effectiveness of census components and the impact that they have on topics such as data quality and coverage. These reports present analysis and interpretation of quantitative and qualitative data pertaining to decennial census operations, processes, systems, and auxiliary data collections.
This evaluation was conducted prior to the 2020 Census to test the effectiveness of 2020 Census television and radio advertisements using quantitative procedures. It analyzed factors such as message recall, message comprehension, behavioral intention to respond to the census, and the Census Bureau’s corporate image. The results will help the Census Bureau understand how quantitative creative testing might be utilized in intercensal years where ad copy is used, as well as in preparation for the 2030 Census.
The 2020 Census was accompanied by a diverse array of paid advertisements, partnership outreach programs, news coverage, and more designed to increase knowledge about the census and motivate households to self-respond, specifically through the internet mode. This evaluation compares self-response rates from the 2020 Census and the 2019 Census Test, which was void of advertisements, to estimate the impact of these programs—collectively referred to as the decennial environment.
By Sept 2023
Will analyze the 2020 Census website paradata to investigate the relationship between digital advertising materials and online self-response.
By Dec 2023
Will help us determine, for privately owned student housing, which addresses are group quarters and which should be considered housing units so that we include them in the appropriate data collection operations.
By Dec 2023
Will study U.S. public sentiment concerning matters that may bear upon 2020 Census participation.
By Dec 2023
Will evaluate the relationship between intended response behaviors and actual response behaviors, as well as the characteristics of nonresponding households.
By March 2024
Will assess respondents’ privacy and confidentiality concerns about responding to the census generally, as well as assessing concerns of certain types of respondents with a follow-up questionnaire.
By March 2024
Will use a panel of survey respondents to measure intent to participate in the census over time in order to evaluate the 2020 Census Communications Campaign.
By March 2024
Will research if dual system estimates could be generated without conducting an independent post-enumeration survey. The census would continue to serve as the first source, but administrative records would serve as the second source, rather than the post-enumeration survey results.
By March 2024
Will evaluate the redesign of the Address Canvassing operation, including the use of in-office address canvassing and interactive review.
By June 2024
Will conduct both an evaluation and an experimental research project about historically hard-to-count populations such as non-English speakers and complex household residents.
By June 2024
Will study paradata from the internet self-response instrument to learn about issues associated with language.
While the Census Bureau plans to conduct this evaluation, it does not plan to externally release the evaluation report due to the proprietary nature of the information included. (As of May 2022)
The goal of this project was to evaluate the specific question wording, instructions, layout, and other features of the 2020 Census stimuli related to the household roster. The target population was households at risk of under-reporting young children, and the research design involved interviewing multiple adults within a household. Both these factors, and experience during the pilot testing stage, indicated in-person interviewing was needed. With the onset of the pandemic, field data collection was suspended soon after the main stage project had begun. The project could not be completed within the necessary timeframe for a 2020 Census evaluation. However, these research objectives will be carried forward and incorporated into plans for 2030 Census research related to historically undercounted populations, including the coverage of young children.
These studies identify potential topics for early 2030 Census life cycle research and testing. Experiments are quantitative or qualitative studies that must occur during a decennial census to produce results that are meaningful for the planners of the next one. In general, decennial census experiments involve comparisons (usually of response rates) between a control group that reflects 2020 Census production methods or procedures and a treatment group(s) that tests modifications to them.
By June 2023
Measured our ability to increase self-response by tapping into the unique environment surrounding the decennial census through the materials we use to contact households such as sticker inserts and direct mail promotional postcards.
By June 2023
Studies the net impacts of the 2020 Census mailing strategy and the overall influence of the online self-response option.
By Sept 2023
Developed an administrative record census in real time in 2020 to determine how the population statistics compare between an administrative record census and survey-style collection within the same time frame.
These studies document final volumes, rates, and costs for individual operations or processes using data from production files and activities and information collected from debriefings and lessons learned. They do not include evaluative analyses. Operational assessments present planned versus actual variances as they relate to budget, schedule, and workloads (production and training). Depending on the operation, the assessment may include frequency distributions and standard demographic or address tables.
The In-Field Address Canvassing operation is a dependent listing activity conducted in the field and limited to the most challenging areas of the nation to identify where people live, stay, or could live or stay. Field staff compare what they see on the ground to the existing Census Bureau address list and either verify or correct the address and location information, adding addresses to the list as necessary. Field staff classify each living quarter as a housing unit or a group quarters.
The In-Office Address Canvassing operation uses empirical geographic evidence (e.g., imagery, comparison of the Census Bureau’s address list to partner-provided lists) to assess the current address list. This operation detects and identifies change from high-quality administrative and third-party data sources to reduce and then determine the In-Field Address Canvassing workload.
The Archiving operation coordinates the storage of materials and data. This operation is responsible for providing the official permanent records data from the 2020 Census (including files containing the individual responses to the 2020 Census) to the National Archives and Records Administration. It also provides similar files to the National Processing Center to use as source materials for the Age Search Service. Finally, the Archiving operation coordinates the storage of data to cover in-house needs.
The 2020 Local Update of Census Addresses (LUCA) operation allowed tribal, state, and local entities the opportunity to review and update the Census Bureau’s residential address list for their jurisdiction. The Census Bureau relies on a complete and accurate address list to reach every living quarter and associated population for inclusion in the decennial census.
The 2020 Post-Enumeration Survey (PES) Independent Listing operation was the first field operation PES conducted. During this operation, field staff independently visited and listed all housing units and potential housing units in each PES sample basic collection unit without using any previously collected address information.
The Census Questionnaire Assistance (CQA) program provided telephone support by answering frequently asked questions and helping callers fill out the census questionnaire. In addition to the inbound call capabilities, CQA conducted the outbound Coverage Improvement (CI) telephone operation, which aimed to resolve potential coverage-related issues that occurred during the initial census data collection.
This assessment presents the item nonresponse and imputation rates for household-level and person-level items from the 2020 Census, and it also compares the 2020 item nonresponse and imputation rates to the 2010 rates. These rates are quality metrics for response data. Item nonresponse occurs when a respondent provides some information but does not respond to all census questions. Imputation is a statistical technique used to fill in missing information and considers both respondent cooperation and response validity.
By June 2023
By June 2023
By June 2023
By June 2023
By June 2023
By June 2023
By June 2023
By June 2023
By June 2023
By June 2023
Please note that this report will consist of the scope of both the formerly planned Content and Forms Design: Paper Questionnaires and Non-questionnaire Material Design Assessment Report and the Content and Forms Design: Content Specifications for Automated Data Collection Instruments Assessment Report.
By Sept 2023
By Sept 2023
By Sept 2023
By Dec 2023
By Dec 2023
By Dec 2023
By Dec 2023
By Dec 2023
By Dec 2023
By Dec 2023
By Dec 2023
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By March 2024
By June 2024
By Dec 2024
By Dec 2024
By June 2025
By June 2025
These studies provide results from the quality control activities associated with a specific operation. These reports present a more in-depth discussion on the topic of quality control than what appears in the corresponding operational assessment report.
The 2020 Census In-Field Address Canvassing (IFAC) Quality Results Report provides the results of the 2020 Census IFAC Quality Control (QC) program, as well as recommendations for future census listing operations. The In-Field Address Canvassing operation is a dependent listing activity conducted in the field and limited to the most challenging areas of the nation to identify where people live, stay, or could live or stay.
The 2020 Update Leave Quality Results Report provides the results of the 2020 Update Leave (UL) Quality Control program, as well as recommendations for future census listing operations. The 2020 Update Leave operation updated the address and coordinate location for an assigned area. A 2020 Census questionnaire package was then left at every housing unit identified in that area.
By Dec 2023
TBD
Will provide a description of operational changes implemented as a direct result of the COVID-19 pandemic with a qualitative assessment of the potential impact each change had on data quality.