Ensuring a Robust and Accurate Data Quality Analysis in the 2020 Census

Asking outside experts to review our work is standard operating procedure at the U.S. Census Bureau. It underscores our commitment to quality and transparency. 

In a blog earlier this month, Acting Census Bureau Director Ron Jarmin touched on the steps we are taking to assess the quality of the 2020 Census. These steps include the establishment last April of the Data Quality Executive Guidance Group (chaired by Dr. Jarmin) to ensure resources are adequately and appropriately focused on addressing and documenting data quality issues. Dr. Jarmin also mentioned that we are working with outside experts who are examining our quality metrics and reviewing our processes, procedures and decisions related to the 2020 Census.

In this blog, I take a closer look at the groups of experts we’ve engaged and at the work they are doing with us.

To date, we have asked two highly respected groups to provide an external assessment of the decennial census and are working to engage a third. These organizations are:

• JASON – JASON is an independent group of scientists and engineers with expertise in a variety of technical areas leveraged to perform studies for government sponsors. JASON has already completed multiple studies for the Census Bureau, covering various aspects of the 2020 Census and other Census Bureau operations.
• The American Statistical Association (ASA) Quality Indicators Task Force – The ASA has long shown an interest in the decennial census, and it includes experts who know our work well. Before, during and following every decennial census, we work closely with the ASA on the research and development of the operational design of the census, and the production and analysis of the data.
• National Academy of Sciences (NAS) Committee on National Statistics – Similar to the ASA, the NAS has worked with the Census Bureau each decade, and it too includes many experts familiar with our work. Over the decades, CNSTAT has established panels to assess each decennial census and to suggest parameters for the research and planning of the subsequent censuses. At the moment, we are still planning on how the NAS/CNSTAT will engage with the 2020 Census. We hope to have an update on this effort in the weeks ahead.

These three groups will tackle different aspects of assessing the Census Bureau’s work. Their reports will advise the Census Bureau on improving future censuses and will help the public understand the quality of the 2020 Census data.  

The JASON study will be the first in this series. Importantly, the JASON study did not examine 2020 Census data. Rather, its goal was to identify strengths and weaknesses in our plans for data quality assessments and metrics, and to inform our efforts to communicate important aspects of data quality that could accompany release of 2020 Census data products.

JASON was given a broad scope to review and assess Census Bureau plans, processes, procedures and metrics around 2020 Census data quality. It conducted its work Jan. 4-Feb. 8, 2021. This engagement provided us with a timely reality check on our plans for assessing 2020 Census data quality. JASON has provided us with recommendations and insights supporting 2020 Census efforts to provide quality data and informative metrics.

The JASON report, set to be released today, calls for implementation of improved communications strategies that explain the 2020 Census in a way that builds confidence that the Census Bureau will achieve an accurate count. It also recommends that we ensure that our post-data collection processing schedule allows adequate time for complete, accurate and transparent processing of the data. This blog series and associated briefings, press releases, press availability, and announcements of our dates for producing the apportionment and redistricting data all reflect our agreement with these recommendations.

JASON also expressed support for the Census Bureau’s proposed metrics as useful to evaluate data quality and provided recommendations for additional analysis such as assessments of data quality across various geographies and for relevant demographic groups. Upcoming blogs and briefings, in addition to our work throughout the year, will articulate our response to these recommendations as well. 

Finally, JASON stresses the importance of leveraging our experience with the 2020 Census to inform planning for and the operational design of the 2030 Census. This work is already underway.

Experts from the ASA are already working closely with us to build on the JASON report. Unlike JASON, the ASA is diving into the internal operational and response data from the 2020 Census to help us understand the accuracy and coverage of our enumeration of the nation’s residents. This independent work will unfold over the next year and will move in real-time along with Census Bureau assessments of data quality. The ASA has posted updates about this work on its website.

As mentioned earlier, we’re still planning the work we will be doing with the NAS/CNSTAT and expect to have more details to share in the weeks ahead.

Throughout our effort to establish and implement quality metrics on the 2020 Census, we will be open and transparent about what we learn. We are charging these external experts to conduct their work independently, and we look forward to their findings. Stay tuned to this blog series and our associated communications efforts to learn more about this important work.