Counting regulations and measuring regulatory impact: a call for nuance

The US regulatory state has significantly expanded since the Administrative Procedure Act's passage. While yielding benefits like cleaner air and safer workplaces, it has also imposed burdens and costs on industries, potentially leading to higher consumer prices and job losses. The resulting impulse to measure the total impact of regulation has led to various attempts to count regulations and quantify their costs and benefits, often informing discussions about regulatory reform. However, accurately counting regulations and measuring their cumulative impact is incredibly difficult. This article reviews existing methods, discusses their shortcomings, and suggests improved approaches, focusing primarily on US efforts, given the greater emphasis on the question of "Are there too many regulations?" in the US compared to Europe.

The author reviews existing literature arguing for and against counting regulations and measuring their impact. Studies often originate from institutions or individuals favoring reduced government regulation, influencing the debate. Arguments for counting emphasize regulation's stifling of innovation and economic productivity, while counterarguments highlight oversimplification and potential undermining of statutory goals. The author counters that well-executed counting, devoid of bias, can be beneficial.

The article examines various methods for counting regulations and measuring their impact. Methods for counting regulations include: 1. **Counting Federal Register pages and documents:** This is a simplistic approach with significant limitations, including the inclusion of non-regulatory documents and the unequal weighting of different rule types. 2. **Counting CFR pages:** Counting pages in the Code of Federal Regulations is an improvement but doesn't account for the varying burden imposed by different requirements. 3. **Counting regulatory restrictions (RegData):** This method utilizes text analysis to identify "binding constraints" within the CFR, offering a more refined approach but still subject to limitations like misinterpreting negations or clarifying questions. 4. **Counting documents reviewed by OMB:** Counting regulations reviewed by the Office of Management and Budget (OMB) focuses on "significant" regulations with an impact exceeding $100 million. This approach improves the quality of the count but suffers from underinclusivity, omitting regulations from independent agencies and those below the significance threshold. 5. **Counting Unified Agenda entries:** This method analyzes the Unified Agenda's "completed actions," providing a more comprehensive count than OMB's review, including independent agencies. However, data quality and categorization issues have existed in the past. Methods for measuring regulatory impact include: 1. **Top-down approaches:** These examine the economy as a whole using regression analysis, with the Crain and Crain (2014) study cited as an example of a widely criticized top-down estimate. 2. **Bottom-up approaches:** This involves aggregating the costs and benefits calculated by agencies for "economically significant" regulations as mandated by Executive Order 12866. This data is compiled in annual OMB reports to Congress; however, these estimates suffer from underinclusivity (omissions of independent agencies, and regulations below the threshold), overinclusivity (inclusion of later overturned regulations), inaccuracies in agency cost-benefit analyses, and a lack of consideration for long-term impacts. 3. **Paperwork burden:** Measuring the burden of information collection requirements using data from the Paperwork Reduction Act's Information Collection Budget (ICB) offers another way to assess regulatory impact, although it includes non-regulatory requirements and also suffers from underestimation and lack of benefit consideration.

The article's key findings highlight the inherent flaws in existing methods for counting regulations and measuring regulatory impact. Simply counting pages in the Federal Register or CFR, or the number of documents, provides an inaccurate and incomplete picture. Even more sophisticated methods like RegData, while improvements, still struggle to accurately capture the nuanced reality of regulatory burden. Furthermore, top-down economic analyses frequently suffer from significant methodological flaws, while bottom-up approaches, relying on agency cost-benefit analyses, are incomplete due to exclusions and inherent inaccuracies in the estimates themselves. The analysis of the Paperwork Reduction Act data provides a different perspective, focusing on administrative burden but again highlighting the significant limitations of available data and the lack of information on regulatory benefits.

The author argues that the flaws in current methods are not a reason to abandon efforts to measure regulatory impact, but rather a call for better methods. While simple counts can be helpful in understanding trends over time or across administrations, they fail to address the core questions of how regulations affect individuals and businesses. The author argues for a multi-faceted approach combining both quantitative and qualitative methods. The lack of attention to the distributional effects of regulation necessitates a shift to analyses that address how burdens and benefits are distributed among various groups in society.

The article concludes by emphasizing the need for a more nuanced and comprehensive approach to measuring the impact of regulations. While tracking significant regulations, Unified Agenda entries, and reported costs and benefits remains valuable, it should be supplemented by leveraging the data from the Paperwork Reduction Act using advanced techniques like large language models and by conducting in-depth qualitative research. Future research should focus on understanding the cumulative impacts of regulations on businesses and individuals, paying equal attention to both costs and benefits and exploring how burdens and benefits are distributed amongst various groups, including disadvantaged populations.

The article's main limitation is its focus on the US regulatory system. While some comparisons are made to the European Union, the findings and recommendations may not be directly applicable to other countries with different regulatory structures. The author also acknowledges the limitations of existing data sources, particularly the underinclusivity and potential inaccuracies in agency cost-benefit analyses and paperwork burden estimates. Furthermore, the qualitative research suggestions, while valuable, require substantial time and resources.