Counting regulations and measuring regulatory impact: a call for nuance

The paper examines whether and how counts of regulations can meaningfully capture the cumulative impact of the US regulatory state. It situates the inquiry in the context of an 80-year expansion of federal regulation that has generated major social benefits but also costs and burdens for firms and individuals. Because regulation affects productivity, prices, innovation, and access to government benefits, stakeholders have sought simple metrics (counts of rules, pages, costs) to summarize its overall impact and to inform policy debates. The author argues that while counting efforts are politically salient and can reveal trends (e.g., midnight regulations, presidential priorities), they often oversimplify a complex phenomenon and can mislead when used to argue for or against regulation writ large. The purpose of the study is to catalog leading approaches to counting regulations and measuring regulatory impact in the United States, assess their strengths and weaknesses, and propose more nuanced methods—including leveraging Paperwork Reduction Act data, large language models, and qualitative research—to better understand cumulative burdens and benefits for businesses and individuals.

The paper reviews debates over counting regulations and measuring impact, noting that many counting efforts originate from actors favoring deregulation (e.g., Crain and Crain; Crews’ Ten Thousand Commandments; Mercatus Center), often asserting that regulatory accumulation suppresses innovation and productivity. Critics (e.g., Short) argue that counting oversimplifies, risks undermining statutory goals, and diverts attention from meaningful policy evaluation. The review covers measurement traditions: top-down macroeconomic studies (e.g., Crain and Crain) that have been heavily criticized for methodological flaws and conflation of non-regulatory factors; bottom-up cost–benefit accounting under E.O. 12866/OMB reporting, which is more grounded but underinclusive and often based on ex ante estimates; page/document counting in the Federal Register and CFR, which suffers from over/underinclusivity and lack of comparability; text-based restriction counts (RegData) that improve specificity but still misclassify constraints and ignore context/benefits; OMB significant/economically significant rule counts and Unified Agenda entries, which better focus on impactful actions but omit independent agencies or rely on burdensome data collection. The paper also references OECD’s Standard Cost Model and European applications of AI/LLMs to legal texts, noting mixed validity and limited ability to capture benefits.

This is a conceptual and integrative review. The author systematically catalogs major US approaches to measuring regulatory scope and impact: (1) counts of Federal Register pages/documents; (2) counts of Code of Federal Regulations pages/sections; (3) text-based counts of binding constraints (e.g., RegData’s flags for shall, must, may not, prohibited, required, with NAICS mapping); (4) counts of OIRA-reviewed significant and economically significant final rules under E.O. 12866; (5) counts of Unified Agenda completed actions; (6) top-down macroeconomic regressions; (7) bottom-up aggregation of costs and benefits from agency Regulatory Impact Analyses as compiled by OMB reports; and (8) aggregation of paperwork/information collection burdens from the Paperwork Reduction Act’s Information Collection Budget (ICB). Where applicable, the paper notes data sources (e.g., reginfo.gov, OMB annual reports/spreadsheets) and calculation choices (e.g., summing agency-reported costs/benefits and converting to 2022 dollars using CPI). The analysis evaluates each method’s coverage, validity, and interpretability, and proposes augmenting quantitative counts with PRA mining using LLMs and qualitative studies focused on firms and beneficiaries.

• Simple counts are misleading: Federal Register page/document counts and CFR page counts overstate or obscure regulatory reach, include non-regulatory material, and cannot distinguish routine from impactful rules or the direction of impact. CFR pages also rarely decrease, complicating interpretation.
• Text-based restriction counts (e.g., RegData) improve specificity but still misclassify requirements (negations, questions, multiple hits per single requirement), ignore exceptions/options, and cannot capture benefits.
• OMB/OIRA counts of significant and economically significant rules (typically ~100–400 significant and ~15–125 economically significant final rules per year) better target impactful rules but undercount independent agencies and non-significant rules that still matter.
• Unified Agenda “Completed Actions” provide a broader, more balanced count across agencies and significance levels and allow rich parsing (e.g., small business effects), but classifications have been inconsistent (e.g., EO 13771 designations), timing lags exist, and data collection is labor-intensive.
• Top-down macro estimates (e.g., Crain and Crain’s $2.028T) are methodologically flawed and conflate non-regulatory factors; they should not be relied upon.
• Bottom-up OMB cost–benefit aggregation is the strongest quantitative basis available but is underinclusive (excludes many independent agency actions, older rules, non-quantified effects, and below-threshold rules) and based largely on ex ante estimates often found to overestimate both costs and benefits. For 2007–2022 (in 2022 dollars), aggregated estimates yield benefits of roughly $415.7–$1,277.7 billion and costs of roughly $102.9–$163.5 billion for the subset of rules with quantified benefits and costs.
• Paperwork (information collection) burden is a large, routinely compiled metric under the PRA/ICB and amenable to disaggregation, but it includes non-regulatory burdens (e.g., tax returns, surveys), omits many non-paperwork regulatory costs, and rarely quantifies benefits/practical utility; accuracy is often unvalidated and may be biased downward.
• Illustrative scale indicators: in 2022 the Federal Register had 80,756 pages (21,750 devoted to rules), including 3,168 rules (28.5% fewer than 2021); the 2021 CFR had 188,343 pages (+1.24% from 2020). Such figures show volume but not substantive impact.
• Promising paths forward: leverage PRA submissions and supporting statements with large language models and adapt the OECD Standard Cost Model to US data to measure administrative burdens; complement with focused quantitative and qualitative studies on industries, communities, firms, and beneficiaries to capture cumulative burdens and benefits and distributional effects.

The analysis shows that commonly cited metrics—pages, raw counts, and even sophisticated text flags—do not validly represent cumulative regulatory impact. More targeted proxies (OIRA-reviewed rules, Unified Agenda completed actions) better isolate consequential activity but still miss large swaths of the regulatory landscape and provide little insight into burdens versus benefits or their distribution. Bottom-up CBA aggregation offers the most policy-relevant quantification but remains incomplete and often inaccurate for aggregation purposes, leaving key questions about cumulative burden and benefits unanswered. These findings address the core question by demonstrating why existing counts cannot resolve whether there are “too many” regulations, and by outlining a more nuanced toolkit that combines: (1) mining PRA/ICB data with LLM-assisted parsing to quantify administrative burdens by affected populations and sectors; and (2) deep, context-rich studies (including qualitative methods) to illuminate how regulatory burdens and benefits accumulate for firms and individuals. Such an approach can inform debates about productivity, innovation, access to benefits, and distributional equity, and temper the political allure of oversimplified numbers.

Counting regulations is useful for tracking patterns but inadequate for judging cumulative regulatory impact. The paper’s contribution is a structured critique of major counting and impact-measurement methods, an updated synthesis of their limitations, and a roadmap for more meaningful assessment. Recommended directions include: continue tracking OIRA significant/economically significant rules and parsing Unified Agenda completed actions; systematically mine PRA/ICB datasets with LLMs and adapt the OECD Standard Cost Model to US contexts to quantify administrative burdens by sector and population; and conduct focused quantitative and qualitative studies of industries, communities, firms, and beneficiaries to capture real-world cumulative burdens and benefits, including distributional impacts. Together, these steps can produce a more balanced evidence base for regulatory design and reform.

The paper is primarily US-focused and does not provide original empirical measurements beyond synthesizing existing data and reports. Recommended LLM- and PRA-based analyses are prospective and require substantial validation, resources, and adaptation (e.g., filtering non-regulatory burdens, verifying burden estimates, and addressing missing benefits). Bottom-up OMB aggregates remain underinclusive and ex ante; Unified Agenda parsing is time-consuming and classification quality varies. State, local, and Tribal regulations and interactions with federal requirements are largely outside the scope. The qualitative approach proposed raises generalizability concerns and requires rigorous methodology.