National Institutes of Health (NIH) grant awards: does past performance predict future success?

The National Institutes of Health (NIH) is the primary funder of biomedical research in the United States. The R01 grant is the cornerstone of this funding, and its acquisition has become a critical metric for evaluating faculty at medical schools. Success in obtaining R01 grants significantly influences decisions regarding faculty recruitment, salaries, promotions, tenure, resource allocation, and space assignment. Medical schools and academic medical centers (AMCs) actively compete to increase their R01 award counts to improve their rankings. A common strategy involves recruiting principal investigators (PIs) who already hold multiple R01 grants, providing an immediate boost in funding and experienced researchers. However, this approach requires substantial investment in higher salaries, recruitment packages, and space allocation. Institutions implicitly assume a long-term return on investment (ROI) based on the continued funding success of these established researchers. This study investigates this assumption, analyzing whether this strategy is sound, and whether highly successful PIs maintain their elevated funding levels over time.

The introduction mentions several works implicitly. The authors cite the inherent assumption that past grant success predicts future success. They also reference the competitive landscape among medical schools and AMCs to attract well-funded PIs to improve their institutional rankings. The paper implicitly references literature that supports the significance of R01 grants in academic career progression and institutional prestige. The discussion section explicitly cites several sources related to regression to the mean in financial markets (Bogle, 1992; Carhart, 1997; Malkiel, 1973; Murstein, 2003), drawing a parallel to the observed pattern in NIH grant funding.

The researchers obtained NIH grant data from the NIH Online Reporter Tool and the Palgrave Dataverse site. The data included all R01 grants awarded to US schools of medicine each fiscal year from 2000 to 2015. They analyzed both the number of R01 grants per investigator (using the NIH Contact Person ID) and the total grant funding amount, adjusting for inflation using the NIH Biomedical Research and Development Price Index. Funding amounts were converted into R01 equivalents ($375,000 per equivalent). To account for unfunded investigators, they used three methods to estimate the percentage of investigators without R01 funding: 1) subtracting funded R01s and non-R01 grants from the total number of applications; 2) directly counting unfunded investigators in their cohort analysis; and 3) comparing their results to previously published NIH data. They analyzed the data to determine if highly funded investigators consistently outperformed the NIH average over 5, 10, and 15-year intervals and also investigated if conversion of R01s to larger programmatic grants explained funding decreases. They also analyzed the longevity profile of investigator NIH funding by examining the changes in the number of R01s held by different cohorts after 5, 10, and 15 years. Finally, they conducted a secondary analysis focusing on continuously funded investigators (those with at least one R01 at the start and end of the analysis periods) to address potential biases from investigator retirement.

The study revealed a remarkably stable NIH investigator award profile over the 15-year period. Approximately 74% of R01-funded investigators held a single R01, while only a small percentage (less than 1%) held four or more. Including unfunded investigators, the average number of R01s per investigator dropped to 0.56. Analysis of highly funded investigators (those with multiple R01s in 2000) showed a significant regression to the mean over time. Even those starting with 5 or more R01s averaged only 1.1 R01s after 15 years. The proportion of investigators maintaining funding above the NIH average decreased dramatically over time across all cohorts, with a significant number losing all funding within 15 years. Converting multiple R01s into larger programmatic grants did not explain the regression to the mean; total funding amounts also regressed to the mean over time. Even when analyzing only continuously funded investigators (excluding those who may have retired), the regression to the mean was still observed. In summary, the study found that across various cohorts, grant success (whether measured by number of grants or total funding amount) exhibited regression to the mean over time. This pattern applied whether considering all investigators or only those continuously funded over the study period.

The findings challenge the common assumption that past grant success is a strong predictor of future success. The observed regression to the mean in NIH grant funding mirrors similar patterns in other areas, such as financial investment, where past performance is not necessarily indicative of future results. The study highlights the importance of considering factors beyond grant success, such as clinical care, teaching, mentorship, and service, when evaluating faculty performance and making institutional resource allocation decisions. While a small subset of investigators receives a disproportionate share of NIH funding, the regression to the mean indicates that this is not due to consistent bias towards specific individuals over long periods, but rather reflects fluctuations in the competitive grant landscape. The study's results suggest that relying solely on past grant success for recruitment and promotion decisions is statistically risky from an ROI perspective. This has important implications for institutions considering highly compensated individuals based purely on past grant success.

This research demonstrates that past NIH grant success is not a reliable predictor of future success. All cohorts of investigators, regardless of their initial funding level, regress towards the mean NIH funding profile over 10–15 years. Institutions should adopt a more holistic approach to faculty evaluation, considering factors beyond grant success, to ensure a balanced and sustainable research portfolio. Further research could investigate factors influencing individual variability in grant success, such as demographic factors and career stage.

The study focused solely on US schools of medicine and AMCs, limiting the generalizability to other types of institutions. The analysis did not account for demographic factors, which could have different impacts on funding success. It was also difficult to accurately account for investigator retirement, a factor potentially correlated with funding loss. The treatment of multi-PI R01s, and lack of consideration of non-NIH funding sources, also represent limitations of this study.