This research investigates the presence of gender bias in the multi-stage peer-review process of research grant applications, specifically focusing on the impact of the gender composition of research consortia. The study acknowledges the existing literature showing that women in academia are often perceived as less capable and skilled in the sciences, leading to devaluation as scientific leaders and hindering their access to research funding. While prior research on gender bias in grant allocation provides mixed results, often focusing on a single evaluation stage, individual applications, or a single country, this study uses data from the EUROpean Collaborative RESearch (EUROCORES) Scheme to analyze a multi-disciplinary, pan-European context spanning 2003-2015. The study's three main objectives are to examine if gender bias impacts research consortia (not just individual PIs), to evaluate the consistency of gender effects across different evaluator groups (expert panels and individual reviewers) and evaluation stages, and to analyze whether reviewers' written reports align with their assigned scores. The study's significance lies in its contribution to understanding the persistent underrepresentation of women in top academic positions and its implications for institutions and policymakers.

The literature on gender bias in grant allocation is reviewed, highlighting the mixed findings of previous studies. Some studies found evidence of gender bias, while others did not. The inconsistent results are attributed to variations in methodology, such as focusing on single evaluation stages, individual applications, and specific countries. The inconsistencies across studies prompted the researchers to conduct a large scale study that addresses the limitations of prior research by examining a large multi-national dataset with multiple stages of review.

The study utilizes data from the EUROCORES scheme (2003–2015), a pan-European funding initiative promoting collaborative research across multiple scientific domains. The EUROCORES evaluation process involves three stages: (1) expert panel evaluation of outline proposals, (2) external reviewer evaluation of final proposals, and (3) a second expert panel evaluation and funding recommendation. The dataset initially included information on 10,533 applicants, 1642 outline proposals, 886 final proposals, 2182 external reviewers, and 491 panel members. After data cleaning to address incomplete records, the final sample consisted of 1347 CRPs, 9158 unique applicants, 467 panel members, and 1862 reviewer reports. The proportion of female PIs within each consortium served as the key independent variable. The dependent variables varied by stage: for stages 1 and 3 (panel decisions), a binary variable (selected/not selected) was used; for stage 2 (reviewer evaluations), the average reviewer scores for scientific quality and PI qualifications were employed along with sentiment analysis of reviewers' written reports. Statistical analyses included probit models (stages 1 and 3) and ordinary least squares (OLS) models (stage 2), with Heckman's two-step procedure addressing sample selection bias. Sentiment analysis was conducted using the VADER algorithm, and evaluative terms were identified using a Word2vec model. The models also controlled for various factors, including consortium characteristics (size, productivity, diversity, network proximity, etc.) and evaluator characteristics (gender, productivity, etc.).

The study's key findings demonstrate a consistent negative association between the proportion of female PIs in research consortia and evaluation outcomes across all three stages. At stage 1, a 1% increase in female PIs significantly reduced the likelihood of an outline proposal progressing to the next stage (~0.2%). At stage 2, a 1% increase in female PIs led to a 0.356% decrease in reviewer scores. Interestingly, sentiment analysis of reviewers' reports showed no significant relationship between gender composition and the valence or emotional intensity of reviews. This indicates a discrepancy between reviewers' qualitative comments and their quantitative scoring, suggesting implicit bias. Reviewers did not explicitly state female PIs were less competent, but their scores were negatively affected by the female ratio. In the final stage, no direct gender effect was found, but the strong influence of reviewers' scores (which exhibited bias) implies indirect bias influenced panel decisions. The influence of network proximity on grant success is also noted.

The findings strongly support the existence of gender bias against research consortia with a higher proportion of female principal investigators. The bias is evident across different evaluation stages and evaluator groups, highlighting the pervasive nature of the issue. The discrepancy between reviewers' qualitative feedback and quantitative scores reveals the potential role of implicit bias in shaping funding decisions. The sequential nature of the evaluation process acts as a filter, amplifying bias at earlier stages to further disadvantage female scientists. The results align with other studies showing that gender bias operates not only at the individual level but also at the group level. The paper contributes to the existing literature by showing the influence of gender bias in a multi-stage grant review process in a large European research funding context.

The study concludes that gender bias significantly impacts the success of research grant applications involving consortia with a higher representation of female PIs. This bias manifests across all stages of the evaluation process, amplified by the sequential nature of the review. The inconsistency between textual and numerical evaluations reveals the potential influence of unconscious biases. The authors suggest the need for more equitable grant peer-review systems to reduce biases and promote inclusivity in research funding. Further research could explore the impact of writing style on grant evaluation and the role of self-selection bias among female applicants.

The study acknowledges some limitations. First, it could not account for differences in the quality of written proposals or writing style based on consortium gender composition. Second, information on applicants' time allocation for work and family responsibilities was lacking. Third, the interdisciplinary nature of most EUROCORES projects hampered detailed analysis of gender bias across specific scientific macro-areas. Finally, the possibility of self-selection bias in female scientists' application decisions could not be ruled out.