Towards understanding the characteristics of successful and unsuccessful collaborations: a case-based team science study

The study addresses how interdisciplinary scientific teams can be evaluated for meaningful collaboration during development and process phases to predict and support successful outcomes. It highlights that complex, large-scale problems require both contributory (discipline-specific) and interactional (socialized) expertise. Prior SciTS work has largely relied on bibliometrics and archival data, often using single-method designs, which do not provide timely insights into team processes. Calls in the literature emphasize the need for longitudinal, mixed-methods, multilevel designs to capture team development and interaction over time. The paper situates its research question—whether and how we can determine if teams are collaborating in meaningful ways that lead to outcomes—within gaps around evaluating knowledge integration and limited practical guidance for funders and institutions to assess team science beyond traditional outputs.

The literature indicates knowledge creation and integration are fundamentally social processes, requiring both contributory and interactional expertise. Reviews of SciTS (e.g., 2006–2016) show heavy reliance on pre-existing and bibliometric data with few mixed-methods or longitudinal designs. Frameworks recommend multi-level, process and developmental evaluation (e.g., outcome, developmental, process evaluations) but have been sparsely operationalized. There are documented gaps across domains (One Health, sustainable agriculture, ecosystem services, sustainability science) on assessing knowledge integration capacity and practical, feasible metrics. Social network analysis has been advocated to understand patterns of interaction affecting team development. Prior findings suggest gender balance can enhance group process and collective intelligence, but mechanisms and actionable process measures remain underexplored.

Design: Exploratory, longitudinal, mixed-methods, case-based study of eight interdisciplinary scientific teams from 2015–2017 at a major research university program designed to catalyze team science. Seven teams were funded program teams; an eighth externally funded team volunteered. Teams spanned topics such as air quality, urban eco-districts, polymers, sensors, microgrid electricity, sustainable agriculture, and genomics. Total participants: 135 (including 17 students). IRB protocol #19-8622H. Case selection and program context: Teams self-formed and applied; finalists participated in a pitch session. Program goals focused on increasing interest in complex problems, leveraging diverse expertise, shifting funding toward collaborative endeavors, and producing large-scale proposals and impactful outputs. Data collection (six methods):

• Social network surveys at three time points (beginning, mid-point, end) capturing scientific collaborations (co-publications, proposals, consulting, student committees) and social relations (mentoring, advice, fun, friendship). Response rates varied by team (39%–93%). Names used to construct full networks then de-identified.
• Participant observation and field notes (2–4 meetings per team; exceptions: Team 1 lacked face-to-face meetings; Team 5 declined observations).
• Turn-taking observations during 1–2 meetings per year per team (who spoke, duration, knowledge types exchanged). Metrics derived: number of turns in 10-minute intervals, median turns per participant, deviations above/below median, and spread between highest and lowest relative to median.
• Interviews and focus groups (as part of the mixed-methods design; details referenced in abstract).
• Outcome data: Quarterly self-reports (for program teams) and NSF reports (for the external team): total proposal dollars submitted, total award dollars received, total publications; also average degree in final publications and grant networks. To account for team maturation, outcomes from the first year (quarters 1–4) were excluded; outcomes considered for quarters 5–9 and beyond funding end. Network construction and measures: Directed networks where nodes are team members and edges indicate perceived relationships (e.g., A names B as mentor). Calculated average degree (comparable across different-sized networks) and betweenness centrality for nodes within mentor, advice, friendship, fun, collaboration (combined co-granting, publications, consulting/research, student committees), and student committee networks. Statistical analysis: Kendall’s rank correlation (τ) to assess monotonic associations between process/development metrics and outcome metrics; permutation-based p-values used; p<0.10 considered marginally significant and p<0.05 significant. Process/development metrics were primarily mid-point measures to predict later outcomes.

• Role of women:
  • Proportion women negatively correlated with final grant network average degree (r = -0.52, p < 0.10).
  • Presence of a woman PI or woman on leadership was strongly correlated with total proposal dollars submitted (r = 0.86, p < 0.01).
  • Top woman betweenness in mid-point mentor network correlated with final publication network average degree (r = 0.60, p < 0.05), total proposal dollars submitted (r = 0.52, p < 0.10), and total award dollars received (r = 0.69, p < 0.05).
  • Top woman betweenness in mid-point collaboration network correlated with total proposal dollars submitted (r = 0.62, p < 0.05).
• Social networks at mid-point:
  • Fun average degree correlated with final publication network average degree (r = 0.60, p < 0.10).
  • Friend average degree correlated with final publication network average degree (r = 0.63, p < 0.10), total proposal dollars submitted (r = 0.60, p < 0.05), and total award dollars received (r = 0.78, p < 0.01).
  • Friend and fun networks were highly correlated (r = 0.90, p < 0.001).
  • Advice network average degree correlated with total award dollars received (τ = 0.55, p < 0.05).
  • Number of isolates in advice network negatively correlated with final publication network average degree (r = -0.69, p < 0.10).
  • Student committee network average degree correlated with total publications (r = 0.64, p < 0.05), total proposal dollars submitted (τ = 0.62, p < 0.05), and total award dollars received (r = 0.69, p < 0.01).
  • Mid-point collaboration network (2016) correlated with next-year (2017) publication network average degree (τ = 0.87, p < 0.05).
• Turn-taking:
  • Number of turns taken per 10 minutes positively correlated with total proposal dollars submitted (r = 0.80, p < 0.05) and total award dollars received (r = 0.80, p < 0.05).
  • Uneven turn-taking (spread between highest and lowest relative to median) negatively correlated with total proposals (r = -0.74, p < 0.05). Field notes linked uneven turn-taking to one person monopolizing time.
  • Proportion women strongly negatively correlated with turns above the median (dominant speaker), r ≈ -0.90 (p ≤ 0.001), indicating teams with fewer women more often had a dominant speaker.
• Overall implications:
  • Even turn-taking is strongly associated with better outcomes.
  • Women’s centrality in mentoring/collaboration networks and presence in leadership relate to stronger proposal and award outcomes.
  • Interpersonal social ties (friend/fun/advice) are more predictive of outcomes than many direct scientific collaboration metrics.

Findings support the central research question by demonstrating that process and development metrics—particularly those capturing interpersonal dynamics and communication patterns—provide timely, actionable indicators of team success. Even turn-taking emerged as a critical process feature, positively associated with proposals and awards, while uneven turn-taking was detrimental. Women’s roles, especially when central in mentoring networks or in leadership, were linked to enhanced proposal and award performance and to more even turn-taking. Mid-point social networks (friend, fun, advice) were strongly related to later scholarly and funding outcomes, suggesting that knowledge integration is driven by the quality of social relationships that sustain collaboration through challenges. The student committee network’s strong correlations with multiple outcomes suggest it may serve as a proxy for strong ties, cross-disciplinary linkage, and opportunities for shared language and trust-building. Traditional scientific collaboration measures (e.g., co-granting/publication ties at mid-point) were less consistently predictive than interpersonal metrics, implying evaluation frameworks should prioritize social integration indicators to understand and foster successful team science.

This study contributes a mixed-methods, longitudinal evaluation approach that integrates social network analysis, observational turn-taking metrics, and outcome data to identify early process indicators of team science success. It presents practical process measures—especially even turn-taking and social tie structures—that correlate with traditional outputs (publications, proposals, awards), and underscores the importance of women’s leadership and centrality in mentoring networks. The work advances the argument that successful team science depends not only on selecting the right experts but on building the right relationships for knowledge integration. Future research should: (1) expand mixed-methods designs and test additional process metrics; (2) deepen investigation into what constitutes even turn-taking across meeting types and contexts; (3) incorporate measures of learning, leadership, trust, inclusivity, and nuanced expertise understanding; (4) examine gender beyond binary categories, intersectionality, and other diversity dimensions; and (5) extend longitudinal follow-up to capture long-term outcomes (e.g., citations, team persistence).

• Real-world setting led to uneven participation and data completeness across teams (e.g., Team 5 limited data; Team 1 lacked face-to-face meetings). Possible observer effects.
• Turn-taking observations did not cover all meetings; observed meetings often mixed scientific and administrative agendas, complicating interpretation of evenness.
• Small sample size (eight teams) limits generalizability.
• Social network surveys capture single time points and self-reports; measures like “fun” derived only from surveys may not reflect ongoing experiences.
• Gender reporting options were binary; respondents all used binary identifiers, limiting exploration of non-binary gender and intersectionality.
• Only 17 students among 135 members; student committee findings may reflect complex roles not fully captured.