Mapping the community: use of research evidence in policy and practice

The paper addresses how research evidence (URE) is used in policy and practice across disciplines and domains, noting increased attention to evidence use alongside challenges in accumulating knowledge and improving evidence use. The authors aim to map the community engaged in URE to provoke dialogue on the field’s current state and future needs. Initial observations indicate multiple disciplinary communities with distinct terminologies that hinder cross-disciplinary collaboration, and a larger, unmapped universe of scholars, funders, practitioners, and policymakers involved in producing, using, and supporting evidence. The study focuses on an initial mapping of scholars studying URE to understand strengths and challenges and identify opportunities for broader coordination. The research questions are: To what extent does URE span traditional boundaries of research, practice, and policy? Of different practice/policy fields? Of different disciplines?

URE builds on the knowledge utilization tradition, defined as activities to increase the use of knowledge to solve human problems, broadly construing both evidence and use (e.g., instrumental, conceptual, political/strategic, symbolic). Historically, interest peaked in the 1970s–1980s with seminal works, and has re-emerged in the last two decades with recognition of links between evidence production (funding, practices, partnerships, infrastructures) and use (communication, dissemination, reception). Practical investments include journals (e.g., Evidence and Policy), UK What Works Centres, US Institute for Education Sciences’ knowledge utilization centers, and philanthropic support (e.g., William T. Grant Foundation). Scholarship on evidence production often sits in science policy, research assessment/evaluation, and science and technology studies (STS), while evidence use is prevalent in applied social sciences (e.g., health, education), with exceptions in innovation/technology transfer and engaged research. Both production and use communities recognize the importance of involving users in setting research priorities. Recent efforts seek to engage an interdisciplinary community under labels like meta-science and research-on-research; however, the field is not truly “emerging,” with contributions from diverse disciplines over decades. Claims of novelty risk overlooking existing knowledge, leading to duplication and missed opportunities for cross-disciplinary learning. The diversity is a strength, offering varied theories and methods, but also presents challenges: dispersed communities, varied terminology, differing venues of dissemination, and geographic/sectoral fragmentation that inhibit boundary crossing.

Design and timeframe: Iterative surveying conducted between 2016–2018 to map the URE scholarly community. Survey content: Participants (scholars, practitioners, funders) were asked to: (1) characterize their discipline, role, and policy/practice sector; (2) identify up to five individuals (scholars or advocates) who influenced their perspectives on evidence production and use; (3) list up to five key references influencing their URE work. Sampling: Seeded with invitees to the William T. Grant Foundation Using Research Evidence meeting in 2016 (n=102), recognized as over-representing US-based and foundation-funded scholars with a focus on child education and welfare. Snowball sampling invited up to five nominees per respondent, yielding 219 invitees total and 80 respondents (39 from original 102; 41 from additional 117). In 2018, the survey was also administered at the Nuffield Foundation’s Transforming Evidence meeting (n=54), an international, cross-disciplinary convening. Combined, the dataset includes 134 participants. Analytic approach: - Descriptive statistics to profile disciplines, policy/practice fields, funding sources, and keywords/literatures. - Social network analysis (UCINet) of nominations to examine network cohesiveness (density), fragmentation, and disciplinary clusters; calculated homophily via respondent egonetworks as the proportion of same-discipline ties (less sensitive to missingness). - Bibliometric-style analysis of nominated references (counts by type, journals, most-cited authors/venues, cross-field overlap) to assess fragmentation or coherence of the knowledge base. Limitations of method include cap of five nominations potentially omitting weak ties and sensitivity of whole-network metrics to missing data and response rates.

Composition across sectors and fields: - Settings: Over two-thirds of respondents work in academic departments or university-based research centers; others are in independent research centers/think tanks (11%), philanthropic organizations/research funders (11%), non-profits/NGOs (5%), and government agencies (2%). - Policy/practice fields (primary focus): Education 26%; Health sciences 22%; Criminal justice 10%; Public administration 8%; Innovation and science policy 8%; Human services (social work/child welfare) 8%; International development 3%; Communications 3%; Social policy 3%; Conservation/environmental science 3%; plus additional areas (e.g., housing, community psychology, sport, urban policy, evaluation). Many work across multiple fields. Funding landscape: Respondents reported a wide array of funders, spanning private philanthropies, national and subnational governments, and international bodies (e.g., William T. Grant Foundation, ESRC, NIHR, NIH, NSF, EU/Horizon 2020, Wellcome Trust, Gates Foundation, Spencer Foundation, etc.), indicating broad but diffuse support that may challenge coordination and sustained programs of research. Disciplinary makeup and topics: - Primary disciplinary traditions: Sociology 27%; Political science 20%; Organizational studies 14%; Psychology 12%; STS 8%; Communications 3%; Economics 2%; Evaluation 1%; Social policy 1%; with occasional mentions of law, learning sciences, race studies. - Topical self-identification (90% selected one or more): Policy studies 21%; Knowledge utilization 18%; Evidence-based/informed decision-making/policy/practice 17%; Research impact 15%; Implementation science 11%; Knowledge mobilization 9%; plus 26 additional, rarely repeated terms. - Keyword diversity: 263 different self-descriptive keywords, reflecting multidisciplinarity and potentially hindering discoverability and cumulative knowledge. Network structure and interdisciplinarity: - Sociogram shows one large cluster with several smaller, largely disconnected clusters, broadly corresponding to policy sciences/social work/public administration. Dense sub-clusters by field (e.g., education, health sciences, organizational studies). - Cohesiveness: very low density (0.003) and high fragmentation (0.990), indicating limited overall connectivity. - Homophily: Respondents’ egonetworks include substantial cross-disciplinary ties; mean proportion of outside-discipline ties ≈ 0.50 (range 0–1), suggesting potential for boundary spanning despite siloing. Knowledge base and citations: - 185 nominated references spanning 50+ years: 62 books; 21 reports/conference proceedings; articles from 57 journals, with strong representation from health and education. Implementation Science was the most referenced journal (five publications). Carol Weiss was the most nominated author (eight distinct publications). - Fragmentation: Of 182–185 references, 140 were nominated by only one person; only 45 were nominated by two or more. Only four references were nominated more than four times, all from politics/public policy, public sector management, and evaluation. There is little evidence of shared, cross-disciplinary canonical literature or of empirical studies building cumulatively on core works.

The URE community exhibits strengths—multidisciplinarity, boundary-spanning roles, emerging conceptual coherence, and broad funding interest—but is hindered by fragmentation in networks and literatures, discipline- and policy-specific terminology, and limited formal infrastructures typical of established fields (e.g., dedicated associations, conferences, journals, career pathways). Network analyses indicate insufficient cross-disciplinary connectivity to optimally advance the field, despite some individual boundary-spanning ties. The fragmented citation patterns make it difficult to articulate coherent best practices or to build cumulative knowledge, risking redundant or competing findings rather than robust, generalizable insights. To enhance connectedness, the community may benefit from field-building structures (associations, cross-disciplinary conferences, inclusive journals) and intentional funding of sustained, interdisciplinary programs. Inclusion must extend beyond academia to policy and practice communities and underrepresented perspectives (e.g., indigenous knowledge), leveraging co-production and research-practice partnerships to center users and improve bidirectional learning. Advancing influence in policy and practice requires synthesizing and promoting coherent, evidence-based approaches to capacity building for evidence use. Long-term sustainability necessitates coordinated, cross-funder recognition of URE as a distinct, cross-cutting area, with investments enabling careers and large-scale, cumulative empirical and theoretical work.

The study indicates that the community engaged in producing and using research evidence is active yet poorly connected, leading to duplication, limited accumulation of knowledge, and over-claiming of novelty. There are no simple solutions to improving evidence production and use; progress requires human interaction, joint learning, and investment in people and careers. The authors call for: - Time and opportunity to identify and map all those working on related questions. - Engagement with different communities to understand their research traditions, terminology, and contributions. - Opportunities to spot and broker links to foster interdisciplinary collaboration. - Leadership, investment, and venues to share learning. With sustained, coordinated support from funders, associations, conferences, journals, and institutions, the community can better connect, become more inclusive, build cumulative knowledge, and transform the use of research evidence across policy and practice.

Findings reflect a partial, potentially biased view of the broader URE landscape: - Sampling bias: Seeded from William T. Grant Foundation URE meetings (US-focused; emphasis on child education and welfare) and the Nuffield Transforming Evidence meeting; likely over-represents US and foundation-funded scholars and specific policy areas. - Incomplete coverage: The larger universe of scholars, funders, practitioners, and policymakers working on evidence production and use remains unmapped. - Network data constraints: Respondents limited to five nominations; response rates and missing data reduce completeness; whole-network measures (density, fragmentation) are sensitive to missingness; cap may underrepresent weak inter-disciplinary ties. - Exploratory nature: Results should be interpreted as preliminary, given sampling and measurement limitations. - Diversity of terminology and self-identification complicates consistent categorization, potentially inflating perceived fragmentation.