Frameworks for evaluating health research capacity strengthening: a qualitative study

Background: Health research capacity strengthening (RCS) projects are complex and challenging to evaluate. To inform health RCS evaluation, this study describes and compares key characteristics of existing health RCS evaluation frameworks—their development process, purpose, target users, structure, content, and coverage of important evaluation issues. A secondary objective was to explore use of the ESSENCE framework, which seeks to harmonise funders’ evaluation requirements. Methods: Health RCS evaluation frameworks published by seven funding agencies between 2004 and 2012 were identified and analysed using a mixed methods approach: structured qualitative analysis of documents, a stakeholder survey, and consultations with key contacts in health RCS funding agencies. Results: Frameworks were intended mainly for internal use by funding organisations and were oriented to organisational performance requirements. They referenced few theories specific to RCS. Generic tools (e.g., logical frameworks) documented activities, outputs and outcomes, with limited emphasis on underlying assumptions or contextual constraints. Reported usage of the ESSENCE framework was limited. Conclusions: Frameworks could be improved by including more practical guidance on conducting evaluations; involving stakeholders more (consistent with evaluation capacity building principles); explaining the frameworks’ underlying rationales; and structuring frameworks to separate generic from project-specific aspects of health RCS evaluation. The latter two improvements may assist harmonisation.