How evidence-based theories of change can strengthen research impact on policy
- By the Institute for Methods Innovation
- 3 days ago
- 3 min read
Updated: 24 hours ago
Research organisations are increasingly asked to show not only that their work has had an impact, but how the impact was generated. Policymakers and funders now expect evidence of the processes that connect research to outcomes. They also require frameworks that can help anticipate and plan future influence.
A decade-long evaluation of Scotland’s Centre of Expertise for Waters (CREW), conducted by Professor Eric Jensen, Professor Mark Reed, Sarah Noles and Pauline Lang, demonstrates how this can be done. By constructing an evidence-based theory of change (ToC) at programme level, the study set out both a retrospective account of impacts and a framework for planning future work (Jensen et al., 2025).
From speculative models to evidence-based accounts
Conventional ToCs are often drafted at the start of a project to outline assumed pathways from inputs to impacts. In practice, many are speculative, developed internally without systematic data. This risks oversimplifying cause–effect relationships and overlooking contextual factors.
The CREW evaluation adopted a different approach. The team built a programme-level ToC from empirical evidence collected across ten years. Data came from surveys, interviews, and participatory workshops with researchers, programme managers, policymakers and beneficiaries. Project-level pathways were pieced together and combined into a coherent model, capturing both intended and unintended outcomes.
Key findings
The analysis showed that more than two-thirds of projects had demonstrable impacts. The most frequent were capacity-building effects, such as improved awareness of water issues, enhanced organisational decision-making, and changes to professional practice. Around 60 per cent of projects were reported to have influenced government policy, particularly within the Scottish Government, SEPA, Scottish Water and NatureScot.
Several enabling factors were consistently associated with a stronger impact. These included the early involvement of policy organisations in project design, active co-production during delivery, impartial project managers bridging the gap between science and policy, outputs tailored to decision-making contexts, and established relationships built on trust.
Longer projects with larger budgets were more likely to generate sustained impact, reflecting the time needed for research to shape policy.
Barriers to impact
The study also revealed recurring obstacles. Some projects shifted towards academic objectives at the expense of the original policy brief, reducing their utility. Communication was another constraint: outputs were at times too lengthy or technical for policy users, while in other cases they lacked the depth needed for regulatory contexts.
Dissemination beyond immediate participants was often limited, with insufficient time and resources allocated. Few projects had explicit impact strategies, leaving project managers to address this informally. Tensions between academic publication priorities and policy organisations’ need for rapid, applied outputs were a further source of difficulty.
Lessons for future programmes
The evaluation provides evidence-based guidance for strengthening research impact:
Involve both policy and scientific expertise at the outset to ensure projects are feasible and well-scoped.
Allocate dedicated resources for dissemination, engagement and impact planning.
Build and maintain trust through co-production processes and ongoing collaboration.
Communicate findings in formats appropriate for different audiences, balancing brevity and technical depth.
Support project managers in their role as boundary-spanners translating between research and policy.
These recommendations are applicable beyond the water sector. They outline general conditions under which research programmes are more likely to produce usable evidence for policy.
Implications
The principal contribution of this study is methodological. By grounding a ToC in long-term empirical data, the authors provided a more accurate picture of how impacts occur. The framework serves two purposes: it accounts for past outcomes and it informs future planning. The findings reinforce that impact is rarely immediate but develops through collaboration and trust built over the years. Programmes that anticipate these dynamics are better placed to deliver timely evidence for policy.
Conclusion
The CREW evaluation shows that research impact can be analysed in a structured, evidence-based way. Rather than relying on anecdote or simplified models, the programme-level ToC was derived from a decade of systematic data. For researchers, the lesson is that impact is shaped throughout the project life cycle, not just at the dissemination stage. For funders, the implication is that investing in impact planning, communication, and collaboration is as important as funding the research itself.
This blog summarises findings from Jensen, E. A., Noles, S., Reed, M. S., & Lang, P. (2025). “How can a research program influence public policy? Evaluating a decade of research impact using an evidence-based theory of change.” Environmental Science and Policy, 171, 104182. https://doi.org/10.1016/j.envsci.2025.104182
