In this blog, we analyse ‘pathways to impact’ in grant applications that led to the top scoring impact case studies in the latest UK assessment of research excellence and impact (REF2014). It is easy to find the top scoring case studies, and straight-forward enough to find good examples of pathways to impact, but this is the first time that pathways have been paired with high-scoring case studies.
To do this, we contacted those responsible for the majority of impact case studies we could identify that scored the highest grade (4*) in REF2014, and asked if they would allow us to share their pathways to impact. In our analysis, we have identified elements that pathways to winning impact case studies have in common, from a wide range of disciplinary areas. In doing so, we provide suggestions that complement official guidance on writing ‘pathways to impact’ and can help you develop a pathway to significant and far-reaching impacts.
A bit of context
Countries around the world are developing systems for evaluating the excellence and impact of Government funded research, and the UK is ahead of most in its assessment of research impact. The 2014 Research Excellence Framework requested, for the first time, that institutions evidence how some of their research had delivered beneficial impacts for "the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia". Research Councils UK (RCUK) “encourage researchers to be actively involved in thinking about how they will achieve excellence with impact and to explore pathways for realising the impact of their research”. RCUK continue to work with the research community to develop its guidance for Pathways to Impact and you can read their guidance, including FAQ’s and top tips, for Pathways to Impact here.
At the beginning of 2015, HEFCE published a searchable database of impact case studies, collected as part of its evaluation of UK research under their REF. We used this database in a blog we wrote on top ways researchers achieve policy impacts and decided to make use of it again for this analysis.
Sarah reached out to 50 researchers from a range of institutions and disciplines that had achieved 4* impact case studies, and we selected six of the pathways to impact we received for analysis. For comparison, we have also included the pathway to impact for a lower scored (3*) case study that Mark wrote. In addition to a detailed thematic analysis of these documents, we gathered general information from the researchers who didn’t have any impact planning documents to share. Most of the successful impact case studies were based on grant applications that were written prior to the introduction of pathways to impact sections, but instead they sent us the most relevant part of their application or other impact planning documents they had developed at the start of the research (Table 1).
Table 1: Impact case studies used for the analysis, showing the nature of impact planning documents provided, the score given under the Research Excellence Framework (REF), the submitting institution and the discipline (REF Unit of Assessment)
Pathway | Case Study Implementing Evidence-based Community Stroke Services
Pathway | Case Study Displaying the Flag: Transforming Conflict in Northern Ireland
Pathway | Case Study Preventing Psychosocial Risks and Work-Related Stress in Europe: Impact on Policy and Practice
Pathway | Case Study Changing Policy And Practice In The Prevention Of Suicide And Self-Harm
Pathway | Case Study Mapping Medieval Chester: driving heritage policy, expanding heritage audiences and creating new cultural and economic opportunities
Pathway | Case Study The Old Bailey Online: Democratising access to social history
Pathway | Case Study Paying for Nature’s Services: Developing the UK Peatland Code
10 lessons from pathways to top-rated impacts
Although impact is now a focus for research funders around the world, guidelines are often vague and there are few examples of good practice available from which to generalise lessons. Here we have identified ten lessons from pathways to top-scoring impact case studies:
1. Clear connectivity from overall vision to objectives and impact
In all the highly rated case studies, the impact planning demonstrated a strong connection from the overall research vision and purpose to the impact objectives and outputs. The take-home message is to make sure there is a strong narrative – a thread running through the whole programme of work. Illustrating this in some way can be helpful to get your team on board and see how everything fits into the bigger research picture. Take a look at this Goals to Impact document provided by Dr Fisher at the University of Nottingham.
This seems like an obvious point but when you compare successful pathways to impact with those that failed to get such a good rating, you really start to notice just how important it is to BE SPECIFIC. It’s very easy to get caught up in answering ‘what’ you will be doing where in actual fact that can usually be described in a few sentences, e.g. “we will develop an online learning platform.” Overall, the real crux of impact information will fall into three categories; the ‘(audiences), the ‘ (methods) and the ‘(dates, phases).
The ‘who’ - Identifying specific audiences.
All of the successful REF impact case studies identified very specific target audiences. It is not necessary to cover a wide breadth of people but demonstrating you have carried out an audience analysis and will be targeting the most relevant groups or individuals makes your impact more targeted and more likely to be successful. Some detailed plans even listed individual names and numbers of individuals within certain target groups. This may not be necessary in all cases but the level of detail is important here. Stating ‘general public’ may be suitable in some situations, but detailing ’we will make contact with 14 members of X institute’ is more tangible. The Pathway to Impact from the University of Hertfordshire planned to create an open-access, on-line and searchable edition of the Old Bailey Proceedings, 1670-1834. In the Pathway to Impact, three specific user-groups were identified and the document stated that “for each of these groups we have undertaken an extensive, but non-statistical, survey of need, based on the distribution of a project proposal and a request for direct (but unstructured) feedback.” The research on the audiences had already been carried out so readers of the pathway knew that the research team were completely ready to go. They had also detailed how many organisations within each user group they had already consulted with. When compared to the groups identified in Mark's 3* case study (e.g. government departments and future buyers), there is a lack of detail that makes the pathway less convincing.
The ‘how’ – how will you impact these groups?
If you’ve gone into detail as to who you are targeting with your impact, then the methods of how you will achieve this need to be just as detailed. For example, if you’ve identified 15-25 year old environmental activists as your target audience for impact, you then need to detail exactly how you will engage with them. This is the point to draw on specific team members that have existing relationships, e.g. “Professor Carle has been working with Activist Group A for four years and will lead a workshop with 15 of his contacts at Organisation X”.
The ‘when’ – plan your impact in clear, logical phases
All of the pathways to impact that achieved great results (and high ratings) followed logical phases when planning impact. You need to make your plan logical in terms of when and where you are planning impact. Implementation for one of the 4* impact case studies approached services across specific geographical areas during certain time-periods. Others divided their impact into clear thematic areas. However they did it, all of the successful pathways had clear ‘phases of impact’.