Section 6: Demonstrating impact with evidence
This section covers:
-
Approaches to evidencing gathering
-
Overview of evaluation
Summary
​
-
While press offices have many methods available for determining the reach of media engagement, evaluating the significance of any benefit/impact arising in that audience is a major challenge.
-
The easiest way to get evidence of impact from media engagement is to persuade researchers to self-report their impacts. The Conversation’s platform prompts authors to add details into their dashboards, for example, which are in turn available to press teams. However, the number of academics engaging with such surveys can tend to be low.
-
Exchanging information between services teams can be beneficial, with research impact teams more likely or able to provide evidence of significance, while press offices can provide evidence of media reach. Bringing all this data together can improve the quality of REF impact case study narratives.
-
The alternative is to collect new evidence of the significance of impacts. The four main approaches to collecting this sort of evidence are: i) quantitative, experimental and statistical; ii) theory of change and logic driven; iii) qualitative and arts-based; and iv) systems and pathway analysis. Methods for evidencing impact from media engagement in these four categories range from highly accurate but costly methods, to those you can use for free if you plan ahead and have enough time.
-
While measuring reach is fairly straightforward, there are no standard metrics that can tell you the significance of the benefits that arose for the audience that was reached. This is a problem if you want to claim impact from media engagement, as reach without significance means very little. Only once it has been possible to prove that (at least a proportion of) those who engaged benefited significantly (that is, they learned something new, gained a skill or changed a behaviour) does evidence of reach have value, because you are able to infer the scale of the benefits that have occurred.
​
Step into the detailed topic below, or for a quick read skip to the next section…