Section 6: How to evidence the significance of impacts from media engagement

This section covers:

  1. Approaches to evidencing gathering

  2. Overview of evaluation

 

Summary 

  • While press offices have many methods available for determining the reach of media engagement, evaluating the significance of benefits arising from those who have been reached is a major challenge

  • The easiest way to get evidence of the impact of media engagement is to persuade researchers to self-report their impacts. However, engagement with surveys about the impact of media engagement tends to be low among researchers. Alternatively, it may be possible to exchange evidence between teams in the University, for example impact teams may be able to provide evidence of significance while press office teams may be able to provide evidence of reach for the same researcher or project

  • The alternative is to collect new evidence of the significance of impacts. The four main approaches to collecting this sort of evidence are: i) quantitative, experimental and statistical; ii) theory of change and logic driven; iii) qualitative and arts-based; and iv) systems and pathway analysis. Methods for evidencing impact from media engagement in these four categories range from highly accurate but costly methods, to those you can use for free if you plan ahead and have enough time

  • Three key methods that are regularly used effectively to evaluate the significance of impact arising from media coverage are: i) social media analysis; ii) a funnel approach in which you direct a proportion of an audience to a website or other mechanism where you can follow up with them; and iii) before/after polling of media audiences

  • Each of the methods does three important things: they identify and describe benefits arising from media engagement; they do this based on a sample of the population who engaged; and they demonstrate cause and effect between the research and the benefits, so any impacts claimed are clearly attributable to the research

 

While measuring reach is fairly straightforward, there are no standard metrics that can tell you the significance of the benefits that arose for the audience that was reached. This is a problem if you want to claim impact from media engagement, as reach without significance means very little. Only once it has been possible to prove that (at least a proportion of)  those who engaged benefited significantly (e.g. they learned something new, gained a skill or changed a behaviour) does evidence of reach have value, because you are able to infer the scale of the benefits that have occurred. 

Watch the video above or read on to find out more…