Evidencing Impact from Media Engagement

Updated: Jul 29, 2020

Many of us think that media engagement and impact go hand in hand, but this is not necessarily the case. Engaging with social and mass media can be one of the most powerful pathways to impact, but because the effects are so hard to measure, it can be challenging to evidence impacts. I meet social media and television stars, researchers who have had documentaries made about their research, and experts who appear on news channels and front pages around the world multiple times per year. But even at these impressive levels of media exposure, I am often the bearer of bad news: “Great, you’ve got global reach and you’re famous”, I tell them. “But until you can tell me that someone somewhere actually understood something that you said, and ideally as a result of that that, they benefited in some way, then your impressive reach has very limited significance or meaning”.

This is a real challenge. How do you know that anyone actually was listening, let alone understood what you said in that media interview? If they understood something, then how do you know that it was important to them in some way? For example, do they understand something new, are they now aware of an important issue, has it changed their attitude, or have they done something differently as a result?

Many researchers see it as their responsibility to respond to every media request their press office sends their way. They do the media interviews and give little thought to what happens next for those who listened. Of course, your press offices can give you lots of metrics around circulation numbers, viewer numbers and social media metrics, but this only gives you evidence of reach. If you have no evidence of significant impacts arising from your media engagement, then all your reach has little meaning. You might have reached every person on the planet, but if they didn’t understand a word you said, what was the point?

Plan for your impact and how you will evidence it at the same time

What What you need is some kind of plan to evidence whether your media engagement actually makes a difference or not. Just doing more and more media work, unless you have a plan to evaluate it, may not be a great use of your time. Clearly there are reputational benefits as you get more widely known, and being visible may mean that you get opportunities for impact that you would otherwise not get. Your university will certainly want you to say “yes” to the media opportunities they give you, as it increases their visibility too, and has benefits for them in terms of their reputation, and press offices are driven by quite different metrics from researchers.

The number of people hearing about research from your institution is a key metric for press offices. But in addition to knowing how many people were reached, researchers need to know whether it actually made a difference to those people. To do that, you need to have an impact plan and as part of that you need to think about how you might evaluate the significance of the impacts you get through media (or any other form of) engagement.

Logic models are a fast and powerful way of planning for impact in ways you can evidence. By thinking ahead with a tool like this, it is possible to identify things you would expect to see on the pathway to impact (milestones) and consider indicators of impact or other evidence that might show when expected benefits have been achieved.

Read my guide to impact planning and download the Fast Track Impact planning template here

1. Identify impact goals for specific beneficiaries

The first step in this tool is to identify specific impact goals and the group who will benefit from the achievement of those impacts. An impact goal should always be a benefit. Based on that definition, reaching people, “dissemination” or social media metrics are not impact goals, because there is no way of knowing whether anyone who has been reached in these ways actually benefited as a result. For all you know, they may have been unable to understand what reached them, misunderstood it or been bored or offended by it. You need to engage with the media with a clear impact goal in mind, for example, you may want to:

  • Raise awareness about an important but little-known issue

  • Enable people to understand something or change their thinking about

  • something important

  • Change opinions and attitudes that are limiting people or causing harm

  • Mobilise action or change behaviours in individuals or households

  • Influence decisions by people with power to affect broader positive change

2. Design media activities to reach those goals for those beneficiaries

Now you know what you are trying to achieve, it is possible to harness the media in very different ways that are specifically designed to deliver those outcomes. For example:

  • You may turn down an opportunity for an interview with CNN International and spend the time writing a press release for Farmers Weekly if you need to raise the awareness of farmers about a threat, technology or action they can take based on your research

  • You might plan a workshop or open day for farmers and integrate that into your article to advertise the event and get greater engagement with the group

  • Alternatively, you might link the article to a new online toolkit or other resource that enables wider engagement with the group across the country

  • Then, you might use the coverage in Farmers Weekly as a route into getting a speaking slot at the Royal Welsh Show

  • While you’re there, you might arrange to do an interview with the Farmer’s Guardian at the show to get wider coverage across that community

  • While you’re doing this, you might work with your press office to get the coverage on social media, and target influential accounts who engage with large numbers of farmers and hashtags like #agrichat to get wider readership and funnel social media readers to your workshop/open day, online resource and show event

  • At each of the events and via your website, you would incentivise people to leave their contact details so you can follow up with them long-term to provide more benefits and ultimately be able to follow up with a short questionnaire to evidence your impact, all GDPR compliant

This highly targeted approach to media engagement stands in contrast to the usual approach which tends to maximise reach without being particularly sensitive to the audience. Raising awareness amongst millions of members of the public about something farmers need to do may have little benefit if few farmers actually hear or understand the message.

Having said that, there is still a role for media opportunities that get you significant coverage, as the more prominent the coverage the more likely you are to include your target groups. You are also like to reach additional groups that you may not previously have been aware were interested in your work. This can be an important way of extending the reach of benefits from your research. However, you have to be ready to walk through the doors that open to you after a major media opportunity, so it is important to make sure that you are easily discoverable via a Google search and have time in your schedule to follow up with the opportunities as they arise.

3. Identify indicators you can use to evidence impact

Evaluation is built into my impact planning template. As you consider what impacts you want to see (point 1 above), and the kind of media engagement you’ll need to get those impacts (point 2 above), you need to consider how you’ll know if the media engagement is working, and whether or not it is generating the benefits you’re aiming for. Press offices are good at collecting data on the success (or otherwise) of your media engagement, collecting all the coverage and telling you the circulation and viewing figures, and social media metrics. Increasingly there are a range of other altmetrics that you can get around your media engagement that will tell you more about the number and types of people who are talking about the work, and where they come from. These kinds of metrics can be a useful starting point for evaluating the significance of your impact. However, in isolation all this data really tells you is your reach.

This is a key weakness of traditional media monitoring, because without evidence of any benefit for the millions of people you reached, it is difficult to argue that there is in fact any impact if you use technical definitions of impact (e.g. from REF) or academic definitions of impact, which all emphasise the significance of benefits as well as their reach. As a result, there is widespread scepticism across the impact community of the value of media engagement as a pathway to impact, and there are increasing reports of academics being told by impact teams to re-prioritise their time away from engaging with their press office, towards more targeted approaches that can generate evidence of significant benefits. This is unfortunate, given the potential scale of benefits that can arise from well-managed media engagement that has been embedded in a well thought out pathway to impact. Based on the points above, you have now created a plan that uses media engagement in a much more targeted way. Now, you just need to make sure you can measure the significance of those benefits in addition to their reach.

The logic model approach makes this easy, because it asks you to identify indicators of success, which do not require any specialist expertise. If your impact goal is to change people’s awareness of the benefits of maggot therapy and shift attitudes to reduce levels of disgust and increase the likelihood that a patient might accept the treatment on the NHS, then you know exactly what you are looking for as an indicator of impact. In this case, you would expect a sample of people exposed to your media coverage to have different levels of awareness and changed attitudes before and after engaging with the coverage, and you would then ultimately expect to see an increase in uptake of the therapy in hospitals. Developing indicators like this does not require any social science expertise – just common sense. Ask yourself what success would look like, and then ask how you might be able to evaluate that.