Research-based Theory of Change

 

Most resources for Theory of Change start with a development project or intervention, but if you are starting with research, this guide will help you develop a research-based Theory of Change. This can enable you to identify if there are parts of your research that do not underpin impact, so you can identify new impacts and activities, or redesign the research to deliver more impact. You can use this guide yourself to develop a Theory of Change, or you can use it co-productively to do this in collaboration with in-country partners (to do the latter effectively will take 2-3 months). 

 

In summary, there are four things you need to do:

  1. Identify in-country partners, stakeholders and publics

  2. Identify impacts that meet evidence-based needs from the bottom-up

  3. Co-design your research to meet evidenced needs

  4. Turn your impact plan into a Theory of Change

 

The rest of this guide explains how to do each of these things, with links to worked examples. 

 

 
1. Identify in-country partners, stakeholders and publics

 

Coproduction is an approach to research that enables in-country partners, stakeholders and publics to work with you to set the research agenda, manage (and in some cases help conduct) the research and interpret your findings, equally valuing and critically testing local knowledge alongside scientific knowledge. Coproduction requires humility, empathy and a genuine desire to seek new ideas, empowering groups who wouldn’t normally engage in the research process to help direct the research and impacts you generate together. Find out more about co-production and other forms of engagement in Prof Reed's recent review paper.

 

The first step in a co-productive approach is to identify in-country partners, stakeholders, and publics as early as possible, before you have fully developed your research ideas or started to think properly about impact. You need to do this systematically so you cover the relevant countries and issues, and avoid marginalizing key groups. The  partner, publics and stakeholder analysis template below can help you do this by thinking about the relative interest, benefit and influence of organisations you might want to work with. To do this, share the first page of the template below with your team, including any in-country partners you have already identified (you will use the second page later on to select project partners and advisory board members). To make this easier, you can turn this into a Google Sheet or similar. If you and your team are struggling to identify potential partners in the region you want to work in, UKCDR have some good advice on identifying partners here.

 

Download stakeholder analysis and partner identification template Word | PDF

 

This is how to use the first page of the template enables you to identify partners, stakeholders, and publics to engage with your work:

 

  • Name of the organization or group. Write the name of the organization (e.g. United Nations Food and Agriculture Organisation) or group (e.g. smallholders in the Kenyan highlands)

  • Likely interest in the project. Rate this as high, medium or low. This is highly subjective, so don’t spend too long agonizing over this, but try and enter something here as we will use it later on

  • What aspect of the research are they likely to be interested in? Any organization is likely to have many interests. You are not trying to capture their full range of interests. Instead, your task here is to consider how their interests might intersect with your research. For example, they may only be interested in your theory or methods, or in one potential finding or case study. You will use this later to group stakeholders and publics around particular benefits arising from your impact goals (in your Impact Summary)

  • Influence and benefit. For stakeholders and partners, consider what level of influence might they have on your capacity to conduct the research and/or generate impact, rating this high, medium or low. For publics, rate the level of benefit might they derive from the research

  • Comments on the level of influence and/or likely benefit. Explain your rating in the previous column, writing about the influence and/or benefit the organization or group is likely to have or get from the research. You can write about either or both of these – just include whatever you think is most useful to know. For example, there may be certain times or contexts in which the organization might have more or less influence over the outcomes of your research, ways they might block or facilitate the research or impact, and different types of benefit they might derive from the research

  • Is interest low but influence and/or benefit high (i.e. might they be hard-to-reach)? Answer yes or no to this question, and if the answer is yes, think about what you might be able to do to overcome barriers to engagement. Some hard-to-reach groups may regularly be marginalized in research, policy, and practice, and you may need to work hard to find ways of overcoming barriers to engagement. Other hard-to-reach groups may turn out to be gate-keepers who may prevent you getting access to data, sites, and people if you do not engage early with them, but your work may not be relevant enough for them to be interested at this stage. 

  • Partner information. If the organization is being suggested as a potential partner, you may want to ask some additional questions at this point, to help you decide which suggestions to accept, for example, why should this organization be prioritized as a partner, and how strong and longstanding is the relationship between the person suggesting the partner and the organisation?

  • Who is responsible for the contact? If you are using this tool across a team (especially if you use a Google Sheet), you will need to keep track of who proposed each partner, stakeholder or public group. Later on, if you want to invite them to become a partner or join your advisory board, this invitation can come from a member of your team who they already know, increasing the likelihood that they accept the invitation

 

To use the assessment grid on the second page to choose partners:

 

  • Assess the fit of each potential partner to the scope of your bid, in terms of research content, achievement of relevant impacts and anything else that is relevant (scoring each partner 1-3, low-high)

  • Assess (on a scale of 1-3) whether there is evidence of an ongoing, strong relationship between the person who suggested the partner and the organization. It is more likely that your team will be able to develop a successful working relationship with the partner and it will be easier to do due diligence if you are funded

  • Assess (on a scale of 1-3) whether the partner is registered and/or based in the countries you want to work in (assessing the relevance of international partners to the region). This is a key relevance criterion but also helps make sure the balance of funding is skewed appropriately to in-country partners

  • Assess (on a scale of 1-3) the potential of this partner to help generate impact at a scale relevant to their operation, including their credibility, influence, connections, resources and capabilities

  • Add your scores to get a total, and then sort your spreadsheet to rank partners by this score

  • Do some screening to make sure you get a good balance of stakeholders. You can add to this list (e.g. screening for coverage of issues or interests), but you may want to consider:

    • Listing the country, so you can see if all your top rated partners are from one country, and scan down your list to choose lower ranked partners from other countries to ensure you have geographical coverage

    • Categorise each organization by type, for example are they a Higher Education Institution, a case study partner or an organization that can enable you to scale findings to higher governance levels or wider geographical scales. You may, for example, want to have at least one in-country HEI per country or case study, and you may want to prioritise engagement with some organisations at higher levels to ensure your impact is as far-reaching as possible

  • Based on this screening, you can now prioritise who to invite as project partners or advisory board members. For more information on constructing a project advisory board, see this guide.

2. Identify impacts that meet evidence-based needs from the bottom-up

 

Once you’ve identified your partners and key stakeholders, you can start to identify potential impacts linked to your research area. For this to be genuinely co-productive, you need to enable all members of your team, especially your in-country partners, to identify potential impacts based on in-country needs and priorities in the general thematic area of your team’s research interests. It is important not to be too prescriptive about the research goals at this point, as these should arise later from the needs that have been identified. For this to work, you need to have an in-depth understanding of the context in which you will be working, including evidence of needs your research can help address. Ideally, there should be baseline data showing the scale of the issues you plan to tackle, how many people are affected from which social groups in which locations, and any relevant trends in livelihoods, poverty or other pertinent data. Although you will be relying significantly on in-country research and delivery partners to identify evidence-based needs, a familiarity of your wider research team with the local context is essential if you are going to develop a Theory of Change that does not make inappropriate assumptions about the local context. For example, familiarize yourself with:

  • Socio-economic, cultural, political and technological factors that might influence your capacity to deliver impact

  • Political factors and power relations between key stakeholders

  • Local capacity, skills, and resources to engage with and deliver research and impacts

 

You can capture this information in the first three columns of the impact planning template, which asks you and your in-country partners to identify potential impact goals, evidence of need and target populations.

 

The template below is a “logic model” and bears some resemblance to Logical Framework Analysis, but with two crucial differences. First, it removes the confusing terminology around inputs, outputs, outcomes and impacts (many outputs and outcomes are technically impacts, even if they will lead to other types of impact in future, e.g. awareness building impacts may lead to attitudinal changes and then to behavior change or decisions that deliver enhanced welfare or economic growth). Second, it is designed to be used for research, rather than development projects. Although you can identify impacts that do not link to your research in an international development project, the majority of your impacts should be underpinned by research, and this template is designed to ask how impacts might relate to research, so you make this link. Send the impact planning template to all your in-country partners and other key research teams, outlining the research area only broadly at this stage (e.g. interaction between local and global drivers of food security and malnutrition in southern Africa).

 

Download Impact Planning Template for international development projects Word | PDF

This is how to use the impact planning template:

  • Identify an impact goal. You may want to ask partners to identify up to three goals to keep this task manageable and avoid people wasting time listing multiple goals, given that you will only select some for inclusion in the proposal. Your impact goal should be a specific benefit for a specific organization, population or group that meets an evidenced need, leading (directly or indirectly) to enhanced welfare and/or economic growth. This narrow, instrumental focus on welfare and economic growth is important because projects have to demonstrate how they meet the definition of Official Development Assistance. However, you can propose many other types of impact, which may indirectly lead to these narrower benefits over longer periods of time. In particular, for example, you will want to include some capacity building impacts for a a funding proposal. To help do this, you may want to consult our what is impact page, which includes a list of 10 different types of impact you may want to pursue. If you are struggling to identify impacts, start instead by listing stakeholders and publics, and identifying their likely interests in the work, and identify benefits linked to these interests.

  • For a funding proposal it is important to be able to justify why you have chosen each impact goal, so the second column of the template asks you to describe any evidence they may have to justify the need to pursue impacts in a given area

  • Target stakeholders or populations. This column asks you to identify who will benefit if you achieve your stated impact goal. Provide as much detail as you can about specific organisations (e.g. including the division or team within that organization that will benefit) and populations (e.g. including population size or characteristics). If you previously completed the partner, publics and stakeholder analysis template, you can draw on this to complete this column. Note that a particular organization or group may be relevant to more than one impact goal, so you may need to copy and paste their details to other rows of the template

  • Reasons for being interested in the project. Explain what aspects of the project this organization or group is most likely to be interested in, leading to impact. The organization may have many interests, but you need to focus specifically on how their interests intersect with the project around your identified impact goal. At this stage, “the project” is likely to be focused around a broad research area that matches the interests of your team, rather than being a well developed research proposal with specific research questions

  • Activities to engage this target group. Identify specific activities that are tailored to the interests and characteristics of each organization or group. Bear in mind that you may need to identify very different activities to engage with different groups around the same aspect or your research and the same impact goal. Consider what resources you might need to conduct these activities and add those to the resources column (further to the right in the template)

  • Indicators of successful engagement [and means of measurement]. How will you know the activities you have planned (in the previous column) are working, and you are on a pathway to impact, rather than heading towards a dead-end or a cliff-edge of unintended consequences? Identify indicators that you could monitor to determine whether or not your activities are working, and make sure you have a way of collecting the relevant data, whether qualitative or quantitative, primary or secondary. Put these means of measurement in square brackets after your indicator to make sure you have something that you can realistically monitor. Consider what resources you might need to conduct these activities and add those to the resources column

  • Indicators of progress towards impact [means of measurement]. How will you know when you have achieved your impact? What milestones do you expect to see on your pathway to impact? Some of these might be interim impacts in their own right that you will want to claim as impact. Again, make sure you identify how you will actually collect relevant data that you can use to assess whether or not you are achieving impacts and reaching milestones, adding any necessary resources to the relevant column in the template. At this point is also worth revisiting the impact goal to make sure it is specific, measurable, achievable, relevant and timely (SMART)

  • Risks. What assumptions have you made along your pathway to impact? Are all these assumptions valid, or might your pathway potentially unfold differently? What are the risks to your activities not working as planned? What are the risks to your impacts not being achieved, or leading to negative unintended consequences? Rather than hide from these, it is important that you show you are aware of these potential challenges and can mitigate them (the next column)

  • Risk mitigation. For each of the risks you have identified, how might you mitigate these risks, or what would be your Plan B if your initial pathway to impact doesn’t work?

  • Who is responsible and what resources are needed? Identify who will be responsible for designing and implementing activities and monitoring progress, costing in staff where necessary. Identify any other costs associated with achieving this impact goal

  • Timing. When do you expect the impact to be achieved? When will you conduct the various activities along your pathway to impact?

 

As completed impact plans come back from your team, you will need to check that these have been completed sufficiently well for you to use (some may have only completed one column or may have misunderstood what was being asked).

3. Co-design your research to meet evidenced needs

 

The next step is to co-produce a research concept note that outlines the key research questions that will meet key needs you have identified via your impact planning template. Not all of the needs identified in the templates will be tractable or relevant for your research. Identify those needs that you could credibly address through your research and focus on these for your initial outline. This could be 1-2 pages, identifying areas where you want inputs from in-country and other partners, or it could be something more detailed, but ideally it needs to be written in plain language. 

4. Turn your impact plan into a Theory of Change

 

Based on the impacts you have prioritized as being tractable via research (the previous step), you can now turn these selected impacts into a Theory of Change. To do this, you need to follow two processes. First, the bottom-up process enables you to create a first draft that is based on the ideas that came from your team. This ensures that your Theory of Change is rooted in local needs and priorities linked to research you can conduct. However, the results may not be particularly systematic or comprehensive. Therefore, the second process is a top-down check that you have covered all the key issues systematically, including all the necessary causal links in a way that is balanced across Work Packages and links to the research.

 

Drafting your Theory of Change from the bottom-up:

  • Extract and thematically group the impact goals from the first column of the impact planning templates you have been sent

  • Identify long-term impacts from the themes that emerge. Some of the impacts that have been identified may already work as long-term goals, but more often your list will be dominated by shorter-term, more specific goals

  • Looking only at the list of impact goals, consider if any of the impacts lead to other impacts in the list, and start to create causal chains of shorter-term impacts that might lead to longer-term impacts e.g. short-term awareness raising and capacity building impacts may lead to subsequent policy, environmental or economic impacts over longer periods

  • Look at the activities listed in your impact planning templates to create causal chains that lead all the way back to your research. Keep asking what you would expect to see happen immediately prior to each impact or activity until you reach your research (this is sometimes referred to as “backwards mapping”)

 

Re-drafting your Theory of Change from the top-down:

  • Make sure you have a clear, over-arching integrative purpose for your project that sums up your high-level project impacts in a single, powerful sentence

  • Consider if there are any obvious long-term impact goals that are missing. Ask yourself why these are missing, discussing with partners as necessary, in case these were missed out deliberately for good reasons. Alternatively, omissions at this level may represent a bias in your team that may need correcting by inviting new members to represent missing disciplines or partners. It may simply be that a key organization failed to return their impact planning template or misunderstood what was being asked of them

  • Consider how robust and complete your causal chains are, identifying and interrogating assumptions and coming up with additional or improved activities that could create more convincing pathways to impact

  • Check the balance of impacts and activities across Work Packages, themes, countries etc, adding activities and impacts to correct major biases as necessary

 

A Theory of Change for a research project will differ from those you may find online for development projects because it will start with research. Theories of Change from the development world tend to use the terminology of inputs, outputs, outcomes, and impacts, which many researchers find confusing. This is because some outputs and outcomes can be considered impacts if impact is defined as a non-academic benefit arising from research (for more on definitions of impact, see here). In keeping with terminology more familiar to researchers, a Theory of Change for a research project may visualise causal links between:

 

Research > Pathways to impact > short-term impacts > long-term impacts

 

To see what this looks like, see the example below led by Prof Reed for a recent GCRF application (click the graphic to download as a PowerPoint slide):

Screen Shot 2018-09-13 at 19.01.46.png

See how this Theory of Change was used in the context of the wider impact sections of this proposal, or visit our Pathway to Impact Best Practice Library for other examples of GCRF proposals.

There is no one way of visualising a Theory of Change. Here are some examples of Theory of Change diagrams for other research projects (thanks to Ed Rowe from CEH Bangor for bottom figure):

Everything you need to know about the final REF2021 guidance on impact in less than a minute

August 29, 2019

1/15
Please reload

Everything you need to know about the final REF2021 guidance on impact in less than a minute

August 29, 2019

1/15
Please reload

Find us on

Facebook

Find us on 

Twitter 

© Fast Track Impact Ltd. 2017. All rights reserved.

Company number SC520657

 

Anti-spam  |  Privacy

Subscribe to our mailing list