I hosted a “Birds of Feather” session at the UK Evaluation Society’s annual conference – 26th May 2022. The question of interest was “How Can Evaluation develop a ‘voice’ on national and global challenges?” and in this blog I summarise main thoughts gathered.
A barrier for our profession is the use of jargon, which gets in the way – this was also highlighted by the closing keynote speaker Professor Ruth Boaden. Evaluation is still not very well known or understood and can feel technocratic. So, part of finding a place and role for evaluation is about our use of language. In my experience, using words like learning, insights or evidence can go a long way in communicating effectively and finding a role for evaluation.
A second barrier we identified was the use of evaluation. Evaluation is still commonly used at the end of programmes, and often findings are not available until decisions for the next cycle have been made. For evaluation to have a voice on national or global challenges, it needs to come to the forefront: when strategies are being designed and developed, when programme decisions are being considered such as replicate, scale up or wind down.
Institutional knowledge and learning play a big role here, as often evaluation findings need to be surfaced beyond a programme cycle and need to survive attrition rates – in other words, someone needs to know where evidence and learning is and bring it to the table at the design and decision-making stage. Ideally you would like to see a leader in charge of a new strategy ask “what have we learned before from similar efforts? Or do we have any evidence that any of the options we are considering might be effective?” which creates demand for evidence and its use at the right time.
But we also identified a third and more complex barrier: what is the question at hand and what role can evaluation play? We specifically discussed the current conflict in Ukraine and realised that how we frame the question matters. If the question is about evidence and what we have learned from conflict stabilisation and peace building, then evaluation findings might have a lot to say. But often the big question is more political in nature, such as: what role, if any, should the UK play in the conflict between Ukraine and Russia? Political values and ideals might be the core criteria for decision-making in that scenario.
Finally, evaluation also faces challenges related to what is palatable to people and society at large. For example, one of the attendees reminded us that criminal justice evaluations have found robust evidence that long-term sentences are not very effective and have suggested to reduce overall prison time. These findings were brought to the foreground by a prominent public figure and went down in fire: it did not feel palatable from a political or public perspective.
So, what have we learned from this session and what tips can we share with you to ensure that evaluation plays a role in national or global debate?
- Do not use jargon: find other ways to refer to evaluation that better resonate with your target audience.
- Questions matter and can tell us whether evaluation can have a voice.
- Evaluation is most useful when it plays a role in strategy design and development – good knowledge management and institutional learning is key.
- Evaluation does not exist in a vacuum; uptake of findings is mediated by the context and other political and societal values and criteria. There is only so much evaluation and do on its own, being clear about it can go a long way in understanding the impact evaluation can have in national and/or global debate
Nexus Evaluation LTD 