Between July and November 2023, I was part of an international group convened by the School of Systems Change. It was a group inquiry on Monitoring, Evaluation and Learning (from now on referred to as “evaluation”) in systems change, with all of us co-facilitating the sessions.
My contribution was to use a systems change approach to better understand evaluation systems: its patterns and how structures, mental models and organisational cultures/capabilities influence how evaluation is practiced and its impact. Three main issues emerged for me as follows:
- The lack of equity, diversity and inclusiveness (EDI) in evaluation
- The need to decolonise evaluation and learning while expanding its current focus on indigenous people and international development.
- And the culture and leadership gap
All of the above are interconnected. I unpack each issue below and hope to share thought provoking ideas that could transform evaluation systems. By doing so, I believe we can ensure evaluation adds real value to decision-making, learning, adaptation and system change efforts.
A) The lack of equity, diversity and inclusiveness (EDI) in evaluation
Systems change leads one to recognise indigenous knowledge as a source of holistic, relational and systemic practice[1]. And once in this space, it is hard to ignore the Euro-Western supremacy over knowledge and consequently evaluation practice[2]. Current structures, behaviours and mindsets have led to institutional and systemic racism and discrimination of non-white people and other minority groups[3]. Yet there is little recognition and action about these issues and the institutions that could otherwise be part of its transformation. This includes professional bodies like evaluation societies and the most reputable evaluation journals worldwide.
Take for example the Evaluation journal – an international journal whose mission is to promote exchange between European, North American, Asian and Australasian voices within the evaluation community. However, there is a remarkable absence of non-white, Asian or LGBTQ+ voices in its publications. When I analysed all 84 authors that published in this journal in 2022, I found that 93% of all of them are white, and that 67% of them were affiliated to a UK, USA/Canada or Australian-based organisation[4].
Eight out of the top 10 most read evaluation websites belong to white males, and eight of those websites are affiliated to US-based organisations[5]. But change is possible, take for example the “Decolonising Evaluation” issue by the Journal of Multidisciplinary Evaluation, where approx. 20% of authors are from a Black heritage or minority group and 73% are women – sadly none of them were from Global South organisations.
B) The need to decolonise evaluation and learning while expanding its current focus on indigenous people and international development.
Decolonising evaluation discourse needs to go beyond its current focus on indigenous peoples and international development[6]if it is to be localised and reflective of the minority groups found in the Global North. For example, in the UK, 9.3% of the total population are from Asian ethnic groups, followed by Black people with African and Caribbean ancestry[7]. And Roma are the largest ethnic minority in Europe[8].
Decolonising evaluation is about:
- Deconstructing the superiority and privilege of dominant groups, particularly Euro-Western knowledge systems and practice[9], and addressing its inherit biases
- Recognising and valuing diverse forms of knowledge and experiences, particularly from non-dominant groups
- And restructuring evaluation values, approaches and tools based on local conditions and diverse cultural nuances[10].
The UK and European Evaluation societies could help transform the sector by:
Roots and Relations link here. And here is the link to the rigour piece.
At the collective and individual level, we can be part of the transformation by asking ourselves, and encouraging others to reflect on, the following questions:
C) The culture and leadership gap
Transforming evaluation will go a long way but it is also important to address the cultural and leadership gap within organisations.
It is widely acknowledged that most organisations still lack a culture of learning and capabilities to recognise what good evidence looks like. This results in poor quality evidence and siloed, results-base management and accountability functions that do not support decision-making, learning nor adaptive management. All these contributes to a long-held perception that evaluation is not useful. Moreover, too often evaluators are asked to build this culture and practice, yet they often lack the seniority and power to do so, and I think it is time to ask more of our leaders and the organisational cultures they create and maintain.
We lack an overwhelming number of leaders who create demand for evidence and learning, even though they are best placed to design and implement organisational change plans, performance management models and bonus schemes that:
- Create and sustain the enabling environments for more equitable, inclusive and diverse work practices.
- Incentivise single, double or triple learning.
- Reward building on good practice and on scaling up promising solutions.
- Redefine reputational and financial risk to allow for more collaborative, contribution-focused and systems change practice to flourish – deemphasising success framed around an individual/team/organisation.
- Allocate adequate resources and time to engage in systems change evaluation.
- Invest in the professional development of evaluation staff to keep them up to date with the latest thinking and practice, or even ahead of the curve and innovating.
- Create safe spaces where innovation, and learning from failure, can happen without negative impact to individuals and their ability to operate effectively.
Please help share this more broadly if anything resonated with you, and reach out if you want to add to this conversation or if you would like to arrange a chat with me and you/your team to reflect and discuss any of the contents in this blog.
Finally, I would like to thank Niki Wood, Julian King, Thomas Aston and Alan Hudson for the candid conversations we had in the lead up to me writing this paper and for the time spent reviewing an earlier draft. Your generosity was much appreciated!
[1] See for example this short video by the School for Systems Change formally acknowledging this (go to 1m 25sec)
[2] Sources: “Special Issue: Decolonizing Evaluation: Towards a Fifth Paradigm” (2023); “Made in Africa Evaluation: Decolonising Evaluation in Africa” (2019), “Involuntary social experimentation: revisiting the case for a moratorium” (2023) and “Locus of power”(2023), which builds on a model by Nan Wehipeihana.
[3] Sources: 2016 Concluding Observations of the Committee on the Elimination of Racial Discrimination (CERD), the UN’s 2019 Report of the Working Group of Experts on People of African Descent; 2020 Human Rights Council Report and 2019 Equality and Human Rights Commission report.
[6] Content cited in reference no.2 has a strong focus on international development and indigenous knowledge/groups.
[7] Data from the UK government.
[8] Data from the European Commission.
[9] Lea Corsetti’s blog (2022).
[10] “Made in Africa Evaluation: Decolonising Evaluation in Africa” paper. Accessible here.
Nexus Evaluation LTD 