Key Resources: Gender, Monitoring, Evaluation and Learning

Monitoring, evaluation and learning
Indicators for use in monitoring, evaluation and learning
Global gender indices
Standards and guidelines

Monitoring, evaluation and learning

Capturing Change in Women’s Realities: A Critical Overview of Current Monitoring & Evaluation Frameworks and Approaches (2010), Srilatha Batliwala and Alexandra Pittman, Toronto: Association for Women’s Rights in Development, www.awid.org/About-AWID/AWID-News/Capturing-Change-in-Women-s-Realities (last accessed April 2014), 43 pp.

In this useful paper, published by the Association for Women’s Rights in Development, the authors critically engage with some of the common challenges and ‘politics’ associated with monitoring and evaluation (M&E) – for example, the difficulty in assessing how change happens and how gender relations have been altered; and the perception that measurement is used more as a tool of enforcement and accountability to the donor than as a means of understanding and learning what works. They identify feminist practices for engaging in M&E in order to strengthen learning and more readily capture the complex changes sought by projects aiming to support women’s empowerment and gender equality. Part II of the paper offers an analysis of a large number of M&E frameworks and tools, along with some of their strengths and weaknesses in assessing women’s rights and gender equality processes and impacts.

Strengthening Monitoring and Evaluation for Women’s Rights: Twelve Insights for Donors (2011), Srilatha Batliwala, Toronto: Association for Women’s Rights in Development, www.awid.org/Library/Strengthening-Monitoring-and-Evaluation-for-Women-s-Rights-Twelve-Insights-for-Donors (last accessed April 2014), 11 pp.

Strengthening Monitoring and Evaluation for Women’s Rights: Thirteen Insights for Women’s Organizations (2011), Srilatha  Batliwala,  Toronto:  Association  for  Women’s  Rights in Development, www.awid.org/Library/Strengthening-Monitoring-and-Evaluation-for- Women-s-Rights-Thirteen-Insights-for-Women-s-Organizations (last accessed April 2014), 16 pp.

A result of the Association for Women’s Rights in Development’s action research undertaken during 2009 and 2010 to study the challenges faced by women’s organisations and their donors in effectively monitoring and evaluating women’s rights work, these two documents present short discussions around each ‘insight’. The discussions serve as both valuable analyses of the challenges presented around women’s rights work and monitoring and evaluation (M&E), and as a set of recommendations for working on M&E in the future. Insight number one from the paper addressing women’s organisations is ‘Make M&E a key ingredient in our learning and accountability’, and Insight number one for donors is ‘Make M&E a learning partnership, not a performance test!’

Feminist Evaluation and Research: Theory and Practice (2014) Sharon Brisolara, Denise Seigart, and Saumitra SenGupta, New York and London: The Guildford Press. ISBN: 978-1462515202, website: www.guilford.com

This excellent edited collection – which is rigorous and comprehensive but accessibly and engagingly written – is divided into three sections. The first, Feminist theory, research and evaluation, provides an overview and introduction to the topic, with the following sections, Feminist evaluation in practice, and Feminist research in practice giving practical examples.

Gender and Monitoring: A Review of Practical Experiences. Paper Prepared for the Swiss Agency for Development and Co-operation (2001), Paola Brambilla, Brighton:  BRIDGE, Institute of Development Studies, www.bridge.ids.ac.uk/go/home&id=52836&type= Document&langID=1  (last accessed  April  2014),  25  pp.

Aiming to be a practical tool that can be used to integrate a gender approach into existing monitoring and evaluation (M&E) mechanisms, this paper first defines M&E, goes on to look at how indicators can be made gender-sensitive, who should be involved in this process, and when they should be used during the project cycle. The paper includes case studies of implementation of gender monitoring at different levels and the following recommendations are made: indicators must be both qualitative and quantitative and take account of contextual factors; there is a need for participation of  women and men  in the target group  in M&E  processes; and gender- disaggregated indicators are necessary, but not sufficient. They must be complemented by qualitative analysis and baseline data in order to track changes of gender relations.

Measuring Women’s Empowerment and Social Transformation in the Post-2015 Agenda (2014), Caroline Harper, Keiko Nowacka, Hanna Alder, and Gaëlle Ferrant, London: Overseas Development Institute, www.odi.org.uk/sites/odi.org.uk/files/odi-assets/pub lications-opinion-files/8838.pdf (last accessed April 2014), 8 pp.

Arguing that gender equality should be central to the post-2015 development framework, and that progress on data collection in recent years has made the capture of social norms increasingly accessible, affordable, and regular, the authors of this report outline a set of transformative indicators under six key measurement areas that, taken together, can track the changes in social norms that indicate the increasing empowerment of women and girls. The authors propose a set of indicators in six measurement areas. These are: women and girls exercise choice over their sexual and reproductive integrity; women and girls enjoy freedom from violence; women and girls enjoy enhanced decision-making ability over land and assets; women attain enhanced participation in political and civic life; equal value is given to girls and boys; and unpaid care is equally distributed between women and men, and girls and boys.

The Conditions and Consequences of Choice: Reflections on the Measurement of Women’s Empowerment, UNRISD Discussion Paper No. 108 (1999), Naila Kabeer, Geneva: United Nations Research Institute for Social Development, www.unrisd.org/80256B3C0 05BCCF9/(httpAuxPages)/31EEF181BEC398A380256B67005B720A/$file/dp108.pdf (last accessed April 2014), 58 pp.

This paper is a very thoughtful and thorough  consideration of the  challenges involved in constructing indicators of women’s empowerment, focusing in particular on the meanings given to these measures, and the values embedded within them – both the values of those whose lives are being assessed, and the values of those who are doing the measuring. It starts out by suggesting a three-dimensional conceptual framework for thinking about women’s empower- ment: ‘resources’ as part of the preconditions of empowerment; ‘agency’ as an aspect of process; and ‘achievements’ as a measure of outcomes. It goes on to consider the ways in which these different dimensions have been measured by economists, demographers, sociologists, and feminists. A number of key methodological points are made, in particular, the need for the cross-checking of measures, to ensure that indicators really do mean what they are intended to mean.

Measuring Empowerment: Cross-disciplinary Perspectives (2005), Deepa Narayan, Washington, DC: World Bank, https://openknowledge.worldbank.org/bitstream/handle/10 986/7441/344100PAPER0Me101Official0use0only1.pdf?sequence=1 (last accessed April 2014)

This thorough and thoughtful work considers empowerment in a range of contexts. Section Two, focusing on Gender and Household, consists of three essays: Women’s Empowerment as a Variable in International Development; Measuring Women’s Empowerment – Learning from Cross-national Research; and Gender, Power, and Empowerment: An Analysis of Household and Family Dynamics. Overall, the section stresses three main points. Firstly, the household is not neutral, but instead a site of unequal formal and informal rules and social norms that result in unequal power relations between women and men. Secondly, the socio-cultural context matters, and can be more important than individual traits in determining women’s empowerment, and thirdly, psychological or mental space plays a key role at individual, and community level. These points help explain the author’s common emphasis on women’s empowerment through collective action.

Review of Evaluation Approaches and Methods Used by Interventions on Women and Girls’ Empowerment (2014), Georgia Taylor and Paola Pereznieto, London: Overseas Devel- opment Institute, www.odi.org.uk/sites/odi.org.uk/files/odi-assets/publications-opinion- files/8843.pdf (last accessed April 2014), 50 pp.

This clearly set-out review assesses the quality and effectiveness of evaluation methods and approaches used to analyse the effects of programmes to promote women and girls’ economic empowerment. The review analysed evaluations that assessed some measure of women and girls’ economic empowerment in one or more of the following eight thematic areas: Financial services; Business development services; Skills training; Asset provision (both financial and physical); Social protection; Unions and fair employment; Trade and access to markets; Regulatory and legal frameworks. Key findings included: mixed-methods (quantative and qualitative) evaluations were more effective in capturing changes in norms, attitudes, and behaviours associated with women’s and girls’ economic empowerment; data and analysis in evaluations are not generally disaggregated by age or life-cycle stages; and, in order to guide the evaluation, it is necessary to undertake a rigorous context and gender analysis and to have a Theory of Change.

Understanding and Measuring Women’s Economic Empowerment: Definition, Framework and Indicators (2011), Anne Marie Golla, Anju Malhotra, Priya Nanda, and Rekha Mehra, Washington, DC: International Center for Research on Women, www.icrw.org/publica tions/understanding-and-measuring-womens-economic-empowerment (last accessed April 2014), 12 pp.

This very useful short report lays out fundamental concepts, including a definition of women’s economic empowerment; a measurement framework that can guide the design, implementation, and evaluation of programmes aiming to empower women economically; and a set of indicators that can serve as concrete examples for developing meaningful metrics for success.

Gender Issues in Monitoring and Evaluation. Module 16 in the Gender in Agriculture Sourcebook (2009), Washington, DC: World Bank, http://siteresources.worldbank.org/INTGENAGRLIVSOUBOOK/Resources/Module16.pdf (last accessed April 2014), 142 pp.

This Module from the Gender in Agriculture Sourcebook is a particularly well-thought-out and well-designed guide to gender and monitoring, evaluation, and learning, and while concentrat- ing on agriculture, much of the information it contains can be applied across all development sectors. The module aims to address gender concerns in designing agricultural and rural development projects and to provide ideas – indicators, principles, approaches, and practical options – for improving monitoring and evaluation of outcomes and impacts.

Gender Issues in Monitoring and Evaluation in Rural Development: A Tool Kit (2005), Washington, DC: World Bank, http://siteresources.worldbank.org/INTGENDER/Resources RuralM_EToolkit2005.pdf (last accessed April 2014), 24 pp.

Aimed at World Bank staff and others involved in World Bank projects, this document is a ‘how to guide’, walking the user through the process of creating gender-sensitive monitoring and evaluation and the integration of a participatory approach. It then provides examples from a number of rural sub-sectors (e.g. sustainable agriculture and natural resource management; and agro-enterprise development) of relevant checklists for gender-related issues and activities during the project cycle, and a framework for results and results monitoring.

‘Measuring gender equality in education’ (2005), Elaine Unterhalter, Chloe Challender, and Rajee Rajagopalan, in Sheila Aikman and Elaine Unterhalter (eds.) Beyond Access: Transforming Policy and Practice for Gender Equality in Education, Oxford: Oxfam GB, pp. 60–79. The book is available to download at no cost at http://policy-practice.oxfam.org. uk/publications/beyond-access-transforming-policy-and-practice-for-gender-equality-in- education-115410 (last accessed April 2014)

While the statistics presented here are inevitably somewhat out of date, having been published in 2005, this chapter nevertheless provides a valuable discussion of measuring gender equality in education, first critically reviewing measures of gender equality in education used by international agencies and governments, then outlining the development of an alternative measure, the Gender Equality in Education Index (GEII). The GEII aimed to include not only the number of girls attending and remaining in primary school, but also whether these girls were able to build on this, in terms of future secondary schooling, leading healthy lives, and making a reasonable living.

A Guide to Gender-analysis Frameworks (1999), Candida March, Ines Smyth, and Maitrayee Mukhopadhyay, Oxford: Oxfam GB,  http://policy-practice.oxfam.org.uk/ publications/a-guide-to-gender-analysis-frameworks-115397 (last accessed April 2014), 144 pp.

A variety of frameworks to analyse gender relations are used in development work. They can be helpful tools in planning gender-sensitive research projects, or in designing development interventions which address gender inequalities. Drawing on the experience of trainers and practitioners, this book contains step-by-step instructions for using different gender-analysis frameworks, and summaries of their advantages and disadvantages in particular situations. An introductory section explains the importance of gender analysis, and the role of the frameworks in development initiatives and research.
Experiments in Knowing: Gender and Method in the Social Sciences (2000), Ann Oakley, Cambridge and Malden, MA: Polity, ISBN: 97807456 22576, website: www.polity.co.uk

Exploring the history, ideology, and implications of methodology in the social and natural sciences, the author of this influential book argues that these disciplines have been subject to a process of ‘gendering’, which has produced an ideological reaction against, rather than a relevant understanding of, the role of ‘quantitative’ and experimental methods. For her, the rejection of ‘quantitative’ ways of knowing, in particular, prevents an understanding of both the parameters of social inequality and the effects of interventions in people’s lives. As a methodological position adopted by feminists, postmodernists, and others, it obstructs the development of a critical and emancipatory social science.

Indicators for use in monitoring, evaluation, and learning

Oxfam Quick Guide to Gender-sensitive Indicators, http://policy-practice.oxfam.org.uk/ publications/quick-guide-to-gender-sensitive-indicators-312420 (last accessed April 2014), 3 pp.

Stating that ‘using gender-sensitive indicators can help us to understand how changes in gender relations happen which enables more effective planning and delivery of future work’, this extremely helpful and accessible short guide explains the different kinds of indicators used in different stages of a project cycle, sets out what is meant by ‘gender-sensitive’ indicators, and briefly describes how to develop gender-sensitive indicators for a project or programme.

Gender and Indicators: BRIDGE Cutting Edge Pack (2007), Brighton: Institute of Development Studies, www.bridge.ids.ac.uk/go/bridge-publications/cutting-edge-packs/ gender-and-indicators/gender-and-indicators&langid=1  (last  accessed  April 2014)

Stating that ‘gender-sensitive indicators and other measurements of change are critical for building the case for taking gender (in)equality seriously, for enabling better planning and actions, and for holding institutions accountable for their commitments on gender’, this Cutting Edge Pack from BRIDGE contains an Overview Report, a Supporting Resources Collection, and a short briefing paper. The Overview Report defines gender-sensitive indicators and measurements of change, outlines their use, and follows with sections focusing on measuring in particular areas, such as gender-based violence, and the gender dimensions of poverty. It goes on to assess some international measures and indices, and concludes with a set of recommendations. The Supporting Resources is a wide-ranging collection of summaries and case studies of writings, tools, and initiatives relating to gender and indicators plus a list of organisations working on gender and indicators, as of 2007. The short briefing paper gives a succinct overview of gender and measuring change, followed by two case studies, one from the project level, the other from the international level.

Gender Indicators: What, Why and How? (n.d.), Justina Demetriades, prepared for the OECDDAC Network on Gender Equality, based on BRIDGE’s Gender and Indicators Cutting Edge Pack 2007, www.oecd.org/social/gender-development/43041409.pdf (last accessed April 2014), 10 pp.

This short briefing paper focuses on the use of gender indicators as a way of measuring change, outlining what indicators are, and why we should develop gender indicators. It also addresses the often political issue of what we should be measuring, providing some broad principles that can be considered in making these decisions, as well as some questions donors can ask themselves when they are developing gender indicators. The document also offers examples of existing indicators, noting that they always need to be adapted to specific contexts.

Global gender indices

Gender Inequality Index, http://hdr.undp.org/en/statistics/gii

The Gender Inequality Index (GII) has been developed by the United Nations Development Programme (UNDP) as part of its work on monitoring human development (see the UNDP’s annual Human Development Reports), alongside the Human Development Index, the Inequality Adjusted Human Development Index, and the Multi-dimensional Poverty Index. All seek to provide a human-centred assessment of development, rather than relying on purely national economic data. The GII replaces UNDP’s previous gender indices, the Gender Development Index and the Gender Empowerment Index, and is a country-level measure of inequality between women and men in three areas; reproductive health, empowerment, and the labour market.

Gender Equality Index, http://eige.europa.eu/content/gender-equality-index

Created by the European Institute for Gender Equality, an independent agency of the European Union (EU), the Gender Equality Index was officially launched in 2013 and provides results for the EU overall, and for each Member State, giving a measure of how far (or close) each Member State was from achieving gender equality in 2010. Categories measured are work, money, knowledge, time, power, and health.

Global Gender Gap Index, World Economic Forum, www.weforum.org/issues/global- gender-gap
The Global Gender Gap Index was developed in 2006 and is published annually by the World Economic Forum (WEF). It calculates the national gender gaps of 136 countries on economic, political, and education- and health-based criteria. Unsurprisingly, the WEF makes an instrumentalist case for monitoring gender inequality, noting ‘the strong correlation between a country’s gender gap and its national competitiveness. Because women account for one-half of a country’s potential talent base, a nation’s competitiveness in the long term depends significantly on whether and how it educates and utilizes its women.

Social Institutions and Gender Index, ww.oecd.org/dev/poverty/theoecdsocialinstitution sandgenderindex.htm#results2012

The Social Institutions and Gender Index (SIGI) is a measure of gender equality developed in 2009, based on the Organisation for European Co-operation and Development’s Institutions and Development Database, and seeks to measure underlying discrimination against women in more than 100 countries. Rather than measuring gender inequality in areas such as education and employment, for example, the SIGI aims to quantify discriminatory social institutions, such as early marriage, discriminatory inheritance practices, violence against women, son bias, restric- tions on access to public space, and restricted access to productive resources.

To Measure is to Know? A Comparative Analysis of Gender Indices, ISS Working Paper No. 2011–02 (2011), Irene Van Staveren, The Hague: Institute of Social Studies, www. indsocdev.org/resources/ISD Working Paper 2011-02.pdf (last accessed April 2014), also available at Review of Social Economy 71(3): 339–72
In this paper, the author uses a human development, or Capabilities, approach to compare and contrast a set of five, cross-country gender indices; the Gender Equality Index (created by the Institute of Social Development), the Gender Inequality  Index (United Nations Development Programme), the Social Institutions and Gender Index (Organisation for European Co-operation and Development), the Global Gender Gap Index (World Economic Forum), and the Women’s Economic Opportunities Index (Economic Intelligence Unit). The author concludes that the indices are ‘clearly not interchangeable, and the selection of a particular gender index should be justified carefully to make its use in scholarly research and policy analysis meaningful’, and to this end she provides three ‘decision trees’ to help in the choice of a suitable index for use by readers.

Standards and guidelines

Oxfam Minimum Standards for Gender in Emergencies (2013), Oxford: Oxfam Interna- tional, http://policy-practice.oxfam.org.uk/publications/oxfam-minimum-standards- for-gender-in-emergencies-305867 (last accessed April 2014), 6 pp.

Developed in the first instance for Oxfam staff to ensure a consistent approach to promoting gender equality in Oxfam’s humanitarian programming, these standards constitute a valuable tool for humanitarian practitioners across the sector. Intended to be referred to throughout the project cycle to inform planning; programme design and implementation; and monitoring, evaluation, accountability, and learning, the document sets out 16 minimum standards, plus a set of key actions necessary for the meeting of these standards.

Guidelines for Gender-based Violence Interventions in Humanitarian Settings: Focusing on Prevention of and Response to Sexual Violence in Emergencies (2005), Geneva: Inter-Agency Standing Committee, https://ochanet.unocha.org/p/Documents/GBV Guidelines (English).pdf (last accessed April 2014) 90 pp.

Prepared by the Inter-Agency Standing Committee (IASC), which is the body that co-ordinates both United Nations (UN) agencies and non-UN partners in humanitarian responses around the world, this set of Guidelines (also available in Arabic, Bahasa, French, and Spanish) aims to enable communities, governments, and humanitarian organisations to establish and implement a set of minimum interventions to prevent and respond to sexual violence during the early phase of an emergency. The multi-sectoral Action Sheets contained within the Guidelines provide a tool for implementing interventions in a range of sectors, including: Water and sanitation; Food security and nutrition; and Shelter and site planning and non-food items.

Ethical and Safety Recommendations for Researching, Documenting and Monitoring Sexual Violence in Emergencies (2007), Geneva: World Health Organization, www.who.int/ gender/documents/OMS_Ethics&Safety10Aug07.pdf (last accessed April 2014), 33 pp.

The introductory section to this document states, ‘In some emergency settings, simply participating in sexual violence inquiries can have serious, even life-threatening implications, not only for the participants themselves, but for the community and those involved in collecting information’. Rather than constituting a ‘how to’ methodological tool for conducting information collection into sexual violence, these recommendations focus on the ethical considerations involved. After a section on key concepts and definitions, the recommendations themselves are organised into eight thematic areas: Risks and benefits; Methodology; Referral services; Safety; Confidentiality; Informed consent; Information gathering team; and Children. An annex provides a useful list of recommended resources and further reading.