Publications

Un modèle est une représentation verbale, graphique ou mathématique de relations sociales ou économiques. Les modèles fournissent un cadre simplifié en privilégiant les liens essentiels et en ignorant certains facteurs considérés comme moins importants. Les modèles mathématiques décrivent les relations sociales et économiques sous forme de notation algébrique. Les plus simples sont les modèles à équation unique.

Un modelo es una representación verbal, gráfica o matemática de las relaciones sociales o económicas. Los modelos proporcionan un marco simplificado ya que se centran en las relaciones de interés fundamentales y omiten los factores que se consideran más marginales. Los modelos matemáticos describen las relaciones sociales y económicas en una notación algebraica. Los modelos más simples son modelos con una sola ecuación. Por ejemplo, la función de producción de salud en la economía expresa un resultado directo en materia de salud, por ejemplo, la tasa de mortalidad infantil en función de los ingresos, la educación femenina, la inmunización, los aportes nutricionales, etcétera.

Sharing of good, practical research practices and lessons learned from development and humanitarian contexts is in high demand not only within UNICEF, but also in the broader international development and humanitarian community, ‘Impact Evaluation in the Field’ complements other methodological briefs by discussing how textbook approaches are applied in often challenging, under-resourced development contexts as well as the innovative solutions that are needed to ensure that practical demands do not compromise methodological rigour. The series will grow over time, allowing UNICEF staff and partners to share new experiences and approaches as they emerge from applied research. The overarching aim is to contribute to strengthening capacity in research and evaluation, improving UNICEF and partners’ ability to provide evidence-based, strategic, long-term solutions for children. This brief documents the impact evaluation design of the Ghana Livelihood Empowerment against Poverty (LEAP) 1000 programme which is being piloted in ten districts in two regions and targets about 6,000 households initially.

Sharing of good, practical research practices and lessons learned from development and humanitarian contexts is in high demand not only within UNICEF, but also in the broader international development and humanitarian community, ‘Impact Evaluation in the Field’ complements other methodological briefs by discussing how textbook approaches are applied in often challenging, under-resourced development contexts as well as the innovative solutions that are needed to ensure that practical demands do not compromise methodological rigour. The series will grow over time, allowing UNICEF staff and partners to share new experiences and approaches as they emerge from applied research. The overarching aim is to contribute to strengthening capacity in research and evaluation, improving UNICEF and partners’ ability to provide evidence-based, strategic, long-term solutions for children. This methodological brief focuses on the qualitative component of the evaluation of the Ghana Livelihood Empowerment against Poverty (LEAP) 1000. Quantitative measures will indicate if LEAP 1000 reduces child poverty, stunting and other measures of well-being, while qualitative research explores in more depth the reasons why and how this may or may not be happening.

The Best of UNICEF competition identifies a number of studies that are assessed to be of particular merit on a number of criteria: in terms of the relevance and interest of the topic and findings; the rigour of their methodology; and the potential for impact, including lessons that could inform programmes elsewhere, or the capacity for replication or scaling up. Issues covered include health, education, WASH, child protection and social inclusion. There was also a strong emphasis on qualitative and mixed methods research, demonstrating the value of rigorous qualitative studies. A number of studies selected as of special merit in 2016 involved research directly with children and there is an increasing recognition that children’s perspectives are of primary importance. There was also a welcome attention to gender in some of the studies, including research with both adolescent boys and girls.

Impact evaluation provides information about the impacts produced by an intervention. It can be undertaken of a programme or a policy, or upstream work – such as capacity building, policy advocacy and support for an enabling environment. This goes beyond looking only at goals and objectives to also examine unintended impacts. This brief provides an overview of the different elements of impact evaluation and options for planning and managing its various stages.

A theory of change explains how activities are understood to produce a series of results that contribute to achieving the final intended impacts. It can be developed for any level of intervention – an event, a project, a programme, a policy, a strategy or an organization. In an impact evaluation, a theory of change is useful for identifying the data that need to be collected and how they should be analysed. It can also provide a framework for reporting.

Evaluation relies on a combination of facts and values to judge the merit of an intervention. Evaluative criteria specify the values that will be used in an evaluation. While evaluative criteria can be used in different types of evaluations, this brief specifically addresses their use in impact evaluations.

Decision makers frequently need evaluation to help them work out what to do to build on strengths and address weaknesses. To do so, they must know not only what the strengths and weaknesses are, but also which are the most important or serious, and how well or poorly the programme or policy is performing on them. Evaluative reasoning is the process of synthesizing the answers to lower- and mid-level evaluation questions into defensible judgements that directly answer the key evaluation questions.

Using participatory approaches in impact evaluation means involving stakeholders, particularly the participants in a programme or those affected by a given policy, in specific aspects of the evaluation process. The term covers a wide range of different types of participation and stakeholders can be involved at any stage of the impact evaluation process, including: its design, data collection, analysis, reporting and managing the study.

One of the essential elements of an impact evaluation is that it not only measures or describes changes that have occurred but also seeks to understand the role of particular interventions (i.e., programmes or policies) in producing these changes. This process is known as causal attribution. In impact evaluation, there are three broad strategies for causal attribution: 1) estimating the counterfactual; 2) checking the consistency of evidence for the causal relationships made explicit in the theory of change; and 3) ruling out alternative explanations, through a logical, evidence-based process. The ‘best fit’ strategy for causal attribution depends on the evaluation context as well as what is being evaluated.

A randomized controlled trial (RCT) is an experimental form of impact evaluation in which the population receiving the programme or policy intervention is chosen at random from the eligible population, and a control group is also chosen at random from the same eligible population. It tests the extent to which specific, planned impacts are being achieved. The distinguishing feature of an RCT is the random assignment of units (e.g. people, schools, villages, etc.) to the intervention or control groups. One of its strengths is that it provides a very powerful response to questions of causality, helping evaluators and programme implementers to know that what is being achieved is as a result of the intervention and not anything else.