Logo UNICEF Innocenti
Office of Research-Innocenti
menu icon

Profiles

twitter
personal website
email

Related Innocenti Project(s):

Tia Palermo

Sr. Social Policy Specialist (Former title)

Tia Palermo is a Social Policy Specialist and oversees projects relating to social protection and equity in low- and middle-income countries focusing on cash transfers and impact evaluations of interventions to combat poverty and exclusion of children. As part of the Transfer Project, her work examines the ability of social policy to improve outcomes among children and adolescents, including those related to schooling, sexual behaviour, mental health, and violence. She joined UNICEF Office of Research – Innocenti in 2014 and has published extensively on topics related to social policy, adolescent well-being, transitions to adulthood, and violence and holds a Ph.D. in Public Policy.

Publications

What Works to Improve Outcomes for Children?: A rapid evidence assessment of cash plus programmes in Low- and Middle-Income Countries, informed by Social and Behaviour Change (SBC) strategies
Publication

What Works to Improve Outcomes for Children?: A rapid evidence assessment of cash plus programmes in Low- and Middle-Income Countries, informed by Social and Behaviour Change (SBC) strategies

Cash transfers are a type of social protection that help to reduce the effects of poverty. They are typically delivered in the form of cash or vouchers that beneficiaries can use for various expenses in the same way as earned income. While cash-transfer programmes have shown positive effects on important first-order outcomes, such as food security, household consumption and education, there are limits to the effectiveness of cash transfers alone in addressing all development needs. Many interventions have had inconsistent or no impact on other important well-being outcomes, such as child nutrition, early marriage, health-seeking behaviour, and sexual and reproductive health. In order to improve the effectiveness of social protection in addressing these multidimensional needs, cash transfers are modified to cash-plus interventions by combining them with additional elements, such as in-kind resources, behaviour change interventions and links to other social services. SBC components of cash-plus interventions aim to address the drivers of behaviours that affect children’s well-being. The aims of this rapid evidence assessment are: • Assess the effectiveness of cash transfers combined with social and behaviour change (SBC) components to improve outcomes for children • Identify which types of SBC are effective in improving outcomes • Identify the contextual factors that are necessary to successfully deliver cash-plus interventions with SBC components
Improving Children’s Health and Nutrition Outcomes in Ethiopia: Qualitative midline evaluation of the ISNP in Amhara
Publication

Improving Children’s Health and Nutrition Outcomes in Ethiopia: Qualitative midline evaluation of the ISNP in Amhara

Integrated social protection programmes are increasingly being pursued as more effective and efficient ways to improve children’s health and nutrition outcomes.
Improving Children’s Health and Nutrition Outcomes in Ethiopia: A qualitative mid-line evaluation of the Integrated Safety Net Programme in Amhara
Publication

Improving Children’s Health and Nutrition Outcomes in Ethiopia: A qualitative mid-line evaluation of the Integrated Safety Net Programme in Amhara

Integrated social protection programmes are increasingly being pursued as more effective and efficient ways to improve children’s health and nutrition outcomes.
Non-contributory Social Protection and Adolescents in Lower- and Middle-Income Countries: A review of government programming and impacts
Publication

Non-contributory Social Protection and Adolescents in Lower- and Middle-Income Countries: A review of government programming and impacts

Adolescents face unique vulnerabilities related to their health, schooling and the intensification of gender socialization. As the next generation next in line to become adults, their transition has major implications for the future health, economic growth and well-being of nations. Yet, children and adolescents have low rates of social protection coverage globally – a missed opportunity for investment. This report examines how social protection can promote adolescent well-being and facilitate safe and productive transitions to adulthood in lower- and middle-income countries. Focusing on government, non-contributory programmes, the following questions are examined: 1) whether and how current non-contributory social protection programmes are adolescent-sensitive and 2) what is the impact of non-contributory social protection programmes on adolescents.

Articles

Enhancing adolescents’ capabilities through adolescent- and gender-responsive social protection
Article

Enhancing adolescents’ capabilities through adolescent- and gender-responsive social protection

Blogs

Five things we learned from research on child survivors of violence
Blog

Five things we learned from research on child survivors of violence

Understanding prevalence, characteristics and motives of disclosure, help-seeking and reporting of violence against children  Violence against children is a pervasive global phenomenon. Estimates indicate over 1 billion children under the age of 18 experience emotional, physical or sexual violence every year from a range of perpetrators – including parents, peers and intimate or dating partners. Despite these high figures, official figures of VAC are just the tip of the iceberg.How much do we know about children’s disclosure, help-seeking and reporting of violence? Most studies in low- and middle-income countries have narrowly focused on either reporting intimate partner violence among adolescent girls, on specific types of violence, or in specific settings. A new publication, analyzing nationally representative Violence Against Children Survey data from six countries, aims to broaden the focus. The publication, just released in BMC Public Health, led by UNICEF Innocenti, in collaboration with other UNICEF offices and government counterparts, examines data from Cambodia, Haiti, Kenya, Malawi, Nigeria and Tanzania.The study had three objectives:Measure the prevalence of informal disclosure (to family and friends among others), formal help-seeking (from social, health and legal services), formal reporting, and receipt of formal help;Document characteristics associated with disclosure and health seeking;Understand reasons why children did not seek help. Using nationally representative data from six countries, the study analyzed reports from children aged 13 to 17 who experienced any physical and/or sexual violence in their lifetimes. These samples of child survivors of violence represented a high percentage of total children across countries: physical violence among children ranged from 50 to 84%, while that for sexual violence ranged from 6 to 36%. What did we learn in terms of disclosure and reporting? Most children have never told anyone about the violence they experience: Across countries the percentage of children who ever told someone informally about their experience was low—23% in Cambodia and 32% in Kenya, to 42% in Tanzania and 54% in Malawi. These estimates show that children may be telling survey interviewers about violence for the very first time and confirms that violence is largely under-acknowledged and “hidden in plain sight.” It also means that children’s existing social networks—including family, friends, and neighbors are often a first source of disclosure and possible support system for children survivors. Formal services are rarely accessed or utilized by child survivors: The percentage of children who reported to formal sources was low—ranging from under 1% in Cambodia to 25% in Tanzania (formal disclosure)—and the percentage who received help was even smaller (1% in Nigeria to 11% in Tanzania, this outcome was not measured in Cambodia or Haiti). These statistics confirm that only a fraction of children attempt to contact health, social or legal services and even fewer receive any support, highlighting the importance of expanding accessibility and reach of assistance. Factors encouraging disclosure, help-seeking and reporting varied by country: Identifying factors that encourage these behaviors could help target services or develop secondary prevention programming. However, few factors were consistently positively correlated with help-seeking behaviors—including factors that are hypothesized to help, like household wealth and residing in urban settings. This lack of pattern underscores the importance of context and the challenges in targeting services using observable characteristics of child survivors. Self-blame, apathy and not needing or wanting services were top factors deterring children from disclosure: Across countries, common reasons cited for not seeking help were responses like “I felt it was my fault (self blame)”, “I did not think it was a problem (apathy)” or “I don’t want or need services.” For example, in Cambodia, the most common reason for not seeking help for physical violence was self-blame, mentioned by 56% of children, while the most common reason in Kenya, Malawi and Nigeria was apathy (25%-39%). Fewer children reported fear of repercussions or helplessness, while lack of access and financial constraints were rarely mentioned. These reasons highlight the role of shame and how the normalization of VAC is pervasive. Better data and methodological innovation is urgently needed: This study underscores the need for innovation in research methodologies to accurately estimate prevalence of sensitive topics. Improvements might include methods allowing self-administration of questions and those which allow for greater confidentiality. In addition, future surveys should include a wider range of household and community level indicators to understand underlying dynamics surrounding the child’s environment—for example, parental (mental health, parenting, time use), household (social and economic vulnerability factors), and community (gender norms, service availability) characteristics. What should we take away from these results? One concrete implication is that statistics based on violence against children reporting to formal sources such as data from health systems, police, or NGO reporting are likely to underestimate the total prevalence ranging from 4 to 940-fold depending on the country. This has implications for the analysis of such data during COVID-19, where there have been fears that children are even less able to access services—and that many cases of violence are uncounted. Our results show this is a huge issue and must be accounted for when interpreting the dynamics stemming from administrative data.Other implications relate to how to improve use of services for survivors. One strategy is to address barriers including social norms that normalize violence, and how these manifest in different contexts. Another is to improve linkages and raise awareness of child protection services and common touchpoints  for children, such as those within the education, health and community-based structures, which are likely to interact with children on an informal, daily basis. These trusted individuals in children’s lives are important entry points for formal services.It is also important to strengthen the capacities of professionals working in health, education and social sectors to be able to identify risks and respond using a survivor centered approach. Given the wide under-reporting and pervasive nature of violence against children, services which are targeted to only one setting or population are unlikely to result in broad uptake of services and assistance. Multi-sectoral responses and well-networked referral systems are necessary.Much more research is needed to unpack the dynamics around help-seeking and secondary prevention for survivors. We hope this analysis will serve as a starting point to advance research and practice to end violence against children and the long-lasting negative effects experienced by children over their lifetimes.***Special thanks to Alessandra Guedes, Alina Potts and Mary Shawa for helpful comments.Amber Peterman, Ph.D. joined UNICEF Office of Research – Innocenti in 2015 as a Social Policy Specialist and now works as a consultant with joint affiliation as an Associate Adjunct Professor at UNC Chapel Hill. Amber focuses on gender, violence and adolescent wellbeing and safe transitions to adulthood with the Transfer Project evaluations of social protection and cash transfers in Africa.Audrey Pereira is a Doctoral Student in Public Policy at The University of North Carolina at Chapel Hill.Tia Palermo is Associate Professor of Epidemiology and Environmental Health at the University at Buffalo (State University of New York) and an Affiliated Researcher with the Transfer Project. Full citation: Pereira A, Peterman A, Neijhoft AN, Buluma R, Kaloga IF, Harvey R, Islam A, Kheam T, Kitembe M, Lund-Henriksen B, Maksud N, Maternowska MC, Potts A, Rottanak C, Shawa M, T Palermo (2020). Disclosure, reporting and help-seeking among child survivors of violence: A cross-country analysis. BMC Public Health 20(1051).
Mind the gender gap: How can a gender-norm lens improve social protection outcomes for adolescents?
Blog

Mind the gender gap: How can a gender-norm lens improve social protection outcomes for adolescents?

Since adolescence is a highly vulnerable period of rapid physiological, biological, and psychological change, researchers and development partners are increasingly asking how social protection can facilitate safer transitions to adulthood, and what additional factors shape these transitions for youth.Vulnerabilities related to adverse outcomes in adolescence are often shaped by gender norms, which can constrain the opportunities available to adolescent girls and jeopardize their health. Our research looks at how social protection programs have the potential to transform the lives of participants if they address these vulnerabilities and structural barriers.Looking at genderOne type of structural barrier is systematic exclusion from services or opportunities due to social class, gender or caste. Discriminatory gender and social norms, can also act as structural drivers of vulnerability among girls, as they perpetuate harmful socio-cultural practices, such as early marriage and gender-based violence. In order to have transformative effects, as they relate to gender norms, social protection would need to have impacts, which promote more equitable gender roles and relations.Social protection definedSocial protection broadly encompasses the sets of programs and policies that aim to reduce poverty, exclusion and vulnerability. Social protection includes, but isn't limited to cash transfers (child or disability grants, pensions, etc.), in-kind transfers of food and other items, waivers for schooling or health-related fees, and insurance schemes, which typically play a protective or preventive role, by either responding to adversity or shocks experienced by poor households (protective), or aiming to prevent future harm by bolstering households’ ability to cope with future shocks such as loss of income or unexpected flooding (preventive).Transitions to adulthood“Safe” transitions can be defined as freedom from violence and hazardous labor, having access to schooling and health services, experiencing positive mental health, and delaying pregnancy and marriage, among other positive outcomes.What is the existing evidence on how social protection, and cash transfers in particular, are helping to change gender norms, as they relate to adolescents? This was among the questions asked when experts convened in London on September 10, 2018, at an event organized by the Overseas Development Institute, UCL Institute of the Americas, and Gender & Adolescence Global Evidence (GAGE) consortium, and the ALIGN project. Other questions that participants grappled with included:How can a ‘gender-norms lens’ be integrated in the existing social protection policy and programming?Can a gender norms lens help advance a gender responsive social protection agenda?Is social protection really the best mechanism to address social and gender norms?Is it cost effective to influence gender norms through social protection?What are the trade-offs of addressing (or not) gender norms through social protection programming?As researchers based at the UNICEF Office of Research - Innocenti working on the Transfer Project  — a joint collaboration between UNICEF, FAO and the University of North Carolina, focused on generating evidence on social protection and facilitating its uptake — the authors of this blog participated in this event. Here are some of our own reflections on discussions generated during this event and our thoughts for moving the agenda forward.Social protection has the potential to transform gender norms and relations through the following pathways:decreasing gender inequalities in schooling attainment;promoting positive attitudes around how girls are valued by their families and communities;promoting women’s financial inclusion (that is, registering a bank account in women’s name to enable her to accumulate savings and grow a business);expanding women’s social support, economic networks, and participation in the community;reducing violence in the home, which generates a cycle of violence as children are exposed and replicate that behavior in adulthood;promoting more equitable distribution of domestic work between women and men.How are social protection programmes targeting or inclusive of adolescence? from UNICEF Office of Research - InnocentiNevertheless, cash transfer programs are not necessarily gender transformative, and they may reproduce discriminatory gender norms and practices, aggravating inequalities between the sexes. For instance, the conditions or “co-responsibilites” that female care providers are expected to fulfill to receive benefits can reinforce gender stereotypes around women’s sole responsibility for caretaking, ignoring their economic responsibilities, and cutting into their available time, and increasing girls’ work burdens, who tend to substitute the work of their mothers.Adolescents are rarely the primary focus of government-run social protection programs, but such programs can provide opportunities to leverage impacts for adolescents. Many social protection programs are aimed at investing in early childhood development, and “breaking the inter-generational cycle of poverty.” Numerous programs target large numbers of poor households with adolescents living in them, and adolescents are key to breaking this inter-generational cycle as they transition to adulthood. This creates an opening to boost impacts of social protection programs for adolescents, by mainstreaming adolescent lens into policy and programming, and providing complementary services, targeted to adolescents, to improve their health, skills, and knowledge.Programs that focus on attitudes and empowerment of individual girls without addressing discriminatory attitudes and practices in the larger community or broader structural barriers, are unlikely to have transformative effects. Many or most of those girls will continue to live in the same communities that limit their opportunities in the first place. They may continue to face limited access to schooling, employment, or financial inclusion, and pressure to marry early.Proposed strategies need to be practical, feasible, and matched to government priorities and institutional capacities and resources. Social norm change interventions are resource intensive and time consuming. Researchers and practitioners cannot be over-ambitious in terms of what social protection (on its own) can achieve. Further, strategies need to be supported by broader socio-economic and legislative policy frameworks.Finally, strategic decisions need to be informed by policy analysis and evidence. On both the research and program sides, a combination of concrete actions can be adopted to push this agenda forward:Adopt a long-term vision and a sequenced approach to programming: This may require starting from easier issues and progressively moving towards more complex normative goals.Undertake formative research to understand how social and gender norms affect adolescent behaviors and outcomes and then re-adjust program objectives accordingly. Existing design features can be tweaked to achieve transformative objectives (for example, larger transfer size for adolescents to combat increasing opportunity cost of schooling over work, adolescent-specific messaging, among others).Consider “cash plus” programming: Link adolescents in cash transfer participating households to existing services, such as sexual and reproductive health information and services, treatment and testing for HIV, or provide complementary programming, such as vocational training, financial inclusion and e-banking, mentorship schemes and safe spaces.Build staff capacity: Paying attention to the key cadre tasked with making inter-sectoral linkages on the ground, such as social welfare and monitoring officers.Measure change: Use a combination of impact evaluations, process evaluation, and qualitative research to help understand 1) how norms affect program impacts of social protection programs and 2) the role of social protection (and complementary schemes) in changing gender norms, and how changes *actually* happen.Facilitate evidence uptake: Use the evidence to engage with policymakers and communities to build their support for transformative adolescent-focused interventions, and advocate for reaching the ‘hardest to reach’ adolescents.  Maja Gavrilovic is a Research Analyst in the Social & Economic Policy Section at UNICEF’s Office of Research – Innocenti, where she conducts research with the Transfer Project.Tia Palermo is a Social Policy Specialist in the Social & Economic Policy Section at UNICEF’s Office of Research – Innocenti, where she conducts research with the Transfer Project.The Transfer Project is working to provide rigorous evidence on programme impacts in an effort to inform future programme design and scale-up. For more information on the Transfer Project’s research on cash transfers, we invite you to read our research briefs here or follow us on Twitter @TransferProjct  
Measuring taboo topics: List randomization for research on gender-based violence
Blog

Measuring taboo topics: List randomization for research on gender-based violence

Gender-based violence is notoriously under-reported—for understandable reasons. Experience of violence is highly stigmatized and victims are often shamed. Respondents may be fearful of retaliation if perpetrators and others find out they have disclosed their experiences. There may also be cultural taboos about disclosing violence, which may be seen as a family issue. This is why most official statistics on gender-based violence are said to be the lower bound of the true prevalence within a population. In an earlier study we published using nationally representative data from 24 countries, we found that most women who experienced physical or sexual intimate partner violence (IPV) had never told anyone. Only a third of women disclosed to friends or family, and only 7 per cent formally reported to police, health facilities, social or legal services. Imagine attempting to improve IPV programming and policy while using data which represents only the tip of the iceberg. Researchers have long invested in designing and implementing surveys to measure the incidence of violence while seeking to both maximize disclosure and minimize harm. However, these careful fieldwork considerations often come at a price. For example, survey instruments which ask behaviorally specific questions for IPV (commonly collected in the DHS and currently the gold standard), can be lengthy to complete. In addition, survey teams must undergo specialized ethics training, provide anonymous referrals to services, employ same sex enumerators, and implement sampling methods to ensure that both women and perpetrators are not interviewed in the same household (or, at times, in the same communities). All this comes at a cost, which may overburden or be logistically infeasible for multi-topic surveys which are not primarily focused on violence. What if there was a way to collect information about violence, while reducing under-reporting without directly asking about violence? A 50 year old woman who has been affected by gender-based violence receives counseling at One Stop Centre that provides support at the Gihundwe hospital in Rusizi district in Rwanda.In a recent publication in Health Economics, we implemented a list randomization to assess the impact of an unconditional cash transfer program on IPV in rural Zambia. List randomization or list experiments, are not a new technique. They have been utilized by Political Scientists for decades to examine public opinion on sensitive topics which respondents were likely to misreport (think racial prejudice, abortion opinions, support for insurgents in war). Economists have increasingly become interested in this method, and others have blogged about methodological considerations. In basic terms, list randomization aggregates a response to a sensitive question alongside responses to non-sensitive questions, thereby masking the respondent’s specific answer to the sensitive question. By randomizing lists with and without the sensitive question, researchers can identify prevalence or incidence of the sensitive item within the population or differences between groups (for example treatment and control), but not attribute the sensitive response on an individual basis. If respondents believe that their sensitive answer is not disclosed to the interviewer, they may be more likely to report private behaviors, such as experience of violence. In our study, we asked one sensitive question to female primary caretakers of children under the age of five at the start of the study: “In the past 12 months, have you ever been slapped, punched, kicked or physically harmed by your partner” in combination with four non-sensitive questions of the same recall period. An example of the non-sensitive questions is: “In the past 12 months have you ever taken care of a sick relative who is unable to care for themselves?” Respondents were asked to report how many of the items they experienced in total (but not which specific items). The data collection was part of the four-year follow-up of a longitudinal randomized control trial where the beneficiary group received an unconditional child grant program (CGP) provided by the Government of Zambia. In our study, we were interested in two things. First, was it feasible to implement the list randomization in a large-multi topic survey—would enumerators collect the data correctly and would prevalence of violence be credible? Second, since the CGP had been highly successful in meeting its main poverty-related objectives, and increased the financial standing of women, we wanted to know if the cash transfer also affected IPV (a potential which has been demonstrated elsewhere). What if there was a way to collect information about violence, while reducing under-reporting and without directly asking about violence?What do we find? First, the list randomization appeared to function as planned, with no evidence of ‘too low’ or ‘too high’ reporting across groups of questions (referred to as ceiling and floor effects). We estimated 15 per cent of women had been exposed to physical IPV in the last year. This is lower than the DHS estimates for Zambia of the same year (2014), which estimate a 21 per cent prevalence for past year physical IPV. This is not surprising as the DHS asks seven questions about specific violent acts which are aggregated to produce this figure. Moreover, when we analyze the impacts of the program, we find that after four years, the CGP resulted in no measurable increases or decreases in IPV. Despite these interesting findings, we are not able to assess the level of underreporting for IPV specifically for the same reasons we could not implement a full IPV module: As the evaluation was not focused on violence or gender topics, there was no logistical room to add complex ethical and logistical arrangements to the evaluation needed to collect IPV measures. However, two other recent working papers have experimented with this approach and shed light on measurement bias. The first paper, by Joseph and colleagues [Underreporting of Gender-based Violence in Kerala, India], examines two types of violence: domestic violence and physical harassment on buses. The authors measure both questions at the household level (e.g. at least one woman in my household has faced physical aggression from her husband; at least one woman/girl in my household has faced physical harassment while traveling on public/private buses). They find that underreporting is over nine percentage points for IPV, however negligible for harassment. They also find a number of sub-groups are more likely to underreport, for example urban households and poorer households are more likely to underreport IPV, as are female respondents. Although a useful demonstration of reporting bias, the analysis is complicated by the fact that the questions are asked at the household level, and thus are not cleanly comparable to the gold standard of women’s self reports. However, it does suggest that respondents in general are more likely to disclose an incident of ‘public’ harassment as compared to ‘private’ abuse—as hypothesized by the literature on underreporting. An 18 year old mother, holds her 2 year old son near the coast shortly after arriving in Cox's Bazar District, Bangladesh. They spent seven days on a bank of the Naf River in Myanmar before making the crossing aboard a fishing boat. There have been numerous reports of gender-based violence affecting Rohyinga refugees fleeing from Myanmar.The second paper, by Agüero and Frisancho [Misreporting in Sensitive Health Behaviors], conducts a list randomization for physical and sexual IPV among female micro-credit clients in peri-urban and rural areas of Lima, Peru. The nice thing about this paper is that the authors conduct nine separate list experiments for each IPV item, thus they are able to compare each IPV item separately (being pushed, being slapped, being threatened with a knife, gun or weapon etc.). In comparing women who were asked the list experiment and those who were asked the direct question—they find no significant differences. However, digging deeper, they find there are differences by level of education of the woman. In particular, women with completed tertiary education report higher levels of IPV under the list experiment as compared to the direct questioning. The implications are that reporting bias may differ by characteristics of the woman, thus changing our conclusions about who suffers from violence, or how interventions affect different sub-groups of women. In this case, authors conjecture that higher educated women may face larger (real or perceived, including higher stigma) costs of being exposed and require higher levels of confidentiality to make them feel safe. However, there are no differential effects by other characteristics of the women, including age, marital status, employment status, memory scores or others, suggesting overall differences between the two methods are limited. Taken together, these three papers build on efforts across disciplines to consider the usefulness and applicability of list randomizations for collection of data on violence. In the best case scenario, we may be able to assess underreporting, similar to other methodologies researchers have investigated, including use of self-administered surveys and qualitative methodologies. In other cases, we may be able to leverage this method for a ‘light touch’ way to monitor potential backlash, increases or decreases in violence in multi-topic and non-sectoral evaluations, which would otherwise not endeavor to collect violence information. We encourage further experimentation and creativity to further understanding of how to best measure, respond to and program for reduction of violence. Amber Peterman and Tia Palermo are Social Policy Specialists at the UNICEF Office of Research—Innocenti working on health, gender and social protection evaluations under the Transfer Project, a multi-organization research and learning initiative evaluating Government cash transfers in sub-Saharan Africa. Explore the UNICEF Innocenti research catalogue for new publications. Follow UNICEF Innocenti on Twitter and sign up for e-newsletters on any page of the UNICEF Innocenti website.
Are randomized control trials bad for children?
Blog

Are randomized control trials bad for children?

There was a time when UNICEF was known in development circles as the agency that “does everything but knows nothing.” Indeed, UNICEF is known for getting things done for children through persuasive advocacy, a human rights approach, and its presence on the ground. Today UNICEF is increasingly committed to evidence-based programming, and researchers around the world are studying the effectiveness of UNICEF’s work. In my role at UNICEF Innocenti, I frequently have discussions with UNICEF country staff who want to know how their programmes are working. A typical discussion with those working on violence against children, poverty reduction, emergency response, nutrition, and more starts with colleagues telling me: “We want to rigorously test how well our programme works, but we don’t want to do a randomized control trial (RCT).” For many in UNICEF, RCT is a bad word. It conjures ideas of cold-hearted researchers arbitrarily withholding programme benefits from some households and villages for the sole purpose of racking up academic publications in journals no one will read. This thinking assumes that other options are equally as good, so we can simply take those evil RCTs off the table and select from other, “pro-children” evaluation methods. For many RCT is a bad word. It conjures ideas of cold-hearted researchers arbitrarily withholding programme benefits from some households and villages for the sole purpose of racking up academic publications in journals no one will read.And while other evaluation methods can provide powerful evidence on programme impacts, and RCTs are not always needed, before choosing a method, we need to first understand, in the words of Rachel Glennerster and Shawn Powers, “what are we judging RCTs against?” Indeed, while RCTs get the most attention when discussing the ethics of impact evaluation, all methods come with ethical implications. To make a random selection of RCT treatment villages for an ongoing social protection programme impact evaluation in Tanzania the names of villages were literally drawn blindly from this hat.Both experimental (RCT) and quasi-experimental methods try to get at causal impacts of programmes and policies. They do so by constructing a “counter-factual,” the term researchers use to describe what would have happened to beneficiaries had they not received the program (also referred to as “treatment” or “intervention”). Since we haven’t yet invented a time machine where we can first give a group of people a treatment, see what happens, and then go back in our time machine and observe what happened without the treatment, we have to use other techniques to measure the counterfactual. RCTs do this by determining who gets the treatment and who doesn’t by chance, which usually ensures there are no systematic differences between the groups. For example, those who get the treatment aren’t getting it because they are more motivated, more informed, live closer to health facilities, or are from a privileged political group, etc. Quasi-experimental methods use other techniques to construct a comparison group of people who did not receive the treatment. However, we cannot be as certain the estimated impacts are a result of the treatment, and not due to other factors. Of all these methods (non-experimental/observational, quasi-experimental and experimental/RCT), RCTs provide the most credible evidence on programme impacts, however, they are not always possible. In my work at UNICEF with the Transfer Project, we use both RCTs and quasi-experimental methods. However, non-experimental and quasi-experimental come with limitations. If we use a poor comparison group (or no comparison group at all), we could end up overestimating or underestimating treatment impacts—and we often don’t know with certainty which is the case. A non-credible or non-rigorous evaluation is a problem because underestimating program impacts might mean that we conclude a program or policy doesn’t work when it really does (with ethical implications). Funding might be withdrawn and an effective program is cut off. Or we might overestimate program impacts and conclude that a program is more successful than it really is (also with ethical implications). Resources might be allocated to this program over another program that actually works, or works better. Mobile health teams provide essential basic health services and collect household data in remote and isolated communities, with a special focus on maternal and neonatal care in Afghanistan.So if RCTs produce the most solid evidence, why don’t we use them everywhere? There are several reasons for this. Sometimes you just can’t randomize who gets a program due to implementation-related reasons (for example, every village in the district benefits from the same road or improved water system). Sometimes you can randomize, but programmers are reluctant to do so because of perceived ethical concerns. In the first scenario, we turn to quasi-experimental methods where possible. Now let’s break down some of the concerns in the second scenario. All research methods (not just RCTs) have ethical considerations to be mindful of. These include, among others, informed consent for research, principles of ‘do not harm’, necessary referrals for additional services if needed and review of national and international ethics review boards to ensure ethical guidelines are adhered to. However, one concern unique to RCTs is that benefits are purposefully given to one group and not to another. Implementers need to consider whether in fact this is ethical. In many cases it is. For example, if roll-out of the programme can’t reach all intended beneficiaries at the same time (say there’s a phased roll-out due to budgetary or capacity constraints) then we can take advantage of the group experiencing delayed roll-out and use them as a control group. Further, if we don’t know whether a programme is effective, it’s not unethical to randomize some individuals to not receive that programme (in fact receiving an ineffective programme may do more harm than good). Finally, we must also ask ourselves: Is it ethical to pour donor money into projects when we don’t know if they work? Is it ethical not to learn from the experience of beneficiaries about the impacts of a program? RCTs can be a powerful tool to generate evidence to inform policies and programmes to improve the lives of children. As with any type of study, researchers must adhere to ethical research principles. However, when choosing the right type of methodology to evaluate a programme, it’s important to keep the ethical implications of each in mind, as well as a clear understanding of all the options, including the option of never knowing what impact your programme is making. Tia Palermo is Social Policy Specialist in the Social & Economic Policy Section at UNICEF Innocenti, where she conducts research on social protection programmes in Sub-Saharan Africa with the Transfer Project. Explore the UNICEF Innocenti research catalogue for new publications. Follow UNICEF Innocenti on Twitter and sign up for e-newsletters on any page of the UNICEF Innocenti website.  

Journal articles

Evidence from Africa shows cash transfers increase school enrollment
Journal Article

Government Anti-Poverty Programming and Intimate Partner Violence in Ghana

Evidence from Africa shows cash transfers increase school enrollment
Journal Article

Uptake of HIV testing among adolescents and associated adolescent-friendly services

Evidence from Africa shows cash transfers increase school enrollment
Journal Article

Impact evaluation of a social protection programme paired with fee waivers on enrolment in Ghana’s National Health Insurance Scheme

Evidence from Africa shows cash transfers increase school enrollment
Journal Article

Perspectives of adolescent and young adults on poverty-related stressors: a qualitative study in Ghana, Malawi and Tanzania

Podcasts

Evidence from Africa shows cash transfers increase school enrollment
Podcast

Tia Palermo on cash transfers, gender and impact of research

Evidence from Africa shows cash transfers increase school enrollment
Podcast

#HEARMETOO: UNICEF Research on Gender-Based Violence for #16Days of Activism

Evidence from Africa shows cash transfers increase school enrollment
Podcast

Tia Palermo on Evaluation of Social Cash Transfers in Sub-Saharan Africa