Logo UNICEF Innocenti
Office of Research-Innocenti
menu icon

Remote data collection on violence against women during COVID-19: A conversation with experts on ethics, measurement & research priorities (Part 1)

Amber Peterman, Amiya Bhatia and Alessandra Guedes
11 May 2020
A woman holds a mobile phone, showing a mobile app for reporting incidents of family violence in the city of Fortaleza, Brazil. 

Innocenti Think Piece (Part 1)

Violence against women (VAW) is a priority global concern especially during the COVID-19 pandemic. Supporting survivors during this time requires understanding the characteristics and magnitude of violence and effectiveness of responses – for which we need rigorous research. Researchers are well positioned to contribute to policy dialogue, drawing both on past evidence to inform critical pandemic responses, as well as studying dynamics as they unfold to inform real-time decisions within future pandemics. This requires that researchers adapt their research methods to remote approaches, place their research on hold, or develop new and creative methods. For example, impact evaluations and routine household surveys are converting in-person surveys into mobile data collection efforts. New surveys planned to monitor and rapidly inform the COVID-19 response are using diverse remote options, including phone, or internet platforms.

While general guidance exists for phone and SMS-based surveys and a recent blog outlined the key considerations for researchers with regard to collecting data on VAW during COVID-19 – questions remain: Is it necessary - or possible - to safely and accurately collect VAW data, given the risks and ethical challenges? And, if so, by whom and under what circumstances should VAW data be collected? Further, if surveys proceed, in past efforts, how have experts overcome ethical and methodological challenges within remote VAW data collection to produce quality and actionable evidence?

Collecting data on experiences of VAW has always required careful attention to ethical and safety concerns throughout the research process. Guidance for this is based on decades of research. ‘Putting women first’ includes special selection and training of interviewers, attention to confidentiality and referral mechanisms, as well as establishing adverse event protocols to ensure rapid response in the case women or data collectors are in danger of immediate harm. In addition, there is an ethical obligation to ensure no individuals or communities can be identified or stigmatized and that findings are properly interpreted, presented and used to advance policy and programming for survivors. All these considerations are relevant - and magnified - in the context of COVID-19. Yet, remote surveys have been undertaken ethically and safely in the past, and we should learn from these experiences.  

In this 2-part “Conversations with experts”, we unpack what COVID-19 means for data collection efforts. In part 1 of this blog, we focus on VAW, and talked with 10 experts who together have led, designed and implemented dozens of data collection efforts: Amber Clough, Dr. Avanti Adhia, Dr. Henriette Jansen, Kaitlin Love, Dr. Kathleen C. Basile, Dr. Kathryn Falb, Olivia Ryan, Dr. Nancy Glass, Sara Grant-Vest, Dr. Sharon G. Smith (learn more about the experts below). What follows is based on informal conversations to frame issues many researchers are grappling with during COVID-19. The responses below draw on personal expertise and experiences, do not represent the official standpoints of their organizations and are not meant to replace official guidance from affiliated organizations.

Is it necessary - or possible - to safely collect VAW measures given the risks and ethical challenges?

While VAW measures have been safely collected over the phone, internet and other remote methods before (e.g. Argentina, Canada, United States) – these have primarily been undertaken in high income countries (HICs) – and without the added survey logistical challenges imposed by COVID-19. Within the rapid response efforts, experts emphasized the importance of critically assessing if and how VAW data will be useful for policy action, and recommended that researchers should: “only collect data when you can collect data safely, when it is clear what it means and what not, and when it is actionable. We must get our priorities right and support vulnerable groups. No situation justifies unethical, unreliable and un-actionable data collection exercises [Henriette Jansen]” Experts argued that it may not be justified or feasible to collect VAW data during lockdown, suggesting that “we already know it is a pervasive issue” and cautioning that “especially in situations of lockdown, we cannot safely and reliably get data on women’s experiences of violence when they are at home with their abuser. If we would collect such data using remote methods, there will be real safety risks and the findings will be hard to interpret; as we probably get a huge underestimation due to non-response, participant bias and non-disclosure. This will further undermine the importance of the issue [HJ].” This sentiment is in line with recent WHO and UN Women guidance on Violence Against Women and Girls data collection during COVID-19, which recommends not to include direct questions on VAW experience within population-based rapid assessments.

A common question raised related to the timing of data collection -- including whether data could be equally beneficial or impactful if collected after COVID-19 restrictions were relaxed. “In humanitarian settings, we are still working through how to best ensure confidentiality of survey participants when asking VAW direct questions remotely, both during and after the survey—and any attempt will require additional risk/benefit analysis [Kathryn Falb].”

Yet others cautioned that we should not ‘reinvent the wheel’ – and reminded us that researchers have operated in public health emergencies – and collected remote data on VAW for over a decade (particularly within trials of web and app based applications), without compromising safety of participants. Further, VAW researchers have “fought for years to collect direct and specific information about VAW in surveys [Nancy Glass/Amber Clough]” and now is not the time to go backwards. With safety protocols in mind, this should not be a time to limit research in a way that “will not actually give us the information we would need to improve services, safety and health for survivors and children [NG/AC].” Thus, while decisions to collect data on VAW -- or not -- clearly depend on context, some argued that experienced researchers should still consider opportunities to fill priority actionable research gaps on VAW, with appropriate training and where careful implementation of safety protocol is ensured. The debate of when and how to integrate VAW questions is not new, including diverging opinions on the appropriateness of conducting stand-alone-surveys versus integrating VAW into multi-topic surveys—however has resurfaced again within COVID-19 discussions.

A worker at the South African NGO ChildLine providing free counselling services for children and adults across South Africa. These centres receive daily calls from children who experience anxiety, abuse, fear or difficulty with homework, or witness or experience domestic violence or online harassment.

What are the minimum conditions that must be in place in order for questions on VAW to be included in a remote survey?

While standard ethical protocols apply – experts also flagged a number of areas where challenges may occur when collecting data remotely. The first set of conditions relate to confidentiality and informed consent. Experience from the National Intimate Partner and Sexual Violence Survey (NISVS), an ongoing, nationally-representative telephone survey that assesses sexual violence, stalking, and intimate partner violence victimization among adult (18 years and older) women and men in the United States suggests a role for ‘graduated informed consent.’ “Graduated informed consent following WHO guidelines as an important step to increase safety … In this way, the initial person who answers the telephone is provided general non‐specific information about the survey topic. The specific topics of the survey are only revealed to the individual respondent who has been randomly selected to participate in the survey [Kathleen C. Basile].” In a survey of Syrian refugee women undertaken by Ipsos in 2017-18, guidelines included the recommendation that “interviewers schedule time with respondents when they will be able to be in a quiet, private place instead of conducting the interview whenever someone picks up the phone [Kaitlin Love]” (Annex 5). It is worth noting that privacy may be particularly challenging during times of quarantine, particularly in low- and middle-income countries (LMICs). As interviewers may not observe the space participants are in, protecting confidentiality may include a discussion to ensure respondents cannot be overheard by anyone. This includes understanding that “safety means not having toddlers or other children able to overhear a discussion or read texts (older children) – this could place children in a difficult and unsafe situation, for example, if the abusive partner demands the child disclose what mom was talking or texting about [NG/AC].” A recent blog post suggested that “a remote interviewer will have less capacity to minimize confidentiality breaches and [address] safety concerns than in-person interviewers” and that because “informed consent processes require respondents to have a good understanding of the inherent risks involved in the study [it is unclear] the extent to which respondents are able to provide informed consent in a study using technology they do not have control over is questionable.”

Experts suggested eligibility requirements to participate in research on VAW could explicitly ensure participants have access to safe devices to ensure confidentiality. For example “if communicating via SMS, include a requirement that the participant can access the study info on a device that their partner does not have access to (e.g. a device their partner doesn’t know about or a trusted family/friend’s) … If they do not have access to a safe device, make sure you ask if you can safely provide a referral number for support (hotline, etc.) … if she gains access to a safe phone, she may reconnect to participate [NG/AC]” This aspect is especially important as both calls, SMS, and internet websites can be traceable and observed by controlling partners after an interview and other household members.

A second set of concerns relates to safety protocols for remote surveys -- as check-ins and signals of distress are not automatic -- and routine referral services may be compromised during COVID-19. For example, as interviewers may be unable to read body language or recognize distress over the phone, special training should be done to ensure these signs are not missed. “Training interviewers to work with respondents to quickly change the subject when someone walks in the room, or providing all respondents with a specific phrase or “safe word” that they can use if someone interrupts the interview and it needs to be continued later [KL].” Similar actions can be taken for internet based surveys, which could include “an exit quickly button on each survey page in case individuals were taking the survey in the same space as their partner, or their partner entered the room. The survey could also be mobile-friendly (not just able to be taken on their computer) so women have more options of where to take the survey while potentially in the confines of their home [Avanti Adhia].” An important part of the safety protocol entails providing follow-up referral services to all participants in a safe manner, ensuring this information is not disclosed to other household members, and updating referral pathways to account for COVID-19. Safety protocol may also include remote follow-up with participants after survey administration is completed to ensure no adverse effects or other developments have occurred post-data collection.

What are the priority research gaps in relation to COVID-19 dynamics and VAW?

Experts emphasized the importance of understanding a variety of factors, around three main themes: (1) how COVID-19 – including mitigation efforts -- affect the risk of VAW; (2) if, and how, women are able to seek help; and, (3) which programming efforts are effective. Experts seemed divided on which of these themes should have greater priority, with some experts suggesting that researchers should not measure the prevalence of VAW using rapid methods, but should instead focus on themes 2 and 3 in order to understand the needs, experiences and strategies women are using to survive and what organizations can do to prevent and respond to violence.

“Previous work suggests that public health crises (e.g., natural disasters) can increase risk for numerous forms of violence … Priority areas for survey questions in times of crisis are questions that can assess recent victimization experiences. For example, using questions that can assess prevalence since the crisis began in a country, municipality, or province can be helpful [KCB].” Researchers stressed that simply understanding current prevalence – or frequency -- will not tell us how much VAW has increased due to COVID-19 – as violence is already experienced by many individuals across diverse settings. To understand the effects of COVID-19 multiple measures are likely required, including both before, during or after the pandemic or other sources of random variation. In addition, pathways related to various mitigation effects were mentioned as important to better understand: “For violence occurring in the home, it would be important to understand how the shelter-in-place recommendations and other stressors (e.g., finances, ability to work, homeschooling children, access to household supplies, access to medical care, anxiety about the virus, substance use) affected victims. For example, are certain aspects of the COVID-19 experience associated with increased violence? [Sharon G. Smith]” Importantly, research on the magnitude of violence should consider risks of VAW both within and beyond the household—and the type of risk may vary greatly by setting. In most HICs, the focus has been on effects of lock downs, whereas in LMICs, broader risks (e.g. economic risks) may be more important.

Experts also described the importance of research on women’s feelings of safety, sources of stress, and ability to seek help over time during the pandemic: “From a knowledge generation standpoint, I would specifically like to know if women are (feeling) safe or not in the current situation (especially lockdown), and whether they have opportunities to seek help or not; and what are the barriers to seeking help, what other help would they have wanted [HJ]” Related to this, another priority is understanding the range and adequacy of services available to women: “It would also be useful to understand what viable alternatives individuals have if they are experiencing violence (e.g., shelters, friends/family, helplines) and barriers to seeking/obtaining help to identify potential interventions and guide resource planning [AA].” Research to inform how existing services could be adapted was also identified as important: “Regarding the immediate time horizon, within humanitarian settings, we need to understand how best to pivot critical life-saving violence response services, such as case management, to mobile and/or remote delivery options, while maintaining quality and responding to the needs of survivors [KF].” Experts noted that although this is an unprecedented situation, it is unlikely it will be the last pandemic, and we must be better prepared for future emergencies.

What other factors can help increase the safety and accuracy of remote data collection on VAW?

Experts stressed that a myriad of other considerations are needed to support the safety and accuracy of data – which depend on the context, population and nature of measures being collected. Some of these included the importance of training and support of interviewers, rapport building, attention to questionnaire design/sampling appropriate for remote data collection and data safety/sharing protocol.

Experts noted that extra efforts are needed to support team members receiving and analyzing the data, especially if teams were spread out geographically and not debriefing face-to-face: “Some prospective interviewers will have histories of violence, and it is important that they evaluate their own emotional ability to serve as interviewers … Additionally, it is important to demonstrate how the data will be used to address the issues under study. For example, show how the data can be used in the community in which they live. These approaches will increase their buy-in of the study and help them feel like a part of the solution [SGS].” Experts mentioned the importance of regular check-ins and debriefs with team members to ensure interviewers could process experiences together and feel supported.

Hadia (fictional name), holding her youngest daughter, has five children and reports being beaten by her husband after an argument about their son. Hadia sought refugee into her brother's house for one week. 

Experts emphasized the importance of rapport-building during phone conversations, and in texts, or email communications for follow-ups or check-ins. This could include building in breaks in the survey, including inspirational messaging or asking participants to “take a moment to take a deep breath (as contextually appropriate) [NG/AC]” or reassuring participants that their answers are appreciated and will be important to inform policy. While under-reporting was mentioned as a key concern with remote surveys, experts also underscored potential benefits as well: “It is plausible that phone or remote surveys increase reporting due to reductions in social desirability bias as some evidence has shown related to audio-CASI approaches [KF].” This sentiment was echoed by those working on phone-based surveys, “when we first started exploring moving away from face-to-face data collection with survivors, we were concerned about a loss of rapport and trust between survivor and RA, essential to recruitment, quality data, safety, and retention. However, doing remote data collection for over 10 years we have learned that a trusted environment can be conveyed remotely and even simulated electronically and have had success in implementing safe and effective study protocols using a variety of remote methods [NG].” Thus it is possible that if done well, respondents will feel safer and less judged in remote surveys as compared to face-to-face interviews.

Researchers also stressed the importance of adapting questions and sampling to survey administration mode. For example, experts mentioned that questions administered over the phone could be simplified – particularly with regard to response options -- while questions based on internet administration could include additional visual components. One recommendation from the aforementioned research with Syrian refugees recounts that in “cases when sensitive questions arise, response options should be yes/no, or numbers (1/2/3), rather than have response options with content which could be repeated out loud or understandable by others in earshot ... This primarily involves shifting away from pre-coded open end or multi-code questions and asking simpler questions [KL].”

Experts cautioned that if research is undertaken online or via the phone, carefully considering which women are included and excluded from studies is essential as the most marginalized women are most likely to be excluded: “Considerable attention also needs to be paid to sampling frames and being explicit about the possibility that women and children may inadvertently be left out of studies due to a lack of access to technology such as phones, etc. [KF].” Based on HIC survey work, others noted that sampling and response rates will differ from in-person surveys – and some of the strategies used to increase response rates may have ethical considerations (e.g. if letters or pre-booking is done, there must be careful attention to not disclose the topic of VAW, similar to graduated consent protocol). Not adequately addressing the issues around quality of training, rapport building, survey design and sampling may lead to inaccurate VAW data which may harm women and hamper effective action.

In absence of being able to collect standard VAW measures, are other existing data sources or proxy measures useful to analyze?

In many settings, where VAW measures cannot be safely collected – what are other options? While there are a variety of proxy measures, including intra-household conflict or disputes, experiences of fear, vignettes around hypothetical violence scenarios, or reporting of community perceptions of violence, some researchers pointed to the limits of selecting proxy measures as substitutes to talking to women directly about their own experiences: “I do not think there are any useful proxies for prevalence of violence (proportion of women experiencing violence as measured using population-based surveys) [HJ].” However, researchers also offered a number of different options—as alternatives to asking direct VAW questions. These included the use of secondary datasets, administrative data, ‘big data’ (e.g. from social media and other platforms) and interviews with community members and service providers. “Examples of useful sources of data to inform gendered responses during COVID-19 are: Secondary analysis of pre-COVID-19 survey data (e.g. who is most at risk, and barriers to using services); analysis of service records pre and during COVID (shifts in reporting; what are the reasons behind that); qualitative methods such as key informant interviews with service providers and frontline workers, mapping of services, case reports, etc. We could find out which services are working and which not and why; if shelters are accessible, if they are full, if there are alternatives for shelters; whether there are safe options for women to report (e.g. in pharmacies) etc. This is actionable: useful to guide response. [HJ]”

Concluding thoughts

While experts all drew on different experiences – they all agreed that new data collection during COVID-19 should be carefully scrutinized and grounded in standard ethical protocols developed over the last decades. In addition, it should draw on experts who have long-standing experience collecting these measures: “Consulting with colleagues who have experience in this area can hopefully reduce the anxiety of shifting to new methods of data collection and increase survivor safety by building upon lessons learned [NG/AC].” They also stressed that just because we can collect data, it does not mean we should. Ultimately, the data should directly contribute to improving the wellbeing of women who take part in the survey, and beyond – putting women first. This means that researchers keen to include VAW questions into existing surveys have to address important ethical and methodological questions and should collaborate with experts to inform and safely carry out surveys. In spite of sometimes diverging opinions, all experts consulted agree: no data is worth placing women’s and researchers’ safety at risk and having poor data that greatly underestimates the levels of violence may be worse than having no data at all.

Contributors

Thank you to all our experts as well as to Elizabeth Dartnall for reviewing this piece, to Lori Heise and Michele Decker who informed the framing of this piece. Stay tuned for Part 2, focused on data collection on violence against children.

[Disclaimer: These opinions are personal statements from experts and do not reflect the official standpoints of the organizations they work for – experts listed in alphabetical order].

  • Amber Clough, M.S.W. Research Program Coordinator, Johns Hopkins School of Nursing
  • Avanti Adhia, Ph.D., is a Postdoctoral Fellow at the Harborview Injury Prevention and Research Center at the University of Washington in Seattle. Her research focuses on the prevention of intimate partner violence by understanding its causes and consequences in addition to the role of policies and interventions in reducing IPV.
  • Henriette Jansen, Ph.D., is Technical Advisor Violence against Women Research and Data in the United Nations Population Fund - Asia and the Pacific Regional Office (UNFPA/APRO) in Bangkok. She initiated and is the technical lead to the innovative kNOwVAWdata Initiative and has led or supported data collection of over 40 national studies on VAW globally;
  • Kaitlin Love is an international development researcher and consultant with 10 years of experience leading projects across the globe. She focuses on applying rigorous qualitative and quantitative methodologies to emerging and fragile contexts, and is a subject-matter expertise in women's empowerment, sustainable development, and migration, refugee, and humanitarian aid. 
  • Kathleen C. Basile is a Senior Scientist in the Division of Violence Prevention at the National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, where she has been since 2000. Her research interests are measurement, prevalence, etiology, impacts, and prevention of sexual violence and intimate partner violence of adults and adolescents.
  • Kathryn Falb, Sc.D. is a social epidemiologist by training and Senior Researcher supporting Violence Prevention and Response and the International Rescue Committee, an international humanitarian aid organization. She has extensive experience conducting surveys collecting VAW in humanitarian settings.
  • Nancy Glass, Ph.D., M.P.H., R.N. Professor and Independence Chair, Johns Hopkins School of Nursing.
  • Olivia Ryan, MSc. is a research manager at Ipsos MORI, and social researcher who specializes in conducting survey research with marginalized groups across the world on issues related to civil rights, discrimination and violence. She holds an MSc Gender from the London School of Economics and a BA (Hons) International Relations and Economics.
  • Sara Grant-Vest is an international social research professional at Ipsos MORI. She has conducted research in over 60 countries over the course of her career including European, Asian and African regions and led multiple surveys measuring the prevalence of VAW, experiences of discrimination and hate crime of among minority groups.
  • Sharon G. Smith, Ph.D., is a Behavioral Scientist in the Division of Violence Prevention, National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, where she has worked since 2005. She currently serves as the project lead for the National Intimate Partner and Sexual Violence Survey and is a subject matter expert for sexual violence, stalking, and sex trafficking.

Authors: Amber Peterman is a Research Associate Professor at UNC and consultant to UNICEF Innocenti, Amiya Bhatia is a Research Fellow in Social Epidemiology and Child Protection at the London School of Hygiene and Tropical Medicine, Alessandra Guedes is the Gender & Development Research Manager at UNICEF Innocenti.