Conducting Primary Research in Yemen: Challenges and Lessons
In the words of Valerie Amos, former Under-Secretary-General and Emergency Relief Coordinator for the UN, “it is both possible and necessary to conduct sound and rigorous research and documentation in the challenging contexts of conflict zones.” The reasons are manifold. Protracted armed conflict most often re-orders local conditions at multiple levels, including power and resource distribution, relationships between groups and dynamics of exclusion and marginalisation. It is essential for any national or international stakeholders looking to support local communities or tackle the causes and effects of conflict to understand its ecosystem and how local conditions and communities have changed, possibly irreversibly. As the conflict evolves, reliable and independent information becomes scarcer, while the narratives about the conflict are dominated by the warring parties. Independent research and data collection also function as vital mechanisms to give civilians and communities – who are very often targeted by warring parties – a chance to voice their concerns, grievances, aspirations and priority needs.
To perform this important function, conflict researchers grapple with a variety of practical, technical and ethical challenges, many of which require a constant re-assessment of and balancing between the potential benefits of the research and the risks involved. Among the multiple Fragile and Conflict Affected States (FCAS) where ARK conducts primary research, Yemen is among the most challenging. The UN considers Yemen the world’s worst humanitarian emergency, and the war is characterised by multiple conflicts, power struggles and foreign sponsors. With the country’s already weak infrastructure now destroyed, COVID-19 is tragically ripping through communities across the country already imperilled by other epidemics, natural disasters and famine. The following non-exhaustive list details some of the challenges that we have faced in recent years conducting primary research in Yemen in the hope they are of use to others seeking ground truth.
Ensuring the (relative) safety of local partners and networks
In a fluid environment such as Yemen’s, it is impossible to be 100 percent confident about duty of care provisions. Beyond detailed preparations, which often involve complex discussions about ethical dilemmas, risk assessments and localised conflict analyses, a certain degree of risk tolerance is necessary to be able to operate and collect data. The challenge is where to draw the line. Even in governorates that are not disputed by the warring parties and that may not have seen recent rounds of fighting, sending in an enumerator and support team can carry its risks. In some target locations, for example, local tribes are highly suspicious of individuals collecting information and often demand to know who the enumerators are, their political affiliation, who are they working for, or who gave them permission to operate. If the answers from the data collection team are not palatable, these interactions can lead to the detention of the enumerator. Plenty of behind the scenes work is then be required by the team leaders or field supervisors to guarantee their release.
Managing research partners and data collection networks
In these conditions, managing local research partners and data collection networks remotely can also present additional challenges. Improvised solutions to the various obstacles that arise may look reasonable to partners on the ground/in the local context but are often unacceptable to clients and those commissioning research. For example, paying local authorities and armed groups to obtain permissions is not an uncommon practice in Yemen but a red line for us. On one occasion, we found out that one of our previous local partners had submitted a fake questionnaire to local authorities in order to obtain permission to implement the research, to then replace it with the real questionnaire in order to conduct the interviews. All the precautionary measures taken to prevent those local authorities from getting hold of the real questionnaire were insufficient to assuage concerns about that prospect and the risk it could pose to enumerators and the project. It is also common for enumerators to refuse to turn on the geolocation on their data collection devices/tablets for security reasons – there is a lot of suspicion about geolocation in Yemen as elsewhere, thanks to drone strikes – which deprives team leaders of a useful tool to monitor, quality-control and select specific locations.
Internet outage and communication problems
Effective and regular lines of communication with local partners is thus essential from a logistical, risk mitigation and technical point of view. However, in Yemen, the internet is expensive and among the slowest in the world, impacting research project timelines and generating expectation gaps between the various stakeholders involved. Earlier this year, a damaged cable in the Suez Canal resulted in significant internet downtime, affecting communication between us and field supervisors and between them and enumerators, impacting duty of care and security considerations. This issue also prevented enumerators from uploading data from their mobile devices to the server and then deleting the information on their devices, which is a standard security and data protection measure in sensitive target locations.
Navigating interference from warring parties
Yemen’s warring parties and myriad armed groups are highly likely to interfere with data collection activities. This interference can take many forms and can happen intentionally or by chance. To move around, data collectors often go through checkpoints where individuals are thoroughly searched and electronic devices such as mobiles and laptops can be checked or confiscated. On a couple of occasions, Focus Group Discussions (FGD) were raided and disbanded by armed actors and the exercises had then to be repeated in alternative locations, causing delays of several weeks. There is also the risk of diversion of funds/fiduciary risks. Earlier this year for example, a 2% levy on aid agencies and civil society organisations receiving external funding was imposed by one armed group in some parts of Yemen, and then retracted following widespread condemnation.
Being Conflict Sensitive
A conflict sensitive approach to primary research seeks to understand the context in which one is operating, the role of key stakeholders, intergroup tensions, the push and pull factors of social cohesion, and how the specific activity one is conducting fits or interacts with that context. The aim is to avoid unintentionally feeding into negative dynamics on the ground, but also to maximise the potential of the specific activity to contribute to strengthening social cohesion or support peacebuilding efforts. But in Yemen today, there are multiple sensitivities and grievances, some of which only surface once the data collection activities are already underway. Certain questions are extremely sensitive for both local authorities and populations. This often requires ample time to socialise the research with relevant local stakeholders through face to face interaction, which may increase both the costs and the risks. It is also common for interviewees to refuse to answer some questions or to be recorded. In some instances, especially in the most sensitive locations where groups that control the area are not supported or are resented by the local populations, interviewees answer questions to then withdraw their permission for the content of the interview to be shared.
Mainstreaming gender
Guaranteeing a suitable percentage of female respondents in the research is also challenging. The conflict has generated opportunities for Yemeni women to take on more active and influential public roles at the local level – at least half of the civil society organisations currently registered in Yemen are run by women. However, the war and its effects have placed a significant burden and levels of stress on Yemeni women, many of whom are currently providing for their families in addition to being in charge of domestic duties and childcare, often in the absence of men. Depending on the theme(s) of the research and specific target locations, identifying female respondents willing or able to participate can be tricky, and gaining their trust to share their views on their local communities and sensitive matters can take time. In addition, interviewing women and explicitly addressing gender dynamics can also raise issues with the local male population, or reinforce perceptions that the international community is biased toward women or imposing a foreign, unwelcome agenda, as we’ve recently heard a few times in the course of a research project.
Choosing target locations and venues for the activities
There is no one size fits all approach to research in Yemen. An approach or an activity that may go smoothly in one target location may be extremely complicated to implement in another one. Within the same governorate, operating in some districts controlled by one armed faction can be a lot easier than operating in other districts controlled by another armed group. In a couple of northern governorates, interviewees and FGD participants were very wary about their involvement in the research. Thus, our network carried out the interviews and the FGDs outside the main city centres and in more remote/peripheral areas in an attempt to reduce risks, although that brought new practical and logistical challenges. In one instance, while ARK’s data collection partner was moderating a FGD in a location considered safe, fierce armed clashes erupted just a few doors away from where the FGD was being conducted, forcing the end of the session.
Ensuring quality and integrity of the data
In these circumstances, measuring sensitive attitudes presents research teams with numerous methodological and technical challenges. Sound questionnaire design is even more vital. The risk of social desirability bias, refusal and item non-response, and preference falsification are especially acute for themes related to governance, service provision and security, or sensitive demographic information. To address these issues, a combination of direct and indirect techniques, observational questions and survey experiments is often necessary. The challenging conditions and the fatigue of both enumerators and respondents mean that questionnaires, scripts and other research tools need to be kept short in terms of time required to complete them.
The depletion of local research capabilities
As expected, the human and financial impact of the conflict has led the great majority of very talented Yemeni researchers to emigrate to other MENA countries (Egypt, Turkey, Jordan), Europe and the US. Yet some have managed to continue producing excellent Yemen-focused research across a variety of themes, supported by both new research institutions led by Yemenis and Western research institutions. However, the conflict has tended to dictate the research agenda, while a lot of knowledge about more structural and deeper dynamics on the ground has been lost.
Adapting to COVID-19
As the arrival of COVID-19 in Yemen tragically exemplifies, local circumstances can always deteriorate further. While the somewhat naïve hopes that the pandemic would force warring parties to impose a ceasefire and push harder for a peace deal have been dashed, COVID-19 has exposed and exacerbated the human tragedy across the country, with the official number of confirmed cases nowhere near the real figures. The restrictive measures imposed at a different pace and varying degrees of intensity by the multiple authorities on the ground have naturally impacted mobility. The duty of care for local researchers, respondents and communities more broadly is difficult to observe in a country with poor or in many cases destroyed infrastructure, lack of clean water, and absence of personal protective equipment and testing capacity. In the worst-hit areas, the inability to meet face-to-face demands the design of remote data-collection techniques adapted to the specific circumstances, which again raises a variety of technical, logistical, ethical and practical challenges for the research team. But as researchers embrace these challenges we must, as though not all risks can be avoided or transferred, they can be mitigated, and even where they must just be accepted, it is because of the acute imperative to provide ground truth about the reality of Yemen’s complex humanitarian emergency as experienced by its people.
Manuel is Head of Research at ARK Group. He is an expert on mix-methods approaches to research in Fragile and Conflict Affect States (FCAS) and has recently co-authored the chapter “Muslim Brothers, Salafis and Jihadis: Lost in the War” in the book Global, Regional and Local Dynamics of the Yemen Conflict, published by Palgrave Macmillan earlier this year. He obtained his Ph.D. in International Relations from the London School of Economics and Political Science.