The humanitarian NGO sector is experiencing two conflicting challenges.
Crises around the world due to issues, such as climate shocks, conflict, and economic crunches, are becoming more difficult, more frequent, and are lasting longer. They are also increasingly being layered one on top of the other in the same locations.
At the same time, accessing the necessary resources to respond effectively to such events is becoming increasingly problematic. Costs are rising across the board while funding across the sector comes under rising amounts of pressure.
This situation leaves non-governmental organizations (NGOs), such as Mercy Corps, facing hard choices about how best to direct the funds available to them. The NGO plans to rebrand as Prosper Global from September.
After experiencing labor capacity constraints last year resulting partially from headcount losses, the organization identified a bottleneck in how its research analysts manually gather and condense information. This is an important but time-consuming part of a wider process that involves contextualizing the information gleaned to better analyze specific crises.
Analysts then provide colleagues on the ground with an assessment report of the situation to enable them to take the most appropriate action. Nayid Orozco, Mercy Corps’ AI Solutions and Delivery Manager, explains:
Our analysts were spending many hours, days and weeks collecting information before they could start analyzing it. So, if there was an earthquake, for example, it could take days, or even weeks, to find the appropriate data, which was often spread across websites in different languages. But when there’s a crisis, early information means faster decision-making, so it can be crucial in response terms.
Using VERA for information gathering
As a result, the organization identified two main aims. The first was to get reliable information into analysts’ hands as quickly as possible. The second was to automate and speed up the information-gathering process to compensate for having fewer analysts sometimes working fewer hours due to budget constraints.
This led the NGO to work with existing partner hybrid data and AI platform provider Cloudera on creating VERA (Verified Evidence & Research Assistant). VERA is an AI chatbot that coordinates various agents to answer queries.
The first agent works out what information is required. It then passes the request on to four specialist agents that locate the necessary data. These agents, the number of which will increase over time to handle additional information sources, focus today on:
Conflict and security data gleaned from sources, such as independent conflict monitoring organization ACLED and humanitarian data analysis house, ACAPs
Displacement information from sources, such as the International Organization for Migration’s Displacement Tracking Matrix
Humanitarian reporting using qualitative data and reports from multiple sources, which include reliefweb, a service provided by the United Nations’ Office for the Coordination of Humanitarian Affairs
Local news and information provided by on-the-ground teams in Colombia and Sudan.
A fifth agent then summarizes all the information available into a baseline report, which includes inline references to source material and appropriate citations. The data is subsequently verified and validated by analysts to ensure outputs align with stated aims. They contextualize it, flag any inconsistencies, and complement it using interviews with on-the-ground humanitarian workers to create a more rounded report.
VERA, which runs on Amazon Web Services, was written using Cloudera’s AI Studios development tools and employs Anthropic’s Claude AI large language models (LLMs). As Orozco points out:
Sometimes local or specific news is missed if you do a search using general purpose LLMs, so we gather information from specific sources, which have always been vetted and reviewed by other organizations. Claude then puts the information into a suitable shape or form for analysis.
Taking roll out slowly but surely
The modular system went live for internal use at headquarters (HQ) in June after six months of piloting but has since been rolled out to teams in Sudan and Colombia. Reports have also been produced to support teams in Ukraine and the Democratic Republic of Congo.
Over the next 12 months though, the aim is to deploy VERA in the other 30 or more countries where Mercy Corps operates by taking “a very responsible approach to scaling”. Orozco explains:
We’ll take it slow, gather evidence, and then analyze it. Each context has its own challenges, so we’re trying to deal with that and add new, local sources as we go.
For instance, in data source terms, the organization is currently evaluating how to best include information generated by its monitoring team on an ongoing basis about local communities’ most pressing needs. Such needs include protection, food, and education.
The aim here is to improve the contextualization of assessments in crisis situations. But to do so, it will be necessary to build new processes for managing such confidential data on top of existing data privacy guidelines and policies.
In scaling terms, on the other hand, it is vital that care is taken, Orzoco believes. The point here is that if the system is rolled out either too early or too quickly and something goes wrong, it could have “profound consequences”.
Therefore, while it may have been quicker to simply roll out an out-of-the-box system, it was instead necessary to tailor the system to the organization’s particular requirements based on the specific frameworks and sources trusted by the humanitarian community.
The benefits VERA brings
As for the benefits VERA has generated so far, meanwhile, there are several. The first is cutting the time spent on information gathering.
For instance, analysts now spend between two and three days undertaking secondary research and analysis for Sudan-based reports compared to a former five or six, generating cost savings of about $1,500 per report. Producing security reports for on-the-ground teams in Colombia likewise takes 90% less time, cutting costs by around $2,000 per report. But Orozco points out:
It’s not just about spending less time collating information. It’s also about saving time to enable team members to make a better job of analyzing, contextualizing, and making judgements, which translate into better decision-making for the communities they serve. If there’s a lack of resources, the only way to cover more ground is to make people more effective and focus on higher value work.
The second benefit involves now being able to gather information from more data sources across a wider range of languages. Orozco explains:
If analysts are under time pressure, one option is to reduce the number of sources they use, which limits the breadth of the report. But VERA allows us to do more with less. So, we’re now testing to see what happens if we create contexualized reports at HQ level to support those countries that don’t have analysts in place. We couldn’t do that before.
As for the key considerations in getting an initiative like this right, the biggest, Orozco believes, is dealing with the AI trust – or lack of AI trust – issue. This means it is vital to be “completely open” with the team and to manage their expectations:
AI sometimes hallucinates or gives the wrong answers, and analysts are trained to be sceptical about sources. So, we didn’t try to push anyone into this. We just showed them how the system worked and let them check outputs. They were surprised by the depth of information, the citations, and that the system explicitly identifies gaps if it doesn’t have all the information available, which is useful. But we also worked with them through the entire design and build process, not just when it came to testing or evaluating the live feed. So, they gave feedback and helped improve the tool until they felt confident with it – and that made all the difference.