How MobileAid & Machine Learning-based Targeting can Complement Existing Social Protection Programs
We propose a new paradigm for aid delivery, combining 1) ML targeting with 2) recipient self-enrollment, and 3) mobile money payments
This post was written by Anya Marchenko (CEGA) and Han Sheng Chia (GiveDirectly), both of whom contribute to CEGA’s Targeting Aid Better initiative.
Summary
In this blog, we propose a new paradigm for delivering social protections, which we call MobileAid. This approach combines (1) machine learning-based targeting with (2) recipient self-enrollment and (3) contactless delivery via mobile money. We discuss the lessons learned from deploying MobileAid in Togo, and describe how governments and NGOs can use MobileAid to complement their existing social protection systems.
Motivation
The COVID-19 pandemic has pushed an estimated 124 million people into extreme poverty globally, the first increase in extreme poverty in 20 years, exposing limitations in current approaches to delivering aid to the poor. Governments and humanitarian aid professionals are recognizing that the scale of the economic fallout from COVID-19 may simply be too great to address rapidly using traditional methods, leaving many without access to critical support.
Many social protection programs require expensive and time consuming efforts to assess beneficiaries’ needs, and then enroll those who need support the most. Asking hundreds of field officers to traverse the countryside, collecting detailed information on household or individual characteristics and registering those who meet a set of criteria, takes significant resources and challenges social distancing best practices. While it is possible to identify families living below a certain threshold using existing national rosters, many low-income countries and development institutions lack the resources or capacity to maintain — let alone expand — up-to-date, national registries.
While these weaknesses in delivering aid are exacerbated during the pandemic, they are not new, and will continue to persist beyond our current crisis. With more people falling below the poverty line, and large scale natural disasters and migration crises on the rise, governments and aid institutions need new tools to reach more people faster.
Introducing the MobileAid approach
To fill this gap, CEGA researchers are working with GiveDirectly (GD), Innovations for Poverty Action (IPA), and the Government of Togo (GoT) to deploy a new approach to targeting aid in Togo — a country of 8 million where 90% of households have access to a cell phone. This approach (1) leverages machine learning and already available, ubiquitous data sources such as satellite imagery and cell phone metadata to identify the extreme poor, and then (2) invites people to self-enroll via by simply dialing *855# and answering a few short questions. Finally, (3) eligible applicants are paid via mobile money. In this blog, we will refer to the coupling of this innovative type of targeting with self-enrollment and mobile money as the “MobileAid” approach.¹
To date, the Government of Togo has been able to use MobileAid to remotely identify, enroll and pay 35,000 individuals living under $1.25 a day in Togo. The approach is now being scaled to pay more than 110,000 individuals with emergency cash transfers. The three step process of targeting, self-enrollment and payments is illustrated below.
Of course, machine learning-based targeting does not solve the problem of how to disburse funds. After identifying who is in need, how can they be reached without making contact with beneficiaries? To solve this problem, GiveDirectly and the Government of Togo deployed a self-enrollment tool that applicants could access from anywhere with cell service. Applicants simply dial *855# on their phone and key in their ID information. On the backend, the self-enrollment tool matches applicants’ IDs and phone numbers against the poverty scores that are determined by the machine learning algorithm. This platform integrates with mobile money providers, enabling eligible beneficiaries to be paid instantly, automatically, and remotely.
You can read a short summary of how this program is being implemented in Togo, written by Berkeley Professor and CEGA Faculty Co-Director Joshua Blumenstock, here.
What might MobileAid mean for existing aid and social protection approaches?
Low-income countries spend an average of 1.5% of GDP on social protection programs such as pensions, school feeding programs, insurance schemes, and cash transfers. But cracks in the existing social safety net are often too big, and many eligible people fall through. There are significant gaps in program coverage, which is especially pronounced in low-income countries, where only 18% of the poorest quintile are covered by existing social safety net programs. And these gaps are not just a problem of funds. Benefit leakage due to poor targeting contributes to limited coverage of the poor. For example, in programs in South Africa and Malawi, more than 30–40% of beneficiaries are not even classified as poor (“Realizing the Full Potential of Social Safety Nets in Africa”, World Bank, pg 73).
So in both pandemic and non-pandemic times, social safety net programs are often unable to reach the people they’re intended to help. How could the MobileAid model fit in?
One way to think about MobileAid vis-à-vis current approaches is to consider complementarity — that is, what are the strengths and weaknesses of each approach, and can MobileAid reach people that existing government social protection programs miss?
In the rest of the post, we will lay out the strengths and weaknesses of the MobileAid approach, and then review several ideas for how this targeting can be integrated into existing social protection systems to create a more rapidly scalable and comprehensive safety net.
Strengths and weakness of MobileAid versus traditional targeting and aid delivery methods
This new MobileAid approach has two major strengths: it can identify and reach people rapidly (speed), and it can do so on the order of millions of people (scale). For example, Blumenstock et al. (2015) were able to estimate poverty across Rwanda using machine learning algorithms applied to mobile phone data, and do so cheaper and faster:
“In developing economies, where traditional sources of population data are scarce but mobile phones are increasingly common, these methods may provide a cost-effective option for measuring population characteristics. Whereas a typical national household survey costs more than $1 million and requires 12 to 18 months to complete (27), the phone survey we conducted cost only $12,000 and took 4 weeks to administer.”
It should be noted that while collecting the phone survey for machine learning training data is inexpensive and rapid (taking 4 weeks in Rwanda and 2 weeks in Togo), setting up the agreements, data governance frameworks and self-enrollment technology ahead of a crisis is crucial to realizing the gains in speed from this approach.
At the same time, the MobileAid approach may not perform as well as certain in-person methods for reaching specific vulnerable groups (e.g., households without SIM cards or the elderly). As with other proxy-means testing based models, the MobileAid approach should be regularly assessed to ensure that prediction biases are well accounted for and mitigated against.
Additionally, the MobileAid model may also suffer in community comprehension. Enrolling in-person provides more opportunity to explain a social protection program to potential beneficiaries, building trust. This is important because strong community comprehension and trust can increase both satisfaction and participation, creating a greater sense of fairness amongst beneficiaries and non-beneficiaries. The dynamic between communities and aid providers deserves further research as both in-person and the MobileAid approach may have different political and communal satisfaction impacts amongst both beneficiaries and non-beneficiaries. To that end, GiveDirectly is in contact with more than 100 community leaders in Togo, and together with the government runs follow up calls with applicants to gather input from communities during the pandemic. The organization and researchers will continue to conduct more assessments and share learnings in this emerging field.
How MobileAid can complement traditional aid and social protection schemes
MobileAid in emergency settings
Given that in-person validation provides valuable personal touchpoints, and MobileAid can be deployed quickly and at scale, countries could supplement one with the other. Emergencies or large scale shocks like famines and flooding may be the most natural use cases for the MobileAid approach to deliver significant early value.
During a crisis, time is of the essence, and policymakers may value the speed that the MobileAid model brings. As the machine learning algorithm screens a large swath of a population at high speed, producing a report of which individuals should be prioritized for aid, ground teams can enter afterward to enroll those missed and further community engagement. On the flip side, if machine learning algorithms over enroll beneficiaries by including those who are not eligible, ground teams could step in to narrow the pool of beneficiaries. In this example, the MobileAid approach is simply a “fast track”, or the first wave of targeting and enrollments, before other help can be mobilized.
In more resource-constrained contexts, instead of sending field teams out, countries can use existing structures such as municipal offices, post offices, or schools to offer applicants options for alternative ways to enroll. However, such approaches may suffer from a reduced ability to verify an applicant’s claims of need or fulfillment of the eligibility criteria — something at which the MobileAid approach, with its reliance on passively collected cell phone data, excels.
MobileAid in non-emergency settings
Even in non-emergency settings, countries with large geographic areas and a lower capacity to run field programs, such as the Democratic Republic of the Congo, may want to utilize the MobileAid approach to augment their regular safety net programs. Conversely, a country with a smaller population and smaller geographic area may find less value in MobileAid.
Part of identifying complementarities also involves understanding the actors available in the ecosystem under each context. Non Governmental Organizations (NGOs) can be a valuable supplement to government programs when the state has a strong safety net. But in settings where a state has less capacity to execute, NGOs may form the bulk of the response, and would benefit from leveraging a more “out of the box” machine learning technology to disburse aid to those in need. It is important to note that a strong foundational ID system, which can be used by all actors, would greatly help combine and coordinate the targeting and enrollment across different actors. Such a system would support a wide range of functions such as deduplication, identity verification and important Know Your Customer (KYC) compliance requirements.
Togo’s COVID-19 pandemic cash transfer program, detailed in the box below, provides a helpful case study into how different actors can play complementary roles during a crisis.
Whether a government relies on their own capacity or partner NGOs, or if it utilizes the MobileAid approach or more traditional methods, it is important to remember that increasing the entry points into the safety net, being flexible in the methods, and finding complementarities between different programs, means more people in need can be reached.
Community comprehension, extra verification, and other complementarities
While the approach in Togo combines the machine learning technology with self-enrollment platforms, there are other ways to mix and match targeting and enrollment technologies to strengthen social protection programs. For example, field officers can be equipped with smartphones with preloaded poverty scores generated by the machine learning technology, and enroll beneficiaries in the village face-to-face based on these scores. This could increase community comprehension while saving time otherwise spent on evaluation. Taking the responsibility of making eligibility determinations away from the field officer, and focusing their role on community comprehension, can also reduce risk of enrollment fraud.
For countries more hesitant about directly incorporating MobileAid technology, machine learning-based targeting could be used as an extra layer of verification on top of a field officer’s evaluation. These predictions could be used to identify individuals that a field officer may have missed, or identify systematic fraud or errors in a field officer’s screening approach. This may be useful in a non-crisis setting where a government or NGO is under less time pressure, so they may trade down the speed that the MobileAid approach brings for increased community comprehension and inclusiveness.
While strategically combining approaches can open up new and exciting possibilities, it should be done with a holistic view of the end-to-end cycle of targeting, enrollment, and payment. Focusing on just the machine learning aspect (i.e., the novel targeting methodologies) without considering the self-enrollment and payment parts of the MobileAid approach can lead to issues. For example, in GiveDirectly’s early pilots of contactless targeting and enrollment in Uganda and Liberia, it became evident that reaching out to previously identified individuals via SMS, and simply expecting them to enroll, was a poor way to drive enrollment. An SMS, even from the telecommunications company, was not sufficient to generate trust with beneficiaries. It was only by coupling targeting methodologies and SMS prompts with community mobilization efforts — such as radio advertisements, outreach to local leaders, and public flyers — that GiveDirectly achieved higher rates of enrollment.
Conclusion
The MobileAid approach is promising in its ability to identify vulnerable people and pay them at tremendous speed and scale. Moreover, it addresses a significant pain point facing policy makers and aid providers: How do we deliver aid contactlessly during a large scale crisis such as COVID-19?
To determine how this approach could fit in with existing safety nets in the long term, policy makers should carefully consider (1) the strengths and weaknesses of available models, and how the MobileAid approach can complement existing methods and actors; (2) the context in which they operate (emergency or non-emergency); and (3) how various elements in the targeting, enrollment, and payment cycle can be strategically combined.
Large scale emergencies, such as COVID-19 or a natural disaster, may provide the most immediate and apparent use case for the MobileAid approach. Even in the absence of an emergency, the MobileAid approach can provide another layer of eligibility verification, contactless enrollment, and other benefits. However, policymakers should keep in mind how the prioritization of different elements, such as speed or community comprehension, determines the design of the program. Incorporating existing technologies (such as available enrollment and payment tools), as well as recognizing who the available actors are (governmental or non-governmental) is key to understanding how the MobileAid approach can be incorporated.
Ultimately, when all parties — government, NGOs, researchers, and funders — thoughtfully apply and integrate MobileAid and traditional approaches, the safety net can be rapidly enlarged, providing more pathways for vulnerable individuals to receive benefits.
Financial Support
Thank you to our funding partners for generously supporting this important work.
[1] The intuition behind this approach is that poor regions often look different from wealthy regions from above — they have less well paved roads, different crop patterns, and homes made of inexpensive material. Similarly, wealthier individuals and poorer individuals use their phones differently. Wealthier individuals spend more on data, airtime, and use more mobile money (Blumenstock et al. 2015).
