What if we could take what was working one place and spread it globally? Wouldn't progress happen so much faster? Luckily, many are working on just that. For example, the Center for Global Development puts out a regular publication on "Millions saved," cataloging successes in health delivery from around the world. There are websites and entire organizations dedicated to identifying practices and making them more visible globally.
But here's the catch: few practices highlighted in these resources seem to catch on and spread widely. Those that do are often supported by a lot of hand-holding and financial resources, and potentially even a lot of nudging from donors.
But why is this? Is it that hard to figure out what works and what doesn't?
Unless you're talking about a drug cocktail or a vaccine that has essentially the same effect on everyone who receives it, heck yeah it is!!
I'll take examples from Bangladesh and India of programs that I know relatively well and are interested in having others adapt their model. The first is BRAC's targeting ultra-poor (TUP) program, which I believe is probably a key model for wide-scale replication if the world is to eliminate extreme poverty by 2030 (which is one of the goals proposed by the United Nations in the Sustainable Development Goals). For reference, definitions of "extreme" or "ultra" poverty vary and obviously depend on context, but generally refer to those who live on under $0.75 a day and/or routinely get less than 80% of their energy requirements despite spending 80% of their income on food. This is the reality for over 400 million people in the world today.
Several years ago, BRAC developed a "graduation model" to tackle ultra-poverty that has now been replicated around the world. In short, it's a combination of a few different interventions: a productive asset (usually livestock) with intensive practical training, weekly cash stipends, support to start saving, health care, and social integration (for a comprehensive description, see this briefing note on the program or the great video below). After receiving ongoing support from BRAC for two years, most participants "graduate," i.e. their life has improved in meaningful ways, including: they eat 2+ meals a day, they have 3+ income sources, they have a vegetable garden, and their children attend school. And by most, I mean 95% of participants!
That's exciting. What's even more exciting is what happens after the program ends and the support from BRAC ceases. Their income continues to increase! So instead of sliding back into extreme poverty, most households continue to thrive. The cost per household in Bangladesh is about $320 over two years, hardly an unreasonable investment for a government or large donor looking to alleviate poverty.
BRAC, in partnership with CGAP and the Ford Foundation, helped others implement the BRAC "targeting ultra-poor" (TUP) model in many other countries - Haiti, Yemen, India and more. Now that the goal of eradication extreme poverty is becoming a priority for the World Bank and others, there's a rich body of research and experience on what works.
But regardless of how documented and studied the program is, replicating successes from one place to another is always tricky. It's easy to say "this worked!" but much harder to explain why. Ultimately, a lot comes down to understanding the context, people, and processes, and the interactions of all three.
With the help of many world-class researchers, BRAC developed the rigorous evidence to say what it had believed all along: that while certainly material assets are a key dimension of poverty, they are only a part of the whole picture. In the Economist article summarizing the results from Bandhan's implementation of BRAC's graduation model in West Bengal, the journalist writes:
"Well after the financial help and hand-holding had stopped, the families of those who had been randomly chosen for the Bandhan programme were eating 15% more, earning 20% more each month and skipping fewer meals than people in a comparison group. They were also saving a lot. The effects were so large and persistent that they could not be attributed to the direct effects of the grants: people could not have sold enough milk, eggs or meat to explain the income gains. .....She [Esther Duflo] argues that it provided these extremely poor people with the mental space to think about more than just scraping by......Bandhan injected a dose of optimism (emphasis added)."
"Injecting optimism" is a MUCH taller order than giving someone a cow, some technical training, weekly cash stipends, etc. It's harder to measure, harder to codify into an operational manual, harder to explain to staff.
One of the contributors to its success is the engagement of the community in the process from the beginning. While the program has a objective set of criteria on what it means to be ultra-poor, and enough staff on the ground to make it easy for select target households that fit, it doesn't do it that way. Instead, the local staff asks the village leaders to come together for a meeting, to map out the community on the ground, and to start identifying households that are ultra-poor. We call this process a "participatory rural assessment," and it's a widely-known tool for community-based organizations.
Using supplementary door-to-door surveys and verification methods, the BRAC team then comes to a consensus with the village leaders about who will be selected for this program. In the process, the leaders also publicly promise to look out for these households and support their well-being.
While participatory rural assessment seems to be a simple activity, one that could easily be overlooked in the interest of getting started with the program enrollment, we have found it to be a powerful addition to ensure the program's success. First, it creates a sense of local transparency: BRAC shares its objectives with the community and engages in a dialog. Later no one will ask, "Why did that household get it and not me?" It also creates a local sense of responsibility in the community's leadership. They chose these households and pledged to support the process. This sets up an opportunity for more social integration of the ultra-poor households. In rural villages, it also is the best insurance against their assets (cows, goats, chickens, etc.) getting stolen. If this happens, BRAC staff can come and hold the leaders more accountable. Part of this is the process, and part of it is BRAC's pre-existing relationship with the community, and long-standing presence in Bangladesh more generally. There's a long history here, a strong sense of trust and continuity, and a mutual accountability. Many of the villages also have other BRAC programs, such as microfinance village organizations, primary schools, and community health workers, that magnify these effects even more.
For an organization adopting the the TUP method, I can see a few challenges to making the participatory rural assessment effective: first, it's possible that the organization doesn't see the inherent value in engaging the community (many don't), and secondly, it's possible that it doesn't have a deep enough relationship with the community to generate productive dialog. In these cases, what should be done? If it takes time to develop that relationship, will the organization stick it out and take the time to build it? Should BRAC allow all deviations, or label parts of the model "mandatory" and other parts "suggested"? These are difficult policy decisions.
The success of most programs is a result of this type of attention to detail and a deep understanding of the problems. Often there's an underlying philosophy or implicit theory of change shared by the implementing team that enables their effectiveness. When transmitting the model to another organization, these dimensions are just as important as the operational checklists, perhaps more, in cases where the context is radically different. There are examples of organizations who have done this well, though many have failed. One well-known case is the one of the many companies that attempted to adopt "the Toyota Way," only to find that it did not yield the same results that it did for Toyota. Researchers at Harvard Business School write,
"Part of the problem is that most outsiders have focused on Toyota’s tools and tactics - kanban pull systems, cords, production cells, and the lokE - and not on its basic set of operating principles."
Unfortunately, just knowing that doesn't solve the problem:
"It is one thing to realize that the Toyota Production System (TPS) is a system of nested experiments through which operations are constantly improved. It is another to have an organization in which employees and managers at all levels in all functions are able to live those principles and teach others to apply them. Decoding the DNA of Toyota doesn’t mean that you can replicate it (emphasis added)."
My point is not to focus on BRAC's targeting ultra-poor program and specific challenges facing its replication. It's simply an illustration of common problems facing most development organizations that have developed a "model that works." We're told to simplify the model, find the core components that make it successful, and try to bottle it up into something that can be sold widely. But there are many dangers to this.
Gram Vikas, a fascinating Indian organization, has struggled with the same challenge for years. Despite having a well-articulated model and set of principles, they've not been able to transmit it completely to other organizations. They have an "all-or-nothing" mentality when it comes to working with rural communities to create latrines and water taps: either every household gets one, or no one does. They'll wait for years while the community argues and debates and collects money (there's a fixed rate of how much must be contributed per household to the maintenance fund, but the wealthier families will need to subsidize the poorer ones). Why? Because their goal is as much about social equality as it is about water and sanitation. They know that there are easier ways to build the hardware, but that's not their goal. They believe there's a transformation that happens in the village's social fabric when everyone has equal access to water and sanitation. If only members of the social elite and higher castes get access, it reinforces social inequality instead.
But convincing other organizations to replicate this model and have the patience to wait for communities to make up their minds has proven very difficult. As written in a recent case study on Gram Vikas, one "senior staff person said that one of the challenges was teaching other people “the ropes” of the model, but that that wasn’t the most difficult part — it seemed the challenge was more ideological — they had to see the value in the approach for themselves and the difference that it made. The challenge was for them to 'get a taste of success' that would inspire them to stick to the 100% requirement (emphasis added)."
How can one inject a new implementer with a dose of optimism? Should you choose a partner based on technical skill or on the organizational philosophy? Replication of complex delivery models, such as Gram Vikas's 100% participation or BRAC's targeting ultra-poor program, is possible and urgently required to tackle some of the world's most pressing problems. To do this effectively requires new practitioners to gain deep immersion in the philosophy and underlying principles. In addition, it's important that leadership and staff "drink the KoolAid" and develop faith in the tenets of the model. If there are adaptations required, it's important for the team to discuss how they align with the principles. Better to spend time discussing the "why's" than rushing to program design and implementation.
Have you seen replication efforts succeed or fail, for some of the reasons discussed here? How have you approached replication--either on the receiving or giving side?
But here's the catch: few practices highlighted in these resources seem to catch on and spread widely. Those that do are often supported by a lot of hand-holding and financial resources, and potentially even a lot of nudging from donors.
But why is this? Is it that hard to figure out what works and what doesn't?
Unless you're talking about a drug cocktail or a vaccine that has essentially the same effect on everyone who receives it, heck yeah it is!!
I'll take examples from Bangladesh and India of programs that I know relatively well and are interested in having others adapt their model. The first is BRAC's targeting ultra-poor (TUP) program, which I believe is probably a key model for wide-scale replication if the world is to eliminate extreme poverty by 2030 (which is one of the goals proposed by the United Nations in the Sustainable Development Goals). For reference, definitions of "extreme" or "ultra" poverty vary and obviously depend on context, but generally refer to those who live on under $0.75 a day and/or routinely get less than 80% of their energy requirements despite spending 80% of their income on food. This is the reality for over 400 million people in the world today.
Several years ago, BRAC developed a "graduation model" to tackle ultra-poverty that has now been replicated around the world. In short, it's a combination of a few different interventions: a productive asset (usually livestock) with intensive practical training, weekly cash stipends, support to start saving, health care, and social integration (for a comprehensive description, see this briefing note on the program or the great video below). After receiving ongoing support from BRAC for two years, most participants "graduate," i.e. their life has improved in meaningful ways, including: they eat 2+ meals a day, they have 3+ income sources, they have a vegetable garden, and their children attend school. And by most, I mean 95% of participants!
That's exciting. What's even more exciting is what happens after the program ends and the support from BRAC ceases. Their income continues to increase! So instead of sliding back into extreme poverty, most households continue to thrive. The cost per household in Bangladesh is about $320 over two years, hardly an unreasonable investment for a government or large donor looking to alleviate poverty.
BRAC, in partnership with CGAP and the Ford Foundation, helped others implement the BRAC "targeting ultra-poor" (TUP) model in many other countries - Haiti, Yemen, India and more. Now that the goal of eradication extreme poverty is becoming a priority for the World Bank and others, there's a rich body of research and experience on what works.
But regardless of how documented and studied the program is, replicating successes from one place to another is always tricky. It's easy to say "this worked!" but much harder to explain why. Ultimately, a lot comes down to understanding the context, people, and processes, and the interactions of all three.
With the help of many world-class researchers, BRAC developed the rigorous evidence to say what it had believed all along: that while certainly material assets are a key dimension of poverty, they are only a part of the whole picture. In the Economist article summarizing the results from Bandhan's implementation of BRAC's graduation model in West Bengal, the journalist writes:
"Well after the financial help and hand-holding had stopped, the families of those who had been randomly chosen for the Bandhan programme were eating 15% more, earning 20% more each month and skipping fewer meals than people in a comparison group. They were also saving a lot. The effects were so large and persistent that they could not be attributed to the direct effects of the grants: people could not have sold enough milk, eggs or meat to explain the income gains. .....She [Esther Duflo] argues that it provided these extremely poor people with the mental space to think about more than just scraping by......Bandhan injected a dose of optimism (emphasis added)."
"Injecting optimism" is a MUCH taller order than giving someone a cow, some technical training, weekly cash stipends, etc. It's harder to measure, harder to codify into an operational manual, harder to explain to staff.
One of the contributors to its success is the engagement of the community in the process from the beginning. While the program has a objective set of criteria on what it means to be ultra-poor, and enough staff on the ground to make it easy for select target households that fit, it doesn't do it that way. Instead, the local staff asks the village leaders to come together for a meeting, to map out the community on the ground, and to start identifying households that are ultra-poor. We call this process a "participatory rural assessment," and it's a widely-known tool for community-based organizations.
Using supplementary door-to-door surveys and verification methods, the BRAC team then comes to a consensus with the village leaders about who will be selected for this program. In the process, the leaders also publicly promise to look out for these households and support their well-being.
 |
| A community lays out all the households in their villages. They sketch it in the mud before drawing it out on a piece of paper with more details on the economic and social status of households. Baniyachong, Bangladesh. |
For an organization adopting the the TUP method, I can see a few challenges to making the participatory rural assessment effective: first, it's possible that the organization doesn't see the inherent value in engaging the community (many don't), and secondly, it's possible that it doesn't have a deep enough relationship with the community to generate productive dialog. In these cases, what should be done? If it takes time to develop that relationship, will the organization stick it out and take the time to build it? Should BRAC allow all deviations, or label parts of the model "mandatory" and other parts "suggested"? These are difficult policy decisions.
 |
| The woman here was one of the first participants in BRAC's ultra-poverty programme, over ten years ago. She's built this house for herself and has adopted a "son" who stays with her. On her right is the village leader and the local BRAC program organizer. She enjoys giving advice to others in the community. |
The success of most programs is a result of this type of attention to detail and a deep understanding of the problems. Often there's an underlying philosophy or implicit theory of change shared by the implementing team that enables their effectiveness. When transmitting the model to another organization, these dimensions are just as important as the operational checklists, perhaps more, in cases where the context is radically different. There are examples of organizations who have done this well, though many have failed. One well-known case is the one of the many companies that attempted to adopt "the Toyota Way," only to find that it did not yield the same results that it did for Toyota. Researchers at Harvard Business School write,
"Part of the problem is that most outsiders have focused on Toyota’s tools and tactics - kanban pull systems, cords, production cells, and the lokE - and not on its basic set of operating principles."
Unfortunately, just knowing that doesn't solve the problem:
"It is one thing to realize that the Toyota Production System (TPS) is a system of nested experiments through which operations are constantly improved. It is another to have an organization in which employees and managers at all levels in all functions are able to live those principles and teach others to apply them. Decoding the DNA of Toyota doesn’t mean that you can replicate it (emphasis added)."
My point is not to focus on BRAC's targeting ultra-poor program and specific challenges facing its replication. It's simply an illustration of common problems facing most development organizations that have developed a "model that works." We're told to simplify the model, find the core components that make it successful, and try to bottle it up into something that can be sold widely. But there are many dangers to this.
Gram Vikas, a fascinating Indian organization, has struggled with the same challenge for years. Despite having a well-articulated model and set of principles, they've not been able to transmit it completely to other organizations. They have an "all-or-nothing" mentality when it comes to working with rural communities to create latrines and water taps: either every household gets one, or no one does. They'll wait for years while the community argues and debates and collects money (there's a fixed rate of how much must be contributed per household to the maintenance fund, but the wealthier families will need to subsidize the poorer ones). Why? Because their goal is as much about social equality as it is about water and sanitation. They know that there are easier ways to build the hardware, but that's not their goal. They believe there's a transformation that happens in the village's social fabric when everyone has equal access to water and sanitation. If only members of the social elite and higher castes get access, it reinforces social inequality instead.
But convincing other organizations to replicate this model and have the patience to wait for communities to make up their minds has proven very difficult. As written in a recent case study on Gram Vikas, one "senior staff person said that one of the challenges was teaching other people “the ropes” of the model, but that that wasn’t the most difficult part — it seemed the challenge was more ideological — they had to see the value in the approach for themselves and the difference that it made. The challenge was for them to 'get a taste of success' that would inspire them to stick to the 100% requirement (emphasis added)."
How can one inject a new implementer with a dose of optimism? Should you choose a partner based on technical skill or on the organizational philosophy? Replication of complex delivery models, such as Gram Vikas's 100% participation or BRAC's targeting ultra-poor program, is possible and urgently required to tackle some of the world's most pressing problems. To do this effectively requires new practitioners to gain deep immersion in the philosophy and underlying principles. In addition, it's important that leadership and staff "drink the KoolAid" and develop faith in the tenets of the model. If there are adaptations required, it's important for the team to discuss how they align with the principles. Better to spend time discussing the "why's" than rushing to program design and implementation.
Have you seen replication efforts succeed or fail, for some of the reasons discussed here? How have you approached replication--either on the receiving or giving side?
great post! and as you say the need for understanding is a prereq for replicating any kind of success, not 'just' in development. going to share this at work.
ReplyDeletevanessa