To provide more context around the Village Enterprise Development Impact Bond (DIB) results and foster greater understanding of its impact, we have put together a list of our most frequently asked questions about the DIB. IDinsight has also shared their own FAQ, which can be found here. For further information on the Village Enterprise DIB, please visit our DIB results web page and see the full report from IDinsight.
IDinsight found that households participating in the Village Enterprise program had about a 6% increase in consumption. What is important about these increases?
In general, solutions that have been rigorously proven to sustainably improve the wellbeing of the extreme poor have been elusive. However, a 2015 article in Science featured results from RCTs that evaluated graduation programming across six countries. These results offered groundbreaking evidence that the graduation program is an approach that can cost-effectively improve the well-being of the extreme poor. The consumption effect seen in that paper was 5% of the control group’s mean. That impact over time is meaningful for extremely poor families as it could mean being able to have enough to eat for their family daily, purchase basic necessities, and increase their overall well-being. Village Enterprise wanted to study the impacts of our graduation program to scientifically improve its effectiveness, use the results to better our programs for our entrepreneurs, and push the entire poverty alleviation sector forward to drive accountability and impact for those living in extreme poverty.
The RCT from IDinsight found that the Village Enterprise program outperformed the RCTs that generated tremendous excitement in the famous six country RCTs published in Science with a consumption effect that is 6.3% of the control group mean (compared to the 5% of the six country control group mean). As mentioned above, this increase is meaningful for extremely poor families and it could mean they have enough food to eat, including healthier and more nutritious meals. In Kenya, this meant increases in green maize, beef, fish, maize flour, and chicken for participating households. In Uganda, households increased consumption of chicken, fish, and tomatoes.
These results were achieved in the context of a significantly lighter touch graduation program, which shows that a lighter touch model when implemented extremely well can match and even outperform the costlier earlier models. This has significant implications for benefit-cost ratios, with Village Enterprise’s RCT now demonstrating the highest benefit-cost ratio when compared to the Science RCTs. The highest benefit-cost ratio in the Science article was 88% from a graduation implementation in India. Village Enterprise achieved a benefit-cost ratio of 140%.
Why is increasing cost effectiveness important?
Increasing cost effectiveness is important as many countries are resource constrained. In fact, even before the pandemic, lower-income nations faced an average annual funding gap of USD 2.5 trillion to reach the Sustainable Development Goals. We view our results and the three-year benefit-cost ratio of 140% as significant progress in terms of the direction the poverty alleviation sector needs to be heading in to improve the wellbeing of the extreme poor at scale. We also believe that we will improve the cost effectiveness of our program in the years ahead as we continue to innovate with the explicit goal of increasing its impact for families living in extreme poverty.
When did the RCT take place? Was it during the pandemic?
The study took place from November 2017 to August 2021. As a result, these outcomes were achieved even in the context of the Covid-19 pandemic. While the final three of seven program cohorts received ongoing mentoring during the pandemic, the first four cohorts had graduated from our program before the onset of the pandemic. The end-line data collection took place one year into the pandemic, six months after the last cohort graduated and 2.5 years after the first cohort had graduated from the program. The RCT showed sustained impact, despite the challenges of the pandemic, between that first and the final cohort.
We believe that this is an important finding as the RCT offers new evidence on the ability of graduation programming to sustain impact over many years and build resilience among participants. Especially in the Sub-Saharan Africa context where families are faced with ongoing shocks, successfully building resilience is considered to be a foundational need for creating sustainable impact. IDinsight estimates the lifetime impact of the Village Enterprise program at $21.06 million.
Village Enterprise’s pre-pandemic impact data found annual household per-capita consumption and expenditure increased by 71%. Why is this increase larger than the 6% increase in consumption and the 6% increase in spending that IDinsight found?
There are a number of reasons that could account for the difference in increases in consumption and expenditure measured by Village Enterprise’s Monitoring and Evaluation (M&E) and by the RCT. Here are some of the factors:
-Our RCT used an intention to treat estimates—this means it measured impact amongst everyone who was offered the program (whether or not they accept participation), whereas Village Enterprise’s M&E looks only at those that successfully completed the program.
-Village Enterprise’s pre/post evaluation does not account for external factors that may also contribute to income growth for our program participants. The RCT was designed to exclude external factors and tells us only what the attributable change was.
-Our internal data collection before Covid-19 showed the DIB was progressing well in terms of the goal of elevating the extremely poor in rural Africa out of poverty. We believe that if our results had been evaluated outside of the context of the pandemic— if there had been an evaluation of only the first few cohorts before the pandemic began or if the pandemic had not taken place—our RCT results would have been higher. Our entrepreneurs in rural Uganda suffered from strict lockdowns multiple times with months of closed borders, suspended public transport, restricted cross-district movement, and imposed nationwide curfews. The evaluators noted that “conducting fieldwork during the Covid-19 pandemic exacerbated some technical risks. In particular, it may have affected the generalizability of the findings to a non-pandemic context.”
-We also recognize there could be a response bias that is greater when our entrepreneurs interact with Village Enterprise enumerators than when they interact with entirely independent enumerators. However, to ensure Village Enterprise is following best practices and to control for this possible response bias, the enumerators who work with Village Enterprise are not our Business Mentors or Field Associates or any other staff members who work closely with our entrepreneurs.
Why were the outcomes of the DIB measured through a randomized controlled trial (RCT)?
This impact bond was meant to reward improvements in income. To approximate income as precisely as possible, the project measured the two uses of income at the household level: consumption and assets. By doing so, and comparing the results with those of a control group as part of the RCT, the impact of the Village Enterprise intervention on income could be approximated.
What do these results mean for the future of results-based financing and the poverty alleviation sector?
These results provide proof of concept that it is possible to commission service provision on the basis of a predefined fixed price-per-outcome, that it is possible to do so even for outcomes directly targeting poverty alleviation, and that doing so incentivizes innovation and accountability, and provides space for the same. Village Enterprise believes this successful proof of concept can provide commissioners, service providers, and investors, where applicable, evidence and inspiration necessary to consider using outcomes based contracting mechanisms in their own portfolios. We believe further growth of the outcomes-based sector will make current development dollars more effective by increasing accountability to impact, incentivizing and providing space for innovation and iteration needed to cost-effectively increase the impact of programming, and channeling dollars to scale solutions that work. We believe more effective use of current development dollars may inspire new funders to enter the sector, and also that the guarantee of predefined fixed price-per-outcomes could achieve the same, thus increasing the overall pool of funding available for outcomes based funding and poverty alleviation. We believe increasing the effectiveness of current aid as well as increasing the total amount of aid available are both necessary to end extreme poverty globally.
How were the outcomes for this DIB selected?
The goal was for the payment metric to be closely tied to the ultimate impact and to be applicable to a wider variety of livelihoods or income-generating projects. For this reason, this DIB rewarded improvements in income as it is closely tied to poverty graduation and is an outcome that a large portfolio of livelihoods, income-generation and workforce development programs share. Measuring income in poor, low-data contexts is difficult and so the two uses of income at the household level, using consumption and assets, were used to approximate income as closely as possible.
How were the outcome prices determined in the DIB?
To determine the specific price per unit of outcome, there were two key considerations:
1. Capturing the social value generated: the price paid for an outcome should not exceed the social benefits created. In this case, a conservative view on social benefits would correspond to the incremental income generated for the treated households, as captured by the payment metrics ($1 of outcome payment for $1 of incremental income). This is a conservative estimate because the poverty Graduation Model’s theory of change also aims to build social capital, increase financial literacy, and build business skills in a way that the monetary gains do not necessarily capture. Therefore, the outcome funders were confident that the payment formula was reasonable from this perspective.
2. Sufficient incentives to encourage progress and compensate for risks: The price per unit determines the intensity of incentives and the effort required of investors and service providers to achieve results. The outcome funders wanted to ensure that the price is such that with the expected results and the planned program size, Village Enterprise and its investors would receive sufficient compensation for the risk being taken and sufficient incentives to improve performance. During design, the group built several simulations using the past RCT and a financial model that confirmed–using the proposed payment formula–that Village Enterprise and the investors could earn a reasonable return when compared to relevant benchmarks of other impact bonds and impact investing opportunities.
Accounting for these considerations, and the intention to pay for the overall increase in income, there was a predefined fixed price-per-outcome of one dollar payment for every one dollar increase in household consumption. Since the program’s impact in consumption may continue, the impact in assets was used as a proxy of the capacity of the household to continue to generate income and, therefore, consume into the future.
How were the targets set for the DIB?
Household targets: The minimum number of households Village Enterprise committed to serving during the project was 12,660 and based on internal capacity assessments we set the target number of households that we would serve at 13,830. Village Enterprise ended up exceeding this target by serving 14,100 households.
Consumption targets: At the onset of the project, Village Enterprise believed it was realistic that we could increase consumption impacts from our first RCT conducted in Uganda by 15%. We outperformed this target by 10X, as DIB project-level consumption impacts were 151% higher than the first RCT. This increase was driven by impacts in Kenya.
Where can I learn more about the Village Enterprise DIB?
See the full report from IDinsight and learn more on the Village Enterprise DIB results web page. The DIB results were also featured in Devex, and IDinsight has produced their own FAQ.