How can we decide if a policy is worth it? Research in development economics is painting a more accurate picture of the impacts of interventions, helping us to conduct better cost-benefit analyses.
When countries first introduced conditional cash transfer programmes in the 1990s, critics argued that they were just an expensive handout. But decades later, through research in countries around the world, we know better: impacts include improved health and school attendance for children, with welfare gains extending to adulthood. If policymakers had only looked at the short term costs, they might have scrapped one of the most successful poverty reduction programmes in history.
So, how does a government or organisation decide if a policy is ‘worth it’?
Often, they will conduct a cost-benefit analysis – adding up all the expected benefits and comparing them to the costs. This approach guides decisions across development, everywhere from private-sector investments to NGO projects and government budgets. In theory, it’s simple: if the benefits outweigh the costs, go ahead. If not, reconsider. But in practice, what counts as a ‘benefit’ is not always so straightforward – capturing and measuring all the benefits of a programme is easier said than done.
When benefits are underestimated or ignored, worthwhile policies might look unappealing on paper. On VoxDev, much of the research we have featured attempts to capture a wider range of benefits. This enables us to conduct more accurate cost-benefit analyses, that don’t simply rely on the most immediate or obvious impacts.
I will not focus on up-front financial costs in this post, since they are typically better understood. Although, even this assumption might not hold in developing countries, for example, land transport infrastructure costs are widely unavailable in low- and middle-income countries.
So for now, let’s focus on the broader impacts of policies, and how new evidence is changing our view of them. By capturing long-run earnings, multigenerational gains, community spillovers and environmental side effects, we can make better-informed policy choices in development.
Holistic cost-benefit analysis
Consider a cash transfer given to a poor individual in a rural village. A narrow evaluation might record the recipient’s food consumption one year later. If we stop there, we have far from a complete picture. The recipient lives in a household, participates in the local community, interacts with the environment around them, will continue to do so after the one year follow up survey, and could go on to have children.
In short, development policies do not generate benefits in a vacuum. So, what do we learn by building a more holistic picture? That’s exactly what research on VoxDev has done, and the following sections highlight some areas where we’re learning about benefits that we were previously missing.
Policies with long-term benefits
One major blind spot of traditional evaluations is the time horizon. Many programmes are assessed only on what they accomplish in the short run, as that’s when data is available, and often because of the obvious political incentives for shorter time horizons. Yet some interventions may show no immediate gains, even though their long-run effects are significant.
On VoxDev, we featured evidence on the long-term impacts of the US New Deal’s youth employment programme, the Civilian Conservation Corps (CCC), which provided young men with public employment during the Great Depression. Aizer, Early, Eli, Imbens, Lee, Lleras-Muney and Strand find that the CCC did not improve short-term labour market outcomes like immediate employment or wages for the enrolees. By that narrow metric, one might conclude the program wasn’t very effective.
However, by tracking CCC participants over the long-term, Aizer et al. find that they ended up taller, healthier, living longer, and earning more over their lifetimes. As they note in their VoxDev article, “our ability to track the health benefits of the programme over the long term changes our evaluation of its cost-effectiveness.” Given that benefits only became evident many years down the line, short-term evaluations would have missed the real payoff.
Research on VoxDev has documented long-term effects of development interventions in a number of settings:
- The long-term effects of conditional cash transfer programmes: Evidence from Colombia Attanasio, Cardona Sosa, Medina, Meghir, Posso & Taboada
- The impact of Mexico’s Progresa programme twenty years on Caridad Araujo & Macours
- The long-term effects of cash transfers: Mexico’s Progresa Parker & Vogl
- International migration's long-term impact: Evidence from a lottery Gibson, McKenzie, Rohorua & Stillman
As we’ll see next, some of those delayed payoffs even extend into the next generation.
Policies that benefit the next generation
Certain policies not only improve the lives of direct beneficiaries, but also benefit their children (and potentially grandchildren). These intergenerational effects were almost never included in traditional cost-benefit analyses, which typically stop at the outcomes for the individuals originally treated. Recent research, however, shows that we may be dramatically undervaluing interventions – especially in education and health – by ignoring how they echo down the family line.
One example comes from an education experiment in Ghana, where a cohort of students were offered secondary school scholarships. Duflo, Dupas, Kremer, Spelke and Walsh outlined the results from their 15-year RCT on VoxDev – they find that subsidising secondary education had huge benefits for both this generation and the next.
The young women who received a scholarship ended up completing more schooling, but also made different life choices – they had fewer unplanned pregnancies, less children and a significantly delayed age of marriage and first childbirth. When they did start having children, those children enjoyed markedly better early-life outcomes – 45% lower under-3 mortality rates and significantly higher cognitive scores. Accounting for these multigeneration benefits leads to much higher estimates of cost-effectiveness.
Other research on VoxDev has found intergeneration impacts in other settings:
- Deworming improves lives across generations Miguel, Ochieng & Walker
- Long-term and intergenerational benefits of early childhood health interventions Barham, Champion, Kagy, Kuhn, Turner & Hamadani
Household spillovers
Not all spillover benefits take decades to appear or require having children – many play out within the household and community right away. Social programmes often target one individual, but the effects on other members of the household can be substantial.
A study in Nigeria provides a compelling case. In their experiment, Carneiro, Rasul and Salvati evaluate an intervention that provided informational sessions and modest cash transfers to pregnant mothers in poor communities, aimed at improving their soon-to-be-born child’s health and nutrition during the critical early years of life. When the researchers followed up a few years later, they found improvements in the target children’s outcomes (better nutrition and health). But they also checked on the older and younger siblings of those target kids.
Many outcomes for the siblings had also improved, suggesting that accounting for these benefits dramatically increases the estimated returns to this early childhood intervention. In this case, Carneiro et al. find that the annualised rate of return rises ten-fold when including gains for older and younger siblings.
Other studies have explored impacts, positive and negative, on others in the household and community in a range of settings:
- The household benefits of an educated older sister Qureshi
- Realising childcare’s full potential in low- and middle-income countries Kabay & Smith
- Reducing child labour: Lessons from Bolivia’s legalisation experiment Lakdawala & Martínez Heredia
- The unintended impacts of formal credit programmes on social networks: Evidence from India Banerjee, Breza, Chandrasekhar, Duflo, Jackson & Kinnan
General equilibrium effects
Most cost-benefit analyses focus on the immediate effects of a programme, such as how many people received a job or a subsidy. But because policies don’t happen in isolation, some of the most challenging – yet important – impacts to account for are general equilibrium effects. These are the ripple effects that occur when a programme interacts with the broader economy, changing prices, wages or other market conditions. The true impact of a policy isn’t just about direct beneficiaries - it’s about how it changes the whole system.
Ignoring these can lead us to undervalue (or overvalue) a programme because we miss both how it affects people who are neither direct beneficiaries nor in the immediate vicinity, and how markets respond.
Franklin, Imbert, Abebe and Mejia-Mantilla examine a public works programme in Ethiopia. In this context, ignoring GE effects can vastly underestimate benefits. If one only considered the participating workers, the programme might look modestly beneficial. But considering the market-level impacts – mainly through higher private sector wages felt across the city, but also from improved services for the community – the programme is far more impactful in reducing poverty than a micro perspective would indicate.
Other research on VoxDev has shed light on these wider impacts from interventions that rarely appear in basic cost-benefit calculations:
- Examining the macro effects of MNREGA Cook & Shah
- Cash transfer programmes can stimulate the local economy: Evidence from Brazil Gerard, Naritomi & Silva
Importantly, accounting for GE effects is not just about positive spillovers – it’s also about recognising any offsetting or negative feedback effects.
- The price effects of cash transfer programmes: Insights from research Kandpal
- Agronomy training in Rwanda had negative spillovers Duflo, Keniston, Suri & Zipfel
General Equilibrium effects are particularly important to consider with projects implemented at scale, as John A. List discussed in Scaling policy ideas in developing countries.
Downstream environmental effects
Economic programmes and policies often have environmental spillovers, which standard cost-benefit analyses may overlook if they focus purely on economic outputs. Research on VoxDev shows that accounting for environmental effects can change our assessment of a programme’s desirability.
Indonesia’s flagship anti-poverty cash transfer program (PKH) provides support to poor households with no explicit environmental goals. Ferraro and Simorangkir 2022 found that villages receiving the cash transfers experienced around a 30% reduction in annual tree cover loss. In practical terms, this meant each treated village preserved about 5.5 more hectares of forest per year than it would have otherwise. Notably, about half of the avoided deforestation was in primary forests, indicating the programme helped protect critical old-growth areas.
In fact, the study estimates that the value of carbon emissions avoided by the drop in deforestation is of the same order of magnitude as the programme’s cost. In hindsight, if policymakers had known that cash transfers would cut deforestation (by easing the need to clear land for income and providing a safety net that substitutes for ‘rainy day’ logging), the programme’s perceived benefits would be even greater.
On the other hand, development policies can sometimes inadvertently harm the environment, imposing costs that should be counted. For example, work on VoxDev has shown how infrastructure projects have had negative downstream impacts:
- Constructing roads connected labour markets but increased harmful crop fires in rural India Garg, Jagnani & Pullabhotla
- The deforestation footprint of transportation infrastructure: Evidence from the Amazon Araujo, Assunção & Bragança
Better cost-benefit analyses
If a policy today sets the stage for a healthier, more productive population in 20–30 years, that is very much a benefit to a society’s future welfare and should influence decisions today, to ensure that we allocate resources to initiatives that truly maximise social welfare.
Failing to account for the long-term, the next generation, the household, the economy or the environment means that cost-benefit analyses (and the policymakers relying on them) under-invest in high-impact interventions. I would highly recommend reading the cited articles in more detail to learn exactly how economists are capturing these benefits.