This opinion piece was originally published on Apolitical by Alex Armand, NCID resident fellow and assistant professor at the University of Navarra. It reflects upon the investigation Identifying the effect of targeted money transfers on women’s empowerment. Apolitical is a global network for government, helping public servants find the ideas, people and partners they need to solve the hardest challenges facing our societies. Please visit Apolitical.co for further articles.

Achieving gender equality and empowering all women and girls is an ambitious goal, and one that is central to the UN Sustainable Development Goals’ post-2015 development agenda. An important domain in which it can be pursued is the family: it’s here that countless decisions affecting women’s futures are taken. Yet the role of women within dwellings’ walls represents a black box: women’s participation in household decisions is hard to observe, and even more arduous to quantify.

Given this measurement challenge, can we even set realistic targets for female empowerment within the family, let alone achieve them?

In the past few decades, measurement has mainly been done through survey questions on gendered participation in household decision-making. A typical set of questions asks respondents to identify the family member in charge of different decisions — for instance, who decides about food expenditures or about schooling. Higher female participation supposedly shows stronger empowerment.

However, reaching conclusions based on these measures can be misleading: social norms or other constraints can dictate certain answers, and indirect asks may not lead to answers that paint a true picture of reality.

Are empowerment programs failing?

Since the 1990s, women-targeted cash transfers have become a common tool policymakers in developing countries use to target female empowerment. A large number of income-supporting social programs have selected women to be the recipients of government money, including Mexico’s PROGRESA/Oportunidades (now relabelled as Prospera).

These interventions promote gender equality in the family by raising female control of household income. Many studies show the effectiveness of these programs in shifting diverse household outcomes in a substantive fashion. But effects on self-reported measures of women’s empowerment have hardly been observed.

Have these programs failed at empowering women? Or are we just failing at measuring it? To find answers, we need new and innovative ways of measuring female empowerment.

One such recent attempt comes from researchers at IIES Stockholm, University of Navarra and University College London. The new approach observes women’s choices in a lab setting, but facing real-life decisions. A participating woman receives an offer: a financial transfer for her household. The recipient is her partner. She can then choose to accept the deal or to pay a small cost to keep the amount for herself. The new measure then quantifies how much a woman would be willing to pay to keep control of the transfer. The researchers suggest that higher willingness to sacrifice money for control reflects weak control of household resources — and thereby means lower empowerment.

Macedonia’s cash transfer program

This new measure was used to study the effects of women-targeted cash transfers, with a focus on the Macedonian “Conditional Cash Transfer (CCT) for Secondary School Education”. This is a cash transfer program supporting secondary school enrolment across poor households.

In conjunction with researchers, the Macedonian government experimented with two different versions of the program. In half of the country, the woman in the household received the transfer. In the other half, the household head, generally the man, received it.

A comparison of these two groups allowed studying the effect of women-targeted cash transfers using the new lab test measure of empowerment. What did it show? The women in the study were, on the whole, willing to sacrifice some household income to receive the money in the lab test. But they were less willing to sacrifice it when they had already been receiving a cash transfer from the program.

The cash transfer really had made a difference to their household control — but that was not captured at all using the traditional measure of self-reported survey questions around empowerment.

Women-targeted transfers are indeed affecting female empowerment. At least in one of its many dimensions. By restricting our analysis to self-reported answers, we have, for years, been mistakenly assuming we have been capturing their full impact.

If we are serious in our aim for gender equality, we need to encourage more experimentation around its measurement. Traditional measures are failing at capturing the real impact of money spent on empowerment programming all around the world.