Taking the easy way to measuring hard things

When I'm out on the streets of Bangladesh, sometimes I wonder: where are all the women?

Does your program empower women?  It's always a tough question to answer.

Measurement is expensive. 
It takes time to collect information, compile it, analyze it, and implement changes from it.   For empowerment, even identifying the most important indicators can be a big challenge.

Part of a good evaluation system is identifying those indicators that really tell you whether or not there is a problem. They may be surprising ones; not the indicators that you thought would be the most informative or relevant, but ones that seem less intuitive but are actually very correlated to change and very easy to measure; and they can hopefully serve as proxies.

For example, I met a public health specialist working in South Africa who worked on the immunization program.  While they collected 43 indicators, including the dose number, vials opened, age of people vaccinated, and more, they couldn't tell how many people were "fully vaccinated" or the overall "immunization coverage," which of course was the goal.  So he suggested that they replace it with three simple indicators: 

• Infants fully immunized by age one
• Tracking stockouts of the "most vulnerable vaccine" (i.e. the one that historically had run out the most)
• (My favorite) Was the refrigerator working?

An interesting proxy I learned about a few years ago was about surgical conditions. Some of my surgeon friends said that if you wanted to know about quality of surgical staff, infection control, and post-operation follow-up at a given facility, look at the mortality rate for cesarean sections.  If it was low, then the hospital knew what it was doing.  If it was high, then something was wrong.

Today one of my colleagues suggested that after giving clients antibiotics, we should follow up and ask them: did you finish your treatment course?  If they say yes, then we know that they have a decent level of health awareness.

These types of measurements only tell you if there is a problem.
They don't tell you where to look or what the root cause is.  But what's helpful about them is that they are simple, verifiable, and easy to collect.  It's a great trade-off: less time gets wasted on routine data collection, so more can be spent into diving into those places where there are issues.

Last month, I went to a global meeting on microfinance where we briefly touched on how we might measure empowerment among our clients, most of whom are poor women. One of the interesting questions we tossed around was: are there proxies for empowerment?  A few good ideas I heard included:

In Nicaragua: Does a woman gets a PAP smear?  
Despite having one of the highest mortality rates for cervical cancer, only 35% of women ever get a PAP smear (or related test, like careHPV).  Likelihood of getting a PAP smear correlates strongly with education, income, and access to healthcare.  Educating disempowered women about the importance of a PAP smear will likely not lead to changes in behavior.  But empowered women may be likely to translate the information into action, particularly if the test is made readily available.

(Note: this proxy may be broadly applicable--nice essay from Goats and Soda's Nsikan Apan about low screening rates in Africa)

In India: Does a woman get to sleep in? (I would rephrase to ask if she and her husband get up at the same time) 

Much less data here.  But given that women typically are responsible for making breakfast for the family, which may require getting water, using a wood stove that takes time to heat up, and cooking from scratch, it can take a long time.  Perhaps empowered women share the burden with their husbands, or have husbands that will settle for a simple breakfast.  I'm not sure exactly how this one would work.

Dinner for two?
Will you be dining sequentially or together?

In the Bangladesh context, how about: Does a woman eat meals with her husband or after he finishes?
Or: Does she have freedom to go to the market?

Like all proxies, none of these are a perfect substitute for measuring empowerment, but they can be easily captured by frontline officers and others that have regular contact with clients. They won't necessarily tell us the full story, but could quickly provide a decent pulse on the empowerment situation with minimal costs.

What is a good proxy measure for empowerment in your context?