After spending nearly two weeks in an intensive course offered by the Institute for Health Metrics and Evaluation for its incoming fellows (and a couple of fortunate UW graduate students like me), I have an even greater appreciation for the role of evaluation in our work in global health and for the complexity and difficulty in doing it well. In her lecture on Evaluating Health Programs, Dr. Emmanula Gakidou, pointed to what is called “The Evaluation Gap” where billions of dollars from international donors and national governments are being channeled into health programs in low and middle income countries but we know relatively little about what programs are working and how well they are working. The reason being is that most of them are not rigorously evaluated.
Even while researchers continue to develop and test new interventions such as vaccines, diagnostic tools, and drugs through thorough clinical research, we don’t know how best to deliver them in countries with weak health systems. The result is an innovation pile-up where proven interventions to prevent and treat disease are available yet millions of people are dying because these interventions don’t reach them.
As a graduate student in public health, it seems to me that the field of global health is turning in this direction and placing a lot more value on measuring impact. As President Obama said in his speech at the MDG Summit; “let’s move beyond the old, narrow debate over how much money we’re spending and let’s instead focus on results-whether we’re actually making improvements in people’s lives.” We need to know what is working and what isn’t so we can better our efforts and get the interventions out to the people who need them.
Unfortunately, evaluation is difficult to do well. As I quickly learned in the IHME course, there are some serious limitations to deal with ranging from poor data quality and availability to the fact that the methodology of conducting a rigorous evaluation just sometimes isn’t possible or is really expensive. As expressed in a Lancet editorial: “Evaluation matters. Evaluation is science. And evaluation costs money. It’s time that the global health community embraced rather than evaded this message.”
VillageReach makes a sincere commitment to evaluation of its programs and has ever since its inception. For example, as we begin to scale-up the Dedicated Logistics System in Mozambique, we are engaged in operations research to inform our program decisions. In addition to routine monitoring, we are conducting baseline evaluations in every province followed by process and outcome evaluations. We want to know what is working and more importantly, what isn’t working and why, so we can ensure that the resources we put into our programs really make improvements in people’s lives and that those interventions make it to the people who need them. We’ll keep you posted on our progress.
Jessica Crawford, MAPS, MPHc
Program Associate
VillageReach
A few weeks ago I traveled to Malawi to work on two projects that use SMS phone technology. One of these projects focuses on providing community health workers (CHW) with an easier and quicker way to communicate with their local hospital, supervisors, and fellow community health workers. I spent an afternoon with 18 of the 21 CHWs in the Kwitanda province to understand how they would utilize such technology, and used that information to develop use cases (e.g. outbreaks, inventory shortages, emergencies, etc.) that will help them provide better health care to villagers in their catchment areas. For the other project, I met with shop owners, assessed the medicines they sell, and discussed the benefits of inventory management with them (which is of personal interest to a supply chain person like myself). The insights I gained through interacting with CHWs and shop owners were then funneled to our technology team, which is working on our upcoming Management Information System (vrMIS3).
