“The pure and simple truth is rarely pure and never simple”
I’m an MPH graduate student and a recent addition to the VillageReach team as a summer Health Systems Group intern. When I was asked to come up with some ideas for how to evaluate active case finding for VillageReach’s new tuberculosis (TB) program in Kwitanda, Malawi, I was ecstatic.
Step one: figure out what active case finding is (preferably before anyone realizes I have no idea). Once I did the research, I realized it’s a pretty simple concept: the more people who have TB, the more people they can spread it to. If we can actively find people with TB and treat them (rather than waiting for them to come to the health center which may only happen after they become very sick or may not happen at all), then we can reduce TB. Easy!
Step two, how do we decide if our active case finding program worked? The health centers keep track of new TB cases so we can use those numbers to evaluate our program. If the numbers of TB cases seen at the health center goes up, then we were successful in finding more cases of TB. If the number of TB cases goes down, then we were successful in reducing the number of people who have TB. So no matter what the numbers say, we can say we were successful? While I was tempted to bring back a no-fail assessment plan to my supervisors, I had a sneaking suspicion that it wasn’t going to fly in an organization that highly values evaluation.
Step three, more research. Turns out lots of people are interested in evaluating TB programs and the Millennium Development Goals have set the goal of increasing the “case detection rate” to 70%. That means for every 10 people who actually have TB we want to find 7 of them. Ok, that could work. We already know how many people we’ve found with TB because of the health center data but how do we find out how many people actually have TB, assuming that many of them are undiagnosed? We would need to go door-to-door in Kwitanda and test everyone for TB (or at least test a large sample of people). That would be expensive. And frankly, if we had that kind of resources we WOULD just go door-to-door and test everyone. Not for evaluation purposes but because it would be the right thing to do!
So back to square one. After driving myself crazy with statistics and indicators for at least several days, I came up with an idea. We already do an annual house-to-house survey of questions – what if we asked every person if they have a cough and if they’ve been tested for TB? That way, we can know the percent of people with TB symptoms who’ve been tested and that number should always go up. If TB incidence in Kwitanda goes down (which we hope it will) the number of people with symptoms should go down too. But, with any luck, the percent of people with symptoms who have been tested will continue to increase until 100% of people who need to be tested can get tested.
All of this got me wondering the same thing non-profits have been wondering for years: how much money do we spend running our programs versus how much money do we spend finding out if we’ve actually run them well? While our indicators for the TB program may not be perfect, they’ll give us enough information about whether or not our approach is working while still leaving lots of time and resources to actually do the work.