Mariam Zameer, Director, Health Systems Global Technical Team; Immunization Lead; and Swetha Srinath, Senior Manager, Data Analytics attended the 2023 Institute for Disease Modeling (IDM) Annual Symposium from May 22-24. The theme of this year’s Symposium was “Frontiers in Modeling & Data Science for Global Health.” Below are some of their key takeaways from the Symposium.
Health inequities are a major global challenge particularly in low- and middle-income countries (LMICs). These regions experience limited access to health care and affordability, resulting in large out-of-pocket expenses[1].
The IDM Symposium sought to address these inequities using data, modeling and technology. It was noted that the growing size and complexity of health data present exciting opportunities for countries to glean new and powerful insights to improve health outcomes. For example, data from social media and mobile phone apps can be combined with data from traditional sources such as medical records, census and health survey data to paint a holistic picture of disease prevalence, care-seeking behavior and the unmet health needs of specific communities.
At VillageReach we aim to transform health care delivery to reach everyone, which is why we were excited to attend the Symposium and to take a deep dive into how modeling outputs are being translated into actionable decisions in global health. Some of our digital solutions include using mobile phone technology to create more responsive primary health care systems, as well as using digital platforms to increase data visibility in public health supply chains.
The Role of Modeling
Data modeling can help understand and visualize relationships between variables; in doing so, it can explain past behavior or even predict the future behavior of these variables. In global health, modeling can be used to forecast disease burden and predict intervention impact. As such, data modeling presents many opportunities for public health organizations to use quantitative techniques for evaluating and informing decision-making and to eventually improve health outcomes.
However, while complex analytics techniques like modeling hold a lot of promise, they can sometimes seem like a ‘black box’. Modeling outputs, resulting from data analyzed in complex ways, are not well understood by many people, including public health practitioners and government stakeholders who often feel they need to accept modeling outputs at face value because they do not always feel empowered to question it. Alternately, if practitioners and stakeholders have not been involved in building of the model or data inputs, they may not trust the outputs. During the Symposium, we certainly saw the importance of questioning models and modeling outputs, especially because it allows us to question the biases that are inherent in the data being input into models. This also emphasizes the role of boundary organizations or knowledge brokers, who understand data as well as policymakers and can help bridge that gap.
For example, when incorporating new analytics techniques and technologies into global health, we need to think about how gender is involved, as both data and modeling are not gender-neutral. When the data inputs and modeling outputs themselves are biased, we run the risk of incorporating biases into the decisions we are making. How do we include gender when thinking about ways to improve the supply chain? We need to further consider gender from a broader health systems perspective in order to improve health care systems and health care delivery.
The Promise and Perils of Artificial Intelligence (AI) and Large Language Models
The IDM Symposium also had fascinating sessions on the very topical subject of Large Language Models (LLMs) and Generative AI. It was clear from all the rousing sessions that while LLMs can feel like new and shiny tools, we need to be mindful of how they are applied and used. We must ensure that the application of these technologies does not widen the digital divide and exacerbate health inequities, especially in under-reached communities. Additionally, much like simpler models, large language models can be notoriously complex and opaque, making it difficult to truly understand how they came to certain conclusions. This can make governments rightfully skeptical of adopting recommendations or outputs coming out of large language models.
There is also the point of LLM applicability, particularly in low- and middle-income countries where English is not the primary language. Existing models are primarily trained with English language data and input, mainly with European accents, which can limit their applicability in LMICs. Let’s consider Malawi where even though the official language is English, many other languages are also spoken. For example, when receiving calls through VillageReach’s Health Center by Phone (Chipatala Cha Pa Foni or CCPF), Chichewa is the language spoken by many of the clients calling into the telemedicine hotline. To truly expand the applicability of LLMs in global health and democratize their use, they must be trained across the breadth of regional languages.
However, all the advancements in Generative AI may be for naught if we do not have a public health workforce that is trained to use it or the resources required to train and troubleshoot LLMs and interpret the outputs of LLMs. LMICs will require support to uptake these new technologies and truly benefit from them.
Solving Real World Problems
Another key aspect that we ruminated on during the Symposium was the ability to apply modeling and modeling outputs to shape process, practice and policy. In addition to being able to understand and question models, we must also ensure that modeling outputs are interpretable and relevant for governments to trust them to meaningfully inform policymaking.
VillageReach emphasizes the importance of co-defining research questions, co-inputting data and co-creating models with governments, to ensure that outputs can be easily appraised and interpreted by governments. This type of end-to-end collaboration is critical for turning modeled evidence into action.
In contrast to others, VillageReach presented key real-world questions that we are grappling with, which could benefit from modeled evidence. In the face of health worker shortages hindering the delivery of routine vaccines, VillageReach is exploring the role of CHWs as vaccinators and the impact this cadre of health workforce can have in reaching zero-dose and under-immunized children.
We posed some key research questions to our audience of data scientists and data modelers, to highlight how we want to generate evidence that models the benefits of CHWs as vaccinators. For example, we are keen to explore the potential impact of task-shifting vaccination administration to CHWs on the workload of other Human Resources for Health, as well as the potential gains in immunization coverage and equity that can be realized by enabling CHWs to play a bigger role in vaccination.
Looking Towards the Future
Producing policy-relevant and actionable evidence is complex and time intensive, and as a result, requires interdisciplinary work and collaboration that spans several areas including data science, public health, advocacy, etc. A collaborative and complex endeavor such as this also needs sustained funding.
The 2023 IDM Annual Symposium was a valuable experience that demonstrated the importance of being able to harness modeled evidence to translate it into appreciable shifts in practice and policy.
In order to adopt modeling and new technologies like large language models and meaningfully apply them to global health, there are several factors that need to be taken into consideration:
- Answering real-world questions from country policymakers and stakeholders, and getting buy-in when data is being inputted.
- Investing in modernized data analytics infrastructure and processes to unlock the power of the data being collected.
- Using modeling best practices for transparency and reproducibility so we can “break” the modeling ‘black box’ and empower practitioners and policy-makers to question and critically review modeled evidence.
- Acknowledging and recognizing the sources of bias in the data being input into models, including LLMs, so that we can recognize the bias that might be present in modeling outputs.
- Training and collaborating with policy-shapers and policymakers to appraise model outputs and translate them into actionable decisions.
Taking these into account will help us bridge the evidence-to-action gap, particularly as we evaluate how new tools, analytics techniques and technologies can improve how we deliver health care for the under-reached.
[1] Frontiers | Editorial: Health Financing and Spending in Low- and Middle-Income Countries (frontiersin.org)