Post authored by Bernard Kagondu, Project Tech Lead
For over a decade, Medic Mobile has used data to support decision making and to inform our Human-Centered Design (HCD) approach while supporting our partners. Medic Mobile is at the forefront of innovation, and data science is part of our vision to sustainable, low-cost community healthcare technology.
In 2018, we partnered with Living Goods to pilot a Predictive Algorithms initiative with the aim of demonstrating that these algorithms can be implemented within tools built on the Community Health Toolkit (CHT) Core Framework. We also wanted to show that these algorithms can impact key health metrics and identify the most in-need populations. Living Goods was a great partner for this initiative, as their network of community health workers (CHWs) have been using an application built on the CHT Core Framework, called the SmartHealth app, for over five years.
Two years in, it has been an exciting ride. In this post, we highlight the experience.
We started with a risk factors selection process by looking at the data accumulated by Living Goods’ networks of CHWs over five years related to use cases including Integrated Community Case Management (iCCM) of childhood diseases, maternal and child care, and – more recently – data relating to immunization and family planning. By doing so, we sought to identify patterns that would point us to areas that would benefit from a data science intervention.
From the original risk factors identified, we worked with Living Goods to use a scoring matrix based on priority, data availability, and potential for intervention to narrow it down to the following four:
Use case selection
- Probability of a child getting sick (ICCM)
- Timeliness of illness identification (ICCM)
- Facility delivery (Pregnancy)
- Probability of a newborn developing danger signs (Pregnancy)
For the ICCM risks, our aim was to predict which children are most likely to be sick and subsequently assessed more than 72 hours after their symptoms develop, and to help highlight these individuals/households to CHWs for increased visits or more proactive check-ins.
For the pregnancy risks, our aim was to predict which women are likely to deliver outside of a health facility, as well as which women are most likely to deliver a baby that will develop danger signs. Additionally, whether there are specific women who are both unlikely to deliver in a facility while also being more likely to have a baby who develops danger signs.
Next, we developed predictive models for each of the four use cases.
Predictive modeling
This step involved evaluating a number of modeling approaches based on accuracy and ease of system implementation.
Individual dynamic and static variables were used. Static variables due to technical feasibility but also because these were family-level variables that are rarely updated. In addition, the slight gain in accuracy did not warrant the additional technical investment required to make them dynamic. This was also true in picking modeling approaches.
The outcome was model approaches that were implementable on the client-side using Javascript:
- Naive Bayes for the binary outcomes: The probability of being diagnosed within 72 hours, the probability of delivery in a health facility, and the probability of a newborn developing danger signs
- Cox Proportional Hazards Model for the survival risk: Time to next illness
Next, we needed to build the algorithms into the SmartHealth App, built using the CHT. The models were developed in R and translated to Javascript. The translation process was not without challenges and lessons. Soon we learned how important it was to have useful and effective tests that covered the translation.
We also developed education-focused workflows to be acted on by CHWs for the clients identified by the model as needing care.
We built dashboards to help the program monitor the initiative and inform design iterations. In addition, we did a lot of data analysis tailored to not only support Medic Mobile’s Data-Driven Iteration (DDI) HCD approach but also derive further insights.
Iterating
As mentioned earlier, HCD was extensively employed resulting in more than six phased iterations during the one-year lifespan of the experiment. CHWs’ perspectives were enlightening, and we used their insights to improve each iteration.
Data, as you would imagine, was important to the success of this experiment and as a result, a lot of effort went into ensuring data sanity, the accuracy of the data analysis, and dashboard visualization.
Conclusion
Through this experience, we were able to solidify our DDI process and gather product requirements among other learnings. We are grateful for the opportunity to run this experiment, and special thanks go to our partners at Living Goods.
This work was one of the many steps in Medic Mobile’s journey towards achieving our data science vision. We could not be more excited for what lies ahead!