Last weekend I had the singular pleasure of spending a night with some of the best and brightest young minds in the country. I’m talking about ID Hack, a 24 hour hackathon put on by the Harvard Developers for Development and the MIT Global Poverty Initiative. Unlike some other hackathons the aim wasn’t to create the next Facebook based platform to allow users to rate attractiveness of their friends cousins or something equally vapid and zeitgeisty, it was to put young developers in touch with globally focused NGOs with real technical needs. [Also to stay up all night and eat pizza] We all showed up with a project or problem, pitched to the more than 100 young brains and they formed teams to tackle them.
Our pitch wasn’t quite as straight forward as some, we weren’t looking for a new website, we wanted deeper analytic measures. Our pitch was this: VaxTrac has piles of data, our stakeholders are used to asking for the typical metrics and we provide them. However, we collect more data than a typical immunization program, what metrics can we create from our piles of data that can inform and empower local decision makers? Once we find a metric, how can we visualize it in an interesting way?
I spent the night working with two students; Michael a freshman in CS at MIT, and Wanli, a masters student in Psychology at Brandeis. We wanted a simple analog (probably over-simple) for the efficacy of any particular clinic. We chose the average schedule adherence and vaccination rates for two vaccines with a single dose in the first year , BCG and Yellow Fever. BCG is given at birth and Yellow Fever at 9 months. We looked at the number of doses of each for each clinic and also the average age at which each child who got the vaccine received it. Based on this we built a map showing which clinics most closely adhered to the set schedule. You could clearly between the two maps, which clinics were retaining the most children from the birth dose to the nine month dose. Not bad for a night’s work!
[Note: We mapped real clinics to random cities, this map isn't representative of actual regional performance. Also we were working with meta-data, not patient records.]
Again, I’d like to thank my team. They chose one of the more abstract problems available and I think they did a fantastic job.
In the months leading up to our preparation to baseline for this new project year, most of our team’s focus went into defining what it was we wanted to measure- our indicators, our evaluation tools, and where we would evaluate. What would be the change we were predicting might happen? That is one of the fun yet crazy parts of doing a project baseline involving new technologies in field. When you perform a normal baseline, you are measuring and observing how the environment is before you introduce a new element that will hopefully change the environment for the better. In other words, you are seeing what the picture looks like before you affect the landscape. Some organizations only do evaluations of their projects’ effects after the project ends. But nowadays the general practice is to mea
sure where you finish against where you started, so thus, you need to set a baseline of prove where you started. One of Vaxtrac’s main pillars is to measure transparently our impact, to know where we help affect change, and to identify areas where we can do a better job. This year was our first ever baseline, and moreover, it would be a baseline measuring things that have really never been assessed before; it would prove a lot more work than we first anticipated.
I have only worked for organizations that have already well-established monitoring and evaluation systems. Vaxtrac being a rather new entity is still ramping up our monitoring and evaluation process. In addition to piloting new projects and technologies, we are literally also piloting our systems in field. The systems, like the monitoring and evaluation process, will help set the foundations for the entire organization in the countries we enter in the future. We also have dedicated ourselves to thinking holistically, broadly, and asking bigger questions like are there newer tools that we could create and use that no one else has yet? We want to push the standard to a higher level.
Even before we traveled into the rural sector of Benin to baseline, we tasked ourselves with a rigorous and heavy workload of preparation. Our team spoke with lead partners in the field of health, such as UNICEF and WHO, to see what evaluation tools they had and applied around health and technology. Our team in Benin went to the experts at the national university and also met with partners on the ground. We essentially realized we were seeking to measure indicators that often are not measured or even been
considered before, and so the tools do not really exist to use as a map to our own tool design.
We created from scratch, building off of other sectors such as education and customer service tools in the USA, tools that look at: clinic work efficiency, work satisfaction, clinic customer service satisfaction, prior technology training and capacity, and also the average profile of our clinic health workers and the parents and guardians bring their children for vaccinations. Week after week, multiple editions of the evaluation tools were done in both English and French. Nearly twenty people fed into the design of our pilot tools for baseline, many of them key experts in West Africa. And many of them were seeing some of these tools for the first time. With anticipation, we are asking ourselves as a team, “Will these tools work well in field? Will they capture patterns and changes that we might otherwise miss? Are we forgetting anything?” It will be an adventurous year, measuring not only the change our project has on Benin’s health system, but also testing our own ambitious ideas and creativity, for better or worse, to go that extra mile.
Arriving in the tourist epicenter that is Bali—surrounded by five-star resorts and haute couture outposts—was a bit of a change from my normal travel patterns, in which I find myself more often in the remote hills of rural Africa. It was difficult to reconcile what I was seeing with the ultimate reason for my trip, which was to learn more about the challenges faced in delivering health care in the emerging market of Indonesia where 35% of one of the world’s largest birth cohorts is not fully vaccinated.
As a quick primer, Indonesia is a country of 17,000 islands with a population of 250 million people. The country’s gross national income—a good proxy for evaluating how “developed” a country is—puts Indonesia at a level on par with Guatemala or Ukraine. All of these factors make the country a great site for our newest project. The higher relative wealth translates to better infrastructure and a health workforce with more training. However, there are still some interesting challenges stemming from the dispersed geography and the cultural elements of being the world’s largest predominantly Muslim country.
I spent the week leading up to Christmas traveling around the country, meeting with health officials and observing firsthand how the health system operates. In only five days, I was able to visit three different islands (only 16,997 to go!) and see numerous health facilities. We even got to register an adorable Indonesian child in the VaxTrac system as a live demo for several health workers.
While we were also in Jakarta and Bali (for work, I swear) we spent the bulk of our time assessing the health system through site visits on the island of Batam, located 12 miles off the coast of Singapore. This provided us a great opportunity to see the true nature of how vaccines are delivered, the realities of which can never be conveyed without observing directly. It didn’t hurt that we also got a chance to sample the incredible local cuisine.
This trip was phenomenally successful, and we have laid the groundwork for what I think will be an impactful project over the coming years. This could not have been done without the support of many individuals from the Ministry of Health, WHO-Indonesia and CDC-Atlanta, but I would like to especially thank Dr. Sandra from MoH, Dr. Michael Friedman from WHO and Dr. Susana Panero from CDC.
VaxTrac is pleased to announce a recent investment of $695,431 from the Bill & Melinda Gates Foundation which will provide new technology and ongoing support for the expansion—both geographic and functional—of mobile vaccine registry systems in health facilities in Benin, West Africa. This investment will enable new health centers to receive the resources and training necessary to effectively integrate the VaxTrac system into their vaccination program with the goal of improving record keeping practices, reducing vaccine waste, and improving schedule adherence, especially for children under the age of five.
The VaxTrac system is a suite of data collection and analysis tools tailored specifically for immunization programs in developing countries. It uses a technology portfolio comprised of laptops, tablets and mobile phones to collect data and fingerprint scanners to accurately identify patients. This serves to overcome the primary driver of the current inefficiency in the vaccine delivery system: a lack of information.
In addition to the geographic expansion, VaxTrac will also lead an effort in Benin to re-evaluate the vaccine information management system from a more holistic perspective, integrating the previously segregated functions of health records, population statistics, stock management, and cold chain performance monitoring. This activity will also identify the broad array of stakeholders throughout the vaccination process and how they interact with system data. Underlying all of these activities is the motivation that technology can—when thoughtfully applied in combination with a thorough understanding of the unique user requirements of community health workers—greatly facilitate efficiency and effectiveness improvements in the vaccine delivery system.
This award is the result of continued collaboration among VaxTrac, the Benin Ministry of Health, UNICEF, and the World Health Organization since early 2012 to implement and support clinic-level vaccine data collection technology in 30 health facilities throughout several districts across the coastal region of Benin. With this new phase of the project, VaxTrac will roll out the latest version of its system to approximately 120 health facilities, fully saturating an administrative division roughly equivalent to that of a US state.
By Jessi Hanson, Monitoring and Evaluation Manager
“Jessi, see here,” Fidel caught my attention. I watched as the young mother sat down and let the nurse position her so she could more easily get the child’s fingerprint. The Vaxtrac biometrics tool was a simple laptop connected to a fingerprint scanner through a USB cable. The computer screen showed a simple, user-friendly system especially developed for health workers with limited literacy. As a reading specialist, a person whose main profession is to work with people who cannot read or write, I found this aspect of the tool the most interesting. Looking at it, I thought, “Well, heck, I could figure that out easy enough, I think.” The scanner took the child’s index print and stored it as an image converted into a special key that would store the patient records to this individualized key. And yes, for any of you who are scratching your heads reading this, I was, too, at this point. But seeing the tool in action, I was seeing it was going to be much simpler than I first thought to measure our indicators, or our ‘how many’. No more Excel spreadsheets, no more counting files. It was all in the computer and uploaded to a central hub for larger tracking numbers.
The boy baby was good to let the nurse fool with his fingers to get the right scan. He didn’t cry once but wanted to play with the scanner as if a toy. The mother looked interestedly at the computer scan of her son’s finger and grinned. It was maybe one of her first times so close to a computer at all. The nurse explained it to the mother and us, showing us slowly with Fidel how it worked and what buttons she selected. Then the nurse pointed to a few molded boxes behind us, with stacks of patient records mushed together in alphabetical order. They were dusty and decaying like the boxes that held them.
“This is the normal way records are kept,” Fidel explained. My eyes were huge at the realization of what we were really doing. Throughout the country, and many countries like Benin, medical records were kept in this fashion, just like the files that Ministry of Education kept in Liberia. Systems were antiquated, cumbersome, and left large holes for beneficiaries to fall through. How did any nation achieve knowing how many of its people were getting access to the services they truly needed with systems built out of rotting files in boxes? How could it possible ensure all its mothers and babies had access to quality medical care? How did a nurse know she was vaccinating a child for the first time, or that the vaccine she was administering was stored properly before it arrived to her clinic? So many questions so quickly eliminated with this little biometric tool…
I looked again to the young mother who had shared her boy with me for a spell. She was fixing him with African fabric to her back to prepare to travel home. She adjusted his little form into the nook of her lower back comfortably. She waved goodbye to us all and made her way out. I watched as the little boy yawned and sank back into sleep before his mother even walked out the door.
My team would not only be measuring in the how many. The true impact of our new work would lay in the qualitative- the efficiencies and effectiveness of treating infants and working alongside their mothers, health workers and governments to track the invaluable.