During Ignite 2020, Microsoft took the wraps off its efforts to detect pathogens before they cause widespread outbreaks. Applying learnings from Premonition, a five-year epidemiological project spearheaded by Microsoft Healthcare, the company hopes to employ a combination of robotics, cloud infrastructure, and sensors to monitor and sample disease carriers and identify known and novel biological threats.
In the coming weeks, technologies developed by the Premonition team will become available in early access to a larger group of partners, including the National Science Foundation’s Convergence Accelerator Program, Johns Hopkins University, Vanderbilt University, the University of Pittsburgh, and the University of Washington’s Institute for Health Metrics and Evaluation. Additionally, Microsoft says it will expand its collaboration with Bayer to investigate vector-borne diseases and the role of autonomous sensor networks for biothreat detection.
Experts agree the key to preventing health crises like the coronavirus pandemic is the early detection of potentially harmful pathogens. Identifying pathogens gives researchers, health organizations, and governments time to develop new treatments, prepare a public response, and minimize exposure.
But pathogens move through the environment in difficult-to-monitor ways. The U.S. Centers for Disease Control and Prevention estimates that 60% to 75% of emerging infectious diseases — Zika, dengue, and West Nile, for example — are caused by pathogens that jump from animals to people. People are most commonly exposed via a host animals’ excrement or slaughter, but bites are another transmission vector.
Five years ago, Microsoft’s Central Engineering group, the hardware lab that provides electromechanical engineering expertise to Microsoft Research teams, helped found Premonition. From the start, the project’s goal was to scale monitoring to detect disease threats early.
The Central Engineering team developed a foot-tall, 64-compartment robot to monitor for insects that might transmit pathogens and collect samples from other animals. In point of fact, most terrestrial animals are arthropods (like insects, spiders, and ticks), which pollinate crops, provide food for larger animals, and recycle biodegradables in the environment. But arthropods, in particular mosquitoes, are some of the most important transmitters of pathogens that cause human disease. The mosquito species Aedes aegypti carries Zika and dengue; Culex quinquefasciatus spreads West Nile virus.
The robot, which went through three prototypes, adaptively lures, identifies, and selectively captures mosquito species where it’s deployed. Using carbon dioxide-emitting and spring-loaded traps, battery-powered microprocessors, and camera-equipped mosquito chambers, the robot runs AI species identification algorithms to snag “medically relevant” mosquitoes with over 90% accuracy. The algorithms look at patterns thrown by the mosquito’s buzzing wing when illuminated with infrared light within the chambers. Microsoft engineers collected wing images and worked with Johns Hopkins researchers to determine breeds and train the algorithms throughout the course of pilot deployments.
Work has begun on a “next-gen library” of classifiers, a Microsoft spokesperson tells VentureBeat, with Harris County Public Health in Texas shipping mosquito species to Microsoft campuses for characterization. The Premonition team anticipates the classifiers being deployed on new devices in Harris County by winter 2021. Microsoft expects it will have more to report next summer and into the fall, after Harris County Public Health’s entomologists evaluate the classifiers’ accuracies.
Systems inside and outside of the chambers digitize mosquito behaviors at high resolution, characterizing how they move through their surroundings. The chambers, which are designed to withstand rain, wind, and other elements, also tell researchers what time each mosquito was trapped and what the temperature, wind, light level, and humidity was when the mosquito flew in.
Data recorded by the robot is beamed to Azure for analysis. In the cloud, Premonition’s metagenomics pipeline estimates the organisms and viruses in a sequenced environmental sample. Algorithms and a large database of reference genomes enable the system to test for sequenced pathogens and detect related viruses and bacteria, while at the same time looking for associations between pathogens and their potential hosts.
Microsoft says Premonition can scan a sample using trillions of genetic signatures to discern not only viruses, but bacteria, fungi, and higher-order life. This allows it to identify the types of animals on which mosquitoes fed.
Premonition was initially trialed with Harris County Public Health in Houston, Texas during the peak of Zika transmission risk in 2016. Microsoft says that it has since been piloted in Southern Florida and Tanzania.
“In the past, we have run shorter pilots on the scale of a few months at a time, to guide engineering and science. We have always removed these traps from the environment after experiments were completed, as they were not designed for long-term deployments,” the spokesperson said. “However, our newly announced partnership with Harris County Public Health will see the introduction of hundreds of devices into selected locations, with the goal of continuously monitoring the environment for long durations (e.g., years). We will work with Harris County Public Health to scale further as we learn how to both optimally utilize these data streams and select sites to improve public health operations.”
Beyond iterating and deploying the robot traps, the Premonition team is working on ways drones could be used to track the spread of illnesses and mitigate disease outbreaks. In the short term, the researchers hope to use drones equipped with computer vision technology to look for areas where mosquitoes that could be carrying diseases congregate.
The Premonition team is also developing a “next-gen” device optimized for public health programs that will be announced in late 2020 or early 2021. That’s when Microsoft plans to detail its early access program and allow select public and private health and service organizations to deploy the full Premonition stack, including software workbenches and cloud compute credits, traps, and anthropoid digitization tools.
Premonition director Ethan Jackson eventually hopes to have thousands of traps and drones around the world. The team’s work would complement initiatives like the Global Virome Project and the U.S. National Institutes of Health’s recently launched pathogen-tracking program that aim to find and sequence every virus that might threaten mankind.
Other efforts to track potentially dangerous pathogens are ongoing. In 2016, Verily, the health-focused subsidiary of Google parent company Alphabet, pulled back the curtain on the Debug Project, which taps AI algorithms to quickly distinguish between male and female mosquitoes of Aedes aegypti. (This year, Verily succeeded in wiping out disease-causing mosquitoes from three test sites in California’s Central Valley.) In 2017, entomologists at Stanford University built an AI-powered, sound-sensing smartphone app that can tell mosquito species apart by the noise their wings produce. And Puerto Rico-based Wovenware put together a dataset of images and labels to train a mosquito-identifying computer vision algorithm.
“There are over 3,600 species of mosquitoes, but if we can catch the right ones and do metagenetic sequencing on all the DNA and RNA we find in them, we can see the diseases on the move,” he told the New York Times.