AI Device Listens for Coughs and Sneezes to Monitor and Forecast Pandemics

Researchers at the University of Massachusetts Amherst have developed a portable AI device that can listen for coughs and sneezes and count the number of people present in public places to make predictions about levels of flu-like illnesses.

The system, called “FluSense,” could be useful in the current COVID-19 pandemic in helping researchers to monitor the situation and determine where and when intervention is needed most.

Successfully addressing the COVID-19 pandemic and future events of similar magnitude may require new surveillance technologies to monitor the spread viruses. In many countries there is currently a backlog in providing test results, while and other techniques are needed to supplement testing in predicting the spread of the virus and potentially being even more proactive in quickly getting the right people to be tested. Monitoring the viral spread in real time could help authorities to make decisions on how best to allocate medical resources, such as drugs or vaccines, or enforce travel restrictions, for example.

This new system consists of a microphone array, a thermal camera and a Raspberry Pi computer system. It records non-speech sounds that could be associated with flu-like illness, such as coughing or sneezing, and can image people at a distance using the thermal camera to obtain their temperature.

The researchers have taught a deep neural network, also housed within the device, to recognize the recorded coughing sounds and also to count the people that it images. In terms of maintaining privacy, the device does not store any recordings of speech or images in which individuals can be distinguished.

The system is conceived as being unobtrusively mounted (when encased in a box it is about the size of a large dictionary) in hospitals, waiting rooms, and other public spaces. “I’ve been interested in non-speech body sounds for a long time,” said Tauhidur Rahman, a researcher involved in the study. “I thought if we could capture coughing or sneezing sounds from public spaces where a lot of people naturally congregate, we could utilize this information as a new source of data for predicting epidemiologic trends.”

To test their device, the researchers placed it in four waiting rooms at a health clinic for approximately six months. Strikingly, the device predicted illness rates in the clinic and the data generated by the system correlated with levels of flu-like illnesses within the patients passing through the clinic, as determined by lab tests.

“We have the initial validation that the coughing indeed has a correlation with influenza-related illness,” said Andrew Lover, another researcher involved in the study. “Now we want to validate it beyond this specific hospital setting and show that we can generalize across locations.”