Research News
Prototype used machine learning to classify sounds in real time
November 10, 2020
Smartwatches offer people a private method for getting notifications about their surroundings — such as a phone call, health alerts or an upcoming package delivery.
Now University of Washington researchers have developed SoundWatch, a smartwatch app for deaf and hard-of-hearing people who want to be aware of nearby sounds. When the smartwatch picks up a sound the user is interested in — examples include a siren, a microwave beeping or a bird chirping — SoundWatch will identify it and send the user a friendly buzz along with information about the sound.
The U.S. National Science Foundation-funded team recently presented these findings at the ACM (Association for Computing Machinery) conference on computing and accessibility.
“This technology provides people with a way to experience sounds that require an action — such as getting food from the microwave when it beeps,” said lead author Dhruv Jain, a UW researcher in the Paul G. Allen School of Computer Science & Engineering.
Jain was born hard of hearing. “I use the watch prototype to notice birds chirping and waterfall sounds when I am hiking. It makes me feel present in nature. My hope is that other deaf and hard-of-hearing people who are interested in sounds will also find SoundWatch helpful.”
The team started this project by designing a system called HomeSound for deaf and hard-of-hearing people who wanted to know what was going on around their homes.
“I used to sleep through the fire alarm,” said Jain.
The researchers tested HomeSound in the Seattle-area homes of six deaf or hard-of-hearing participants for three weeks. Participants were instructed to go about their lives as normal and complete weekly surveys.
Based on feedback, a prototype used machine learning to classify sounds in real time. The researchers created a dataset of over 31 hours of 19 common home-related sounds, such as a dog bark or a cat meow, a baby crying or a knock on the door.
—
NSF Public Affairs,
researchnews@nsf.gov