HomeSound: In-Home System for Deaf or Hard of Hearing Users

Graduate student Dhruv Jain of the Makeability Lab at the University of Washington  demonstrates HomeSound, an in-home system for improving sound awareness for deaf or hard of hearing (DHH) users. HomeSound consists of a microphone, visual display, and multiple devices installed in the home. The system detects and visualizes, in real time, common household sounds like appliance functions, fire alarms, running water, speech, and more. A smartwatch supports wearable alerts.

Transcript

00:00:00 DHRUV "DJ" JAIN: I am DJ, a computer science student at the University of Washington. I work on human- computer interaction, specifically for people who are deaf and hard of hearing. This project was inspired by my own personal experience as someone who is hard of hearing. [KNOCKING] My product is called "Home Sound," contains a series of IoT displays,

00:00:23 deployed in different locations in the home. This is the server running on my desktop in my office, and it's connected to my home, where the IoT displays and the smart watch are installed. Let's say we have one tablet in our living room, one in the bedroom, and one in our kitchen. And these tablets sense the information in their environment, and provide that feedback in terms of a specific sound-- [BEEPING]

00:00:47 --like a microwave beep, or a fire alarm-- [ALARM SOUNDING] --and alert this information through the smart watch. I chose the Paul Allen School because of my advisors, Professor Jon Froehlich, and Professor Leah Findlater, who have unparalleled expertise in human computer interaction and accessibility. It was a great opportunity for me to collaborate with them-- [PHONE RINGING] --on projects that are important to me,

00:01:10 which is accessibility for people who are deaf and hard of hearing. I am DJ, and I am CSE.