Abstract
Noisy rooms with multiple active sound sources create problems for hearing-impaired listeners. Unwanted
masking sounds reduce the quality and intelligibility of speech that listeners want to hear, especially listeners
with hearing deficits. We propose a novel assistive listening system called HWIW (“Hear What I Want”) that
“scrubs” (i.e., removes) noise and other unwanted audio components from complex real-world environments
containing multiple acoustic sources. HWIW has been designed for integration into NIH’s Open Speech
Platform initiative for hearing aids and other personal audio devices. HWIW will leverage STAR Corp’s Multiple
Algorithm Source Separation (MASS) application framework of “pluggable” acoustic separation signal
processing modules. MASS is compatible with the Open Speech Platform and available on GitHub.
HWIW is a room-centric system that delivers listener-specific audio to users through their smartphones.
HWIW employs multiple microphones distributed around a room and connected to a room-specific dedicated
server. An initial HWIW setup procedure is used to name permanently positioned “noisemakers” in the room
such as speakers and appliances and characterize their acoustic radiation and reflection patterns. The HWIW
Room Server processes audio signals from multiple HWIW mics to scrub the noisemaker-generated sounds
from any microphone in the room a listener chooses to monitor. Multiple listeners are supported simultaneous-
ly. Each listener uses a HWIW Listener App to specify which mic to monitor for sounds of interest and which of
the known noisemakers to scrub. The HWIW Room Server computes an individualized scrubbed audio stream
for each listener and transmits it wirelessly to their Listener App. The Listener App outputs this audio stream to
the listener’s hearing aid, personal audio device, or earbuds as a standard line level or Bluetooth audio signal.
HWIW is room-centric, sensor image-based, latency-optimized, and listener-aware. Important system
components are embedded in the acoustic space itself, rather than in the user’s ear (the hearing aid). HWIW
calculates the acoustic image of masking sounds in sensor response mixtures so that images of unwanted
sounds can be removed. It computes the latency of its signal processing and balances the quality benefits of
longer-latency scrubbing against the perceptual advantages of faster response times. HWIW employs listener-
specific acuity profiles, information about the sound-isolating properties of each listener’s hearing aid or ear
piece, and the listener-specified masking sounds to determine whether which maskers are audible given the
listener’s acuity; and thus what the optimal noise scrubbing strategy is for that listener.
In Phase I, we will implement three HWIW MASS scrubbing modules, and a prototype of the Listener App.
We will objectively measure the ability of the scrubbing modules to scrub noise from microphone responses,
calibrate those measurements against perceived residual noise, and evaluate Listener App useability.
The HWIW system will help hearing impaired listeners hear what they want more clearly in noisy rooms.