Speech Localization and Enhancement for Next-Generation Hearing Devices - ABSTRACT
Human listeners can perceive the locations of concurrent sound sources in the surrounding environment.
The localization is based on a variety of sound cues, which vary with sound locations and frequencies.
Perception of speech is often improved when speech and background noise come from different spatial
locations, because the brain has the ability to spatially separate sound sources. However, those spatial
cues are largely compromised in hearing devices. The goal of the proposed study is to involve
undergraduate students in the development of a binaural system that could provide spatial sensitivity to
hearing devices, so that the user can localize multiple sound sources and selectively amplify or attenuate
certain sound sources. The proposed algorithm for source localization is based on the fact that speech, as
well as a variety of daily signals, is sporadic in a two-dimensional time-frequency domain. By examining the
spatial features for each time-frequency point, we can decide which sound source it belongs to and can thus
decide whether or not to keep it. The first aim of the proposed study is to build a system that simultaneously
decodes sound locations along the horizontal and vertical axes. The potential user of the system will be
provided with a spatial map showing all the available sound sources with their locations. Undergraduate
students will form three groups to work on the construction of the system and learn some basic simulation
skills. Next, the user will be able to select certain sound sources to be removed or kept. Those time-
frequency points that have been previously classified as unwanted sources will thus be eliminated. Then the
modified time-frequency signal can be transformed back into the time domain to reconstruct the sound
waveform that contains mainly the sound of interests. Undergraduate students will run psychophysical
experiments to measure speech intelligibility from normal-hearing listeners to examine if the modified sound
is more intelligible than the original sound of mixtures. In summary, the ultimate goal is to develop a real-
time binaural-hearing system to perform sound localization and speech separation. The proposed project
serves as the foundation for applying the algorithms to sound devices for hearing-impaired listeners in the
future. It agrees with the mission of NIDCD in assisting people with hearing impairment and communication
disorders.
