Improvements in Sign Language Detection for Videoconferencing
GoVoBo, LLC, a company formed in September 2022, collaborates with Gallaudet University and AppTek to enhance videoconferencing experiences for the Deaf and Hard of Hearing (DHH) community through innovative Artificial Intelligence (AI) technologies. The project addresses significant challenges faced by DHH individuals in using mainstream videoconferencing platforms.
The spotlighting feature in videoconferencing platforms is a critical tool for enhancing DHH communication accessibility. It enhances visibility and focus by prominently displaying the current speaker or signer, crucial for clear comprehension in group settings. To date, spotlighting for sign language must be handled manually. Automating it would foster better engagement and participation from DHH individuals in virtual meetings, webinars, or educational settings, making it an indispensable element in digital communication platforms.
This project focuses on developing an automated spotlighting feature that intelligently detects sign language turn-taking. This advancement aims to provide more accessible and equitable communication for the DHH community, by making sign language a first-class citizen in turn-taking, co-equal with spoken language. This initiative aligns with the accessibility principles advocated by the World Wide Web Consortium and addresses improvements asked for by deaf and hard of hearing consumers.
The project leverages recent advancements in AI and Computer Vision, particularly in Sign Language Detection. It plays a crucial role in differentiating sign language from non-signing movements (gestures or hand signals), thereby reducing false positives and enhancing the user experience in videoconferencing. By accurately tracking and highlighting sign language, it ensures that the videoconferencing system’s spotlight feature is activated appropriately, focusing on signers when actual signing occurs. The new feature in GoVoBo's application will connect with existing videoconferencing platforms in a platform-agnostic manner.
The project will undergo a two-phase evaluation, combining technical and user-based assessments, to ensure the efficacy and usability of the developed features. This is an SBIR Phase I proposal with the aim of evaluating and inquiring about the feasibility of the technology on a larger scale.
This project not only aims to enhance the technical capabilities of videoconferencing tools but also to foster a more inclusive environment, thereby empowering the DHH community. By addressing the specific needs of sign language users and ensuring seamless communication for all participants, GoVoBo is poised to make a significant impact in the realm of digital accessibility
