Posted:
Google has introduced a sign-language-to-text feature that lets users sign to their phone's camera and have the signs converted into written text. The feature was announced at the Made by Google event on August 12 and is powered by a new model from Google DeepMind called SL2T. It launches first on the Pixel 11 series, supporting American Sign Language (ASL) to English, with the company stating that more devices and additional sign languages will follow.
The feature is built into Gboard, Google's keyboard, and the Live Transcribe app. In Gboard, users can sign instead of type to search the web, draft messages and documents, or ask questions of Google's Gemini assistant. In Live Transcribe, which displays real-time captions of spoken conversation, users can sign their own responses during in-person conversations rather than typing them. Google describes the feature as the sign-language equivalent of voice dictation.
Google says model was trained on more than 100,000 hours of sign language data spanning over 50 sign languages, though the initial release understands only ASL. The company states the feature was developed in partnership with the Deaf community, guided by Deaf employees and an AI Sign Language Advisory Committee. To protect privacy, the system tracks body movements on the device, while translation into text occurs on Google's servers.
Sign languages are independent languages with their own grammar and vocabulary, which means converting them to written English requires translation rather than a direct word-for-word transcription. The feature is offered at no additional cost on Pixel 11 phones. Real-world performance across different signers, lighting conditions, and signing styles will become clearer as more people use it.
Learn more here.
