This combo of pictures shows a sign language interpreter gestures during a speech of Cem Oezdemir (not pictured), co-leader of the German Green party, at a party convention of the German Green party “Buendnis 90/Die Gruenen” at the fair grounds in Hannover on November 16, 2012.
JOHANNES EISELE/AFP via Getty Images
For the roughly fifteen years that voice dictation has existed on smartphones, hearing users have been able to speak to their phones instead of typing — searching the web, drafting messages, querying AI assistants — through nothing more than their voice. For an estimated 70 million Deaf and hard-of-hearing sign language users worldwide, no equivalent existed. Yesterday that changed. Google DeepMind’s new sign-language-to-text model, SL2T, launched inside Gboard and Live Transcribe as part of the Pixel 11 announcement, and it is the first time SL2T model launch announcement has shipped inside a mainstream consumer product. The Pixel 11 will ship August 20, 2026, at which point those devices will be the first smartphones in the world where a Deaf user can sign directly into any text field — a web search, a Gemini query, a WhatsApp message — without typing a single character.
Why Every Prior Sign Language AI Got It Wrong
To understand why SL2T is significant, it helps to understand why previous approaches failed to scale. The field spent years pursuing a method called gloss-based translation. A gloss is a word-for-word English label for an individual sign: the ASL sign for the concept “run” gets labeled with the English word “RUN.” A system would first convert each sign into its gloss label, then attempt to assemble those glosses into an English sentence, as the AISLAC report details in its overview of prior gloss-based methods.
The problem is that sign languages are independent natural languages with their own grammars, syntaxes, and structures that have no word-for-word correspondence with spoken languages. ASL, for instance, shares more syntactic structure with Japanese than with English. Its grammar is partly conveyed through non-manual markers — the position of eyebrows, the direction of a head tilt, the shape of the mouth — none of which a gloss captures. It uses spatial grammar, where the location of a sign in three-dimensional space carries meaning that a flat word label cannot represent. It uses classifier handshapes, where a handshape predicated on an object carries motion and positional information simultaneously. Every one of these grammatical features is invisible to a gloss-based system.
SL2T bypasses this entirely. Rather than producing an intermediate gloss layer, the model translates directly from the signer’s body movements to fluent English text. As Google DeepMind explains, translating directly from landmarks removes artificial vocabulary limits and allows translation quality to scale directly with data. This is the same architectural insight that powered the shift from phrase-based to neural machine translation for spoken languages — and it is the reason SL2T’s output quality substantially exceeds anything the field has previously produced.
Pipeline Architecture and Privacy Design
SL2T operates through a two-stage architecture designed around privacy. The first stage runs entirely on the Pixel 11 itself. An on-device model built on MediaPipe Holistic — Google’s framework for simultaneously tracking face, hand, and body pose landmarks in real time — converts the camera feed into a sequence of 130 geometric landmark coordinates covering the face, body, and hands. The raw video feed is discarded immediately. Only the abstract coordinate sequence — essentially a stick-figure-like wireframe — is transmitted to Google’s servers for translation.
The second stage is a transformer neural network on Google’s servers that takes the coordinate sequence and generates an English text translation autoregressively, streaming words back to the device while the user is still signing. no user input logs retained are retained on Google’s side unless explicitly authorized by the user in the context of a model evaluation study.
The privacy architecture has practical benefits beyond reassurance. Because the server receives abstract landmarks rather than video, the model is insulated from background content, lighting variation, and clothing — factors that would otherwise complicate translation. One caveat worth noting: researchers have argued that dense facial landmark coordinates could theoretically constitute biometric identifiers under biometric data privacy laws, since a sufficiently detailed facial geometry scan approaches the specificity of biometric identification. Google has not addressed this specific concern publicly; the landmark representation provides strong practical privacy protection relative to raw video, but is not a guarantee of non-identifiability in the strictest legal sense.
Training Corpus Built Across Fifty Languages
SL2T was trained on 100,000-plus hours of training spanning over 50 sign languages, with roughly one quarter of that data in ASL. That cross-lingual training was deliberate: cross-lingual training outperformed single-language models caused the model to learn shared structural patterns across sign languages, outperforming models trained on ASL alone. The multilingual foundation is the same principle that made multilingual large language models for spoken language more capable than single-language models — applied here for the first time at scale to sign language.
The result is benchmark performance no prior published system has matched. On the FLEURS-ASL benchmark — an ASL-to-English translation test created by Google researcher Garrett Tanzer using recordings from five Certified Deaf Interpreters — zero-shot BLEURT score of 70, substantially higher than any previously reported score. BLEURT is a learned evaluation metric that correlates more reliably with human quality judgments than the older BLEU metric. On the PRESTO-ASL benchmark — a dataset of AI assistant phrases performed by Deaf signers to a docked tablet — SL2T scores 85 BLEURT. On fingerspelling recognition, the FSboard benchmark shows SL2T achieving 64 percent exact-match accuracy.
Sample translations published by Google show the model rendering complex, multi-clause sentences — legal and geographic language, rugby schedules, paleontological descriptions — with high fidelity. The model’s documented failure modes, acknowledged openly by Google and AISLAC, include rare signs, rapid fingerspelling (where individual letter sequences can be misread, producing “prey” as “grey”), passive constructions, and failure modes including fingerspelling errors.
Engineering for Real Signing, Not Just Benchmarks
Academic benchmark performance and usability in actual daily signing are not the same thing. Google’s engineering approach to SL2T appears to have treated the gap between them seriously.
Roughly 10 percent of signers are left-handed. left-handed signer training parity by applying left-right mirroring transformations during training. One-handed signing — the posture any signer adopts when the other hand is holding a smartphone — was addressed by simulating one-handed signing during training, masking keypoints from the non-dominant arm to teach the model to translate with reduced visual input. Streaming latency was engineered so that translation surfaces on screen while the user is still signing, rather than waiting for a complete utterance. Hallucination suppression mechanisms prevent the model from generating text in response to incidental motion in the frame.
The AISLAC impact report notes that tongue landmarks not tracked by MediaPipe — which is a critical ASL articulator — and that the representation captures no information about the signer’s environment, which can be used for referencing real-world entities. These are documented technical constraints, not hidden ones, and they are active areas of research.
Built With Deaf Community, Not Just For It
The model’s governance structure is as notable as its technical architecture. Google established the AI Sign Language Advisory Committee (AISLAC), bringing together Deaf organizations — the National Association of the Deaf, the World Federation of the Deaf, Rochester Institute of Technology’s National Technical Institute for the Deaf, and the Deaf Professional Arts Network — alongside independent subject-matter experts, as detailed in the joint impact report.
AISLAC was given pre-release devices running test builds of Gboard and Live Transcribe in July 2026. Committee members evaluated the model through unstructured daily usage and structured task-based testing across personal, administrative, and professional workflows. Their collective feedback is documented in a joint impact report co-authored by Google and AISLAC representatives — a transparency practice that is structurally different from the industry norm of consulting communities after deployment rather than before.
The committee’s overall assessment was highly favorable. Advisors described the technology as intuitive, responsive, and potentially transformative for everyday digital accessibility. They noted particular value in short-form tasks — keyword search, Google Maps navigation, Gemini queries — and in the Live Transcribe dual-interface layout that shows incoming speech captions on the upper half of the screen while providing a dedicated Live Transcribe dual-interface layout, enabling more fluid face-to-face conversations with hearing individuals.
The project’s origin belongs to Sam Sepah, a Deaf Googler who is credited with conceptualizing the SL2T initiative from the inside. Deaf perspectives shaped data collection methodology, user study design, and deployment evaluation throughout development.
What SL2T Cannot Replace
This is the section that Google and AISLAC were most explicit about in the joint impact report, and it deserves the same prominence here.
SL2T 1.0 is explicitly and formally ruled out as a substitute for certified human interpreters in high-stakes settings. The AISLAC report lists the following as prohibited contexts where the model must not be used: healthcare and medical settings including clinical consultations and emergency services; legal and law enforcement settings including police interrogations, courtroom proceedings, and attorney-client consultations; academic evaluation and K-12 or higher education classroom instruction; employment settings including job interviews and disciplinary hearings; and government and public services including administrative hearings and benefits determinations — all listed in the joint impact report’s prohibited contexts section.
More directly: Google DeepMind and AISLAC explicitly state that this model must not be used by third parties to evade statutory obligations under accessibility laws. The joint report states: “We do not believe that SL2T 1.0 satisfies legal obligations for reasonable accommodations under the Americans with Disabilities Act (ADA), Section 504/508, or international disability frameworks that are met by human interpreters.”
The AISLAC members raised this as a Priority Zero (P0) — the highest severity level in their classification framework — concern: that public agencies, healthcare providers, or employers might use the tool to avoid hiring certified interpreters to save money. The risk is real enough that Google and AISLAC published the prohibition in launch documentation rather than leaving it for Terms of Service fine print.
What SL2T Is Actually Built For
What SL2T is designed for is the long tail of everyday informal digital tasks: messaging, web search, note-taking, casual face-to-face conversations at a coffee shop or retail counter, and Gemini queries, all detailed in the AISLAC report’s supported scenarios. For these, it keeps the signer in the loop throughout: both apps present a text preview the user can review and edit before sending, and in Live Transcribe the Deaf user controls when translated text is displayed to their conversation partner.
SL2T Roadmap and Future Development
The roadmap includes three near-term additions documented by the AISLAC report. First: dictation support for punctuation, new lines, emojis, and basic editing shortcuts — commands users cannot currently issue through sign input. Second: extending conversation memory across sequential clips in Live Transcribe and longer contexts in Gboard, which will improve multi-turn and longer-form translation quality. The each clip processed independently, without retaining context from prior exchanges in the same conversation.
Third, and perhaps most consequential: Google has signaled that programmatic access via Google Cloud APIs and the possibility of open-weights model releases (named in the report as “SignGemma”) is under active discussion with AISLAC. If realized, this would allow Deaf researchers, developers, and Deaf-owned businesses to build on the SL2T architecture — potentially expanding it to sign languages, dialects, and use cases the Google team has not yet reached.
The longer-term ambition, stated explicitly in the DeepMind blog, is sign language generation roadmap: a model that produces signing as output, not just text, which would enable true two-way machine-mediated signed communication.
The Pixel 11 will ship August 20, 2026, with SL2T available at no additional cost in Gboard and Live Transcribe, as confirmed by Engadget’s launch-day coverage. Expansion to additional Android devices and additional sign languages beyond ASL is described as coming soon, with no specific timeline committed. For the estimated 70 million Deaf and hard-of-hearing sign language users worldwide, the feature arriving on Pixel 11 represents what voice dictation represented to hearing users roughly fifteen years ago: the first real proof that the technology can work well enough, in an actual product, to change how you use a phone.
Frequently Asked QuestionsHow does SL2T translate ASL without sending video to Google?
An on-device model called MediaPipe Holistic runs entirely on the Pixel 11 itself, converting the camera feed into a geometric coordinate sequence that maps 130 landmarks across the face, hands, and body. The raw video is discarded immediately. Only the abstract coordinate sequence — think of it as a stick-figure wireframe — is transmitted to Google’s servers, where a transformer neural network translates it into English text. No video of the signer reaches Google’s infrastructure.
What is a gloss, and why did gloss-based sign language AI fail?
A gloss is a word-for-word English label for an individual sign — for example, labeling the ASL sign for “run” as the English word “RUN.” Prior sign language AI systems converted each sign to its gloss label, then tried to assemble those labels into sentences. The problem is that sign languages are not English on the hands, as Google explains in its overview of SL2T technical design. Their grammar is partly carried by facial expressions (non-manual markers), by where signs are placed in three-dimensional space (spatial grammar), and by handshape-based constructions that encode motion and object descriptions simultaneously — none of which a word label can capture. SL2T bypasses the gloss layer entirely, translating directly from the signer’s body coordinates to fluent English. The Wikipedia article on sign language grammar and structure provides additional background on what makes sign languages independent natural languages.
Can a hospital, school, or employer use SL2T instead of hiring a certified interpreter?
No — and both Google DeepMind and AISLAC have said so explicitly. The joint AISLAC impact report states that SL2T 1.0 does not satisfy legal obligations for reasonable accommodations under the Americans with Disabilities Act (ADA), Section 504/508, or equivalent international disability frameworks. Its use in medical consultations, legal proceedings, police interactions, academic settings, job interviews, or government hearings is explicitly prohibited. Google and AISLAC have publicly stated that the model prohibition on evading accessibility laws.
Which phones and sign languages will SL2T support beyond ASL on Pixel 11?
At launch, SL2T supports ASL-to-English only, and is available on Pixel 11 devices (shipping August 20, 2026). Google has described expansion to more Android devices and additional sign languages as “coming soon,” but has not committed to a specific timeline. The training corpus already spans 50-plus sign languages, so expansion is a question of evaluation and localization rather than retraining from scratch. Sign language generation — producing signing as output, not just translating it to text — is also on the long-term research roadmap.