Tensorial T1 Live: a real-time multimodal conversation model

Tensorial T1 Live is our real-time multimodal conversation model: you talk to it with your voice, it sees through your camera, and it talks back — at human-conversation latency. It is already in production as the engine behind the real-time conversation feature of the Visionauta app, our visual assistant for blind and low-vision users, where every millisecond of latency and every perception mistake has real consequences.

This post explains what T1 Live does, how we think about the latency problem, and brings two things we'd rather show than describe: a live terminal — you can talk to the model right now, on this page — and our first numbers on a public benchmark for proactive streaming video understanding.

Try it now

The demo connects to the same endpoint that serves the Visionauta app in production. Sessions are limited to 5 minutes and capacity is shared with real users — if it's full, try again in a few minutes.

What a "live" model is

A live model is not a chatbot with a microphone. It is a system that keeps a continuous bidirectional connection (WebSocket) where audio, video and text flow in both directions at the same time. That changes the engineering requirements radically:

• You don't know when the user finished speaking. Detect the end of turn too fast and you cut people off; too slow and you create awkward silences.
• The user can interrupt the answer at any moment (barge-in) — and the model must go quiet instantly, mid-sentence.
• The camera is living context, not an attachment: the scene changes while the conversation happens.
• Mobile connections drop — and the conversation must not be lost with them.

Latency is the product

In spoken conversation, the time between the end of your speech and the first sound of the answer (time to first audio, TTFA) decides whether the interaction feels like a conversation or like a walkie-talkie interrogation. In production, T1 Live answers with a TTFA of ~1.2 seconds, measured end to end in real sessions — from the user's last phoneme to the first audio sample of the reply, network included.

We got there with a fully streaming pipeline — no stage waits for the previous one to finish — and two endpointing tricks worth detailing:

Speculation inside the VAD hangover

Every voice activity detector waits a few hundred milliseconds of silence before declaring "the user is done". That period (the hangover) is dead time in most systems. In T1 Live it is working time: the moment silence begins, speech recognition and the model already start generating the answer covertly, behind a send gate. If the user resumes speaking, everything is silently discarded — they never know. If the silence is confirmed, the answer is already there, ready to go. That's hundreds of milliseconds recovered on every single turn.

Adaptive endpointing

The silence wait is not fixed: when the partial transcript ends in final punctuation — a complete sentence — the turn closes earlier; when it ends mid-thought, the system waits longer. Together with barge-in (talk over it and it stops instantly), the result is a conversation rhythm that doesn't feel like software.

Inside the model

The brain of T1 Live is a multimodal language model fine-tuned in-house by Tensorial, adapted for streaming spoken conversation: short answers with speech prosody (not chat text), proactivity discipline, and mid-turn tool use. Around it, a fully streaming pipeline handles the rest: voice activity detection, incremental speech recognition, and sentence-by-sentence speech synthesis — each stage hands off to the next as soon as the first piece is ready.

Real-time vision

T1 Live receives continuous camera frames and treats them as living conversation context: you can ask "what's written here?", "what color is this shirt?", "is this gate open or closed?" just by pointing the camera. Redundant frames are deduplicated at the edge of the pipeline so a static scene doesn't burn compute — cost grows with visual change, not with time.

Proactivity: the task sentinel

The model's newest capability is speaking without being asked — at the right moment. You say "let me know when the bus shows up" and T1 Live silently watches the scene, deciding at every moment whether it's time to speak. It is a deceptively hard problem: speaking early is a false alarm, speaking late is useless, and narrating the scene non-stop is unbearable. How we measure this is in the benchmark section below.

Production robustness

• Session resumption: if the connection drops (elevator, tunnel, network switch), the session resumes with full context — the conversation continues where it left off.
• Multilingual with automatic detection: the model detects the spoken language on every turn and answers in the same language — Portuguese, English, Spanish, French and Korean, zero configuration.
• Voices: two product voices, Juliana and Marco, with expressive streaming synthesis.
• Tool calling: the model calls client-declared functions in the middle of spoken conversation, with streaming preserved.
• Polite shutdown: session limits warn first (goAway) instead of dropping the connection.

Compatible with the SDK you already use

T1 Live implements the Gemini Live API's BidiGenerateContent contract over WebSocket. In practice: the official google-genai SDK works without patches — just point it at the endpoint. Migrating an existing application is a two-line change:

from google import genai

client = genai.Client(
    api_key=YOUR_T1_KEY,
    http_options={"base_url": "https://<your-t1-endpoint>"},
)

async with client.aio.live.connect(
    model="models/tensorial-t1-live",
    config={"response_modalities": ["AUDIO"]},
) as session:
    ...  # audio, video, tools — the same code as before

Benchmark: OmniPro

Proactivity was the hardest capability to measure — until OmniPro (2026), the first public benchmark dedicated to proactive streaming video understanding: 2,700 human-verified samples across 9 sub-tasks. In online mode — the one that matters for a live model — the model receives the video at real-time pace and decides on its own when to speak; scoring requires speaking at the right moment (±3s tolerance) with the right content.

We evaluated T1 Live on OmniPro's three proactive-alert tasks (instant event alert, semantic condition alert, realtime state monitor), using the samples of those tasks that do not strictly depend on the video's audio track — T1 Live watches the world through a camera, and our connector's protocol sends frames only. Video is streamed at real-time pace (1 fps), exactly like a live camera. The numbers below are partial (48 of the 218 evaluable samples; the full run is in progress and this post will be updated with the final results).

Model (online mode)	Event Alert (F1)	Condition Alert (F1)	State Monitor (F1)
Tensorial T1 Live (partial)	47.1	15.4	42.4
MiniCPM-o 4.5 (9B)	44.2	33.1	24.3
MMDuet2 (3B)	12.5	21.4	14.9
LiveStar (8B)	9.7	14.7	0.0

The median latency between the event happening on screen and T1 Live starting to speak was 2.5 seconds.

Methodology notes: the other models' numbers come from Table 2 of the paper (measured by the authors on the full dataset, including samples that require the audio track — where those models can score and our connector doesn't compete). Our numbers use the same greedy ±3s temporal matching of the paper's online protocol, computed by our own harness, with automatic-judge content verification. Comparisons across different harnesses carry inherent noise; we publish the connector and raw results for anyone who wants to reproduce. We are expanding the evaluation to the other public benchmarks for live models — Full-Duplex-Bench (full-duplex turn-taking and latency) and VoiceBench (voice assistants) — and will publish the results here.

Where this already works

T1 Live is not a lab prototype: it is the production engine behind the real-time conversation in Visionauta, used by blind and low-vision people to understand the world around them — reading letters, identifying products, describing places, navigating new situations with a talking pair of eyes in their pocket. It is the most demanding use case we could imagine for a live model, and it calibrates every latency and robustness decision described above.

Partners: request access

We are accepting partners to test Tensorial T1 Live — companies and developers who want to build real-time voice-and-vision experiences (customer service, accessibility, assisted monitoring, education, robotics). Access is whitelist-based: tell us what you want to build and we'll get in touch.