JioStar has integrated a conversational voice-visual assistant into JioHotstar to help viewers find films, shows, match moments and cricket statistics through natural language. The company designed the feature to operate at the platform’s scale—more than 500 million consumers and peak concurrency of 72.5 million viewers—while returning fast, relevant results.
Design tradeoffs for scale
Engineering choices were driven by a three-way tradeoff among accuracy, latency and cost, with voice interaction making low latency and reliability particularly important. JioStar reported that over 60% of users prefer voice to text, so responses had to remain timely and economically viable at high volume. The team used the OpenAI API as part of the stack and employed Codex to accelerate tasks like generating evaluation cases and system updates.
Evaluation, validation and launch
The project combined an offline evaluation framework with an in-production validation layer. Before launch, teams seeded eval datasets with domain experts—sports-magazine editors for cricket statistics—to capture the variety of serious fan queries. In production, customer behavior and failure cases feed a feedback loop to generate new evals. “Evals, evals, and evals,” Vijay Seshadri, Chief Architect at JioStar, said when describing the importance of extensive testing.
To manage answers, the team implemented guardrails and a maker-checker pattern: one model suggested a response while another LLM acted as a validator. The validator’s design itself required evaluation to ensure reliability.
Timeline and leadership lessons
Work began in October–November and the small core team launched the experience in roughly eight weeks to coincide with a major cricket tournament in February. The feature supports more than 15 languages and unifies entertainment discovery, sports moments and detailed cricket stats in a single interface.
Seshadri advised executives to focus on outcomes rather than activity, invest in production monitoring of model efficacy alongside traditional metrics such as P99 latency, and assign skilled personnel to validate and operate AI systems in production. The system was deployed to JioHotstar for the tournament after the accelerated development cycle.
Original source: OpenAI News