Smart Displays with Generative AI-Driven Voice Intelligence
Powering 10,000+ smart displays with a GenAI-driven voice assistant that delivers real-time control, regional language intelligence, and sub-2-second responses.
At a Glance
The global smart display market is forecasted to reach around USD 59.38 billion by 2034, growing at a CAGR of 20.47% from 2025 to 2034. The increasing integration of voice assistants drives this growth, the adoption of smart home technologies, and the growing use of interactive displays.
This case study discusses Azilen’s collaboration with one of its clients to develop a collaborative GenAI Smart Control Solution that can be operationalized across over 10,000 smart display devices simultaneously.
Let’s discover some details about this smart display ecosystem with a GenAI-powered virtual assistant that provides language support for Brazilian Portuguese, Argentinian Spanish, and Mexican Spanish with sub-2-second latency for the client’s target market in the LATAM region.
Key Highlights
Headless Voice Intelligence Engine
LLM-based Voice Command Intent Recognition
OpenAI Whisper Model for Speech Recognition
TV-specific Knowledge Base for Virtual Assistant
3rd Party Integrations – Weather, News, IMDb & Wikipedia
LLM & ASR Model Hosting with AWS Bedrock
Challenges
- LLM & ASR Model Evaluation & Selection with enterprise-grade strategy
- Local language support for the LATAM region customer base
- Real-time command execution with sub-2-second latency
- Cost-aware training and fine-tuning of LLM & ASR models
- Improving ASR model accuracy by reducing word error
About Client
The client is a dedicated visual digital entertainment company committed to developing, manufacturing, marketing, and supporting well-known branded TV, Smart Display, and Audio products across Europe, LATAM, and APMEA regions.
MFG.
Industry
Electronics
Domain
Netherlands
Location
How we helped?
Azilen engaged with the client to blueprint the envisioned concept from idea to execution with a voice AI solution that assists smart display users as a virtual smart assistant with a TV-specific knowledge base and language-specific adaptation for Brazilian Portuguese, Argentinian Spanish, and Mexican Spanish.
10,000+
Smart displays powered globally
95%+
Multilingual accuracy supporting LATAM regional dialects
1 Million+
Concurrent monthly voice interactions supported
The Solution
The generative AI-powered voice assistant for smart displays enables seamless processing of voice interactions—from user speech input on smart displays to intelligent command execution and contextual response delivery—all orchestrated through AWS IoT Core and developed backend microservices.
Below highlights the Functional Flow of the Solution:
1. Voice Capture via Smart Display
→ User activates the microphone on the smart display using a remote.
→ The microphone captures the voice input and converts it into an audio file.
→ The audio file is securely transferred to AWS IoT Core through MQTT.
→ The MQTT Data Listener triggers the ASR model.
2. Speech-to-Text Conversion
→ The Whisper-based Automated Speech Recognition service processes the audio input.
→ Converts the captured speech into transcribed text, with language translation if required.
→ The transcribed text is forwarded to the backend services for further processing.
The evolution of smart display ecosystems marks a shift from command-driven to conversational interfaces—where context, emotion, and personalization define engagement.
Azilen’s approach demonstrates how Generative AI, when integrated with asynchronous IoT frameworks, can transform everyday devices into adaptive digital companions.
The solution exemplifies a blueprint for future-ready entertainment systems that learn, respond, and evolve with user behavior.
3. Intent Identification Service
→ The backend Intent Recognition Service, powered by an LLM on Amazon Bedrock, analyses the transcribed text.
→ Determines whether the user request represents a Command or a Conversation.
→ Acts as the central routing logic for subsequent services.
4. Command Handling Service
→ Triggered when the intent is identified as a device command (e.g., volume control, mute, channel change).
→ The Command Handling module processes the instruction and interfaces with device functions.
→ Generates a structured command response for execution on the smart display.
5. Conversational & Integration Handling
→ Triggered for general conversational queries.
→ Routes requests through an Integration Service connected to third-party APIs such as: Weather | IMDb | News | Wikipedia
→ Uses asynchronous communication and caching for optimized response delivery.
6. Smart Display Virtual Assistant (Knowledge Layer)
→ Implemented with the Cache-Augmented Generation (CAG) mechanism.
→ Retrieves verified product knowledge, manuals, and FAQs stored in AWS S3.
→ Generates concise and user-friendly responses for product or troubleshooting queries.
7. Response Delivery & User Experience
→ The final response—whether a command confirmation or conversational reply—is published back to AWS IoT Core via MQTT.
→ Delivered to the smart display and rendered through the built-in Text-to-Speech (TTS) engine.
Critical Success Factors – Key Highlights
1. Model Analysis & Selection:
→ Conducted research and workshops to choose the best ASR and LLM models with the right mix of accuracy, cost efficiency, and performance.
2. ASR Training & Fine-Tuning (LATAM):
→ Trained the Whisper model with a step-by-step approach using regional voice data to lower word error rates and improve local dialect accuracy.
3. Model Evaluation:
→ Regular testing, feedback, and benchmarking maintained high model quality and reliable performance.
Talk to Our
Consultants
Chat with
Our Experts
Write us
an Email
