GitHub - wpydcr/NanoAvatar: Real-time, high-quality talking avatars, even on a old Android phone: 41 FPS with just 103 ms to the first frame. · GitHub

What

NanoAvatar is an open-source, real-time talking-avatar generation engine designed to run locally on mobile devices and desktop GPUs without relying on cloud rendering servers. Targeted at edge-AI engineers and interactive application developers, it resolves the high infrastructure costs and network latency associated with server-side avatar video generation. It is positioned as a lightweight, low-latency lip-sync framework capable of delivering real-time rendering on consumer edge hardware.

Features

• On-Device Mobile Inference: Delivers localized video generation on Android chipsets via dedicated Full and Lite packages, eliminating the need for recurring cloud GPU rendering costs.
• Low-Latency Streaming Animation: Begins rendering lip-synced video in approximately 0.3 seconds when fed by streaming text-to-speech pipelines, minimizing response latency during live interactions.
• Quantized Desktop Runtime: Executes mixed INT8 and W8A16 TorchScript models directly in PyTorch on NVIDIA GPUs, avoiding the operational complexity of custom CUDA dynamic-link libraries.
• Conversational AI Mode: Connects directly with Alibaba Cloud DashScope APIs (supporting Qwen language models and CosyVoice text-to-speech) to drive immediate, voice-based avatar dialogues.

Helpful Tips

• Review Model Licensing Terms: While the base code is MIT-licensed, the core lip-sync model weights fall under CC BY-NC 4.0, which prohibits commercial use without contacting the author for explicit licensing.
• Match Target Hardware to Model Variants: Deploy the Lite model package when targeting older hardware like the Snapdragon 8 Gen 1 to maintain acceptable memory footprints and responsive frame rates.
• Verify Web Runtime Prerequisites: Ensure the deployment host is running Python 3.11 and compatible PyTorch CUDA builds to prevent runtime environment mismatches during local Web setup.

OpenClaw Skills

The NanoAvatar repository does not document any native integration with the OpenClaw ecosystem. As a hypothetical use case, an OpenClaw agent could be configured to route generated speech audio directly into a local NanoAvatar runtime, allowing autonomous agent workflows to output animated visual video feeds on edge devices. This architecture could enable automated customer service or kiosk applications to provide interactive visual agents entirely on local hardware without incurring external streaming video infrastructure fees.