Hey Product Hunt! Kalyan here, head of sync content and marketing.
We’ve been building AI lipsync for a while now, and today we’re launching Sync-3, our most advanced model release yet.
Here’s the short version: previous lip-sync models (including our own) processed video into small, isolated pieces and stitched them together. sync-3 takes a fundamentally different approach. It creates a global understanding of a person throughout the shot and produces all the frames together. The result is consistency and realism that bridges the gap between real footage and dubbed footage.
Some things Sync 3 handles that others don’t do well:
– Close-ups and partial faces (the whole face does not need to be visible)
– Extreme angles including side profiles, over the shoulder, non-frontal
– Obstacles like hands, mics, scarves – automatically detected and handled.
– The style and emotion of the speaker is safe, not flat.
– Low light and different light scenarios.
It’s 32 times larger than our previous model (16B vs 400M parameters), supports 95+ languages, and outputs in 4K.
You can use it right now at sync.so, through our Adobe Premiere plugin, or via the API.
We think about it, from perfecting lip sync to unlocking facial reanimation, the model doesn’t just match the mouth, it understands the performance.
Try it out and let us know what you think. We answer questions here all day long.