Rhymes AI releases Allegro

🎬 Rhymes AI just released Allegro, the very first open-source commercial-grade video generation model!

It matches or beats most commercial solutions, ranking just behind Hailuo and Kling in overall quality.

Key insights:

🌟 Innovative two-stage architecture

▸ Video VAE converts pixels to compressed visual tokens (4x8x8 compression)

▸ Video Diffusion Transformer predicts frames with self & cross attention

📚 Training dataset

▸ 106M images, 48M videos with highly associated captions

▸ Complex filtering pipeline for top quality: aesthetics, motion, semantics…

🚀 Progressive training in 4 stages

▸ Image pre-training

▸ Basic video at 360p

▸ HD video at 720p

▸ Fine-tuning on premium data

📈 Results put it among the very best

▸ Beats all open-source models on VBench benchmark

▸ Only behind Hailuo & Kling in commercial space

▸ 91% of users preferred it to Dream Machine, 96% preferred it to OpenSora

This is huge: we now have detailed insights into what makes high-end video generation tick, from data curation to architecture choices. And it’s all open-source under Apache 2.0, available on GitHub and HF!

📸 Demo gallery 👉 rhymes.ai/allegro_gallery

Model on the Hub 👉 https://huggingface.co/rhymes-ai/Allegro