NVIDIA's LoRA Post-Training Boosts Cosmos 3 Nano Video QA Accuracy from 54% to 93% in One Day
NVIDIA reports that a one-day LoRA post-training workflow on its Cosmos 3 Nano model improved video question-answering accuracy from 54% to 93%, with 7x fewer GPU hours than full fine-tuning.
This article was drafted with AI assistance from multiple sources and was reviewed and approved by a human editor before publication.
NVIDIA has demonstrated that post-training its Cosmos 3 Nano model with LoRA can lift video question-answering accuracy from 54.41% to 93.35% in a single day, while cutting GPU usage by roughly a factor of seven compared to full-parameter supervised fine-tuning.
The company's Cosmos 3 is an open physical AI world foundation model built on a mixture-of-transformers (MoT) architecture that handles text, images, video, ambient sound, and action tracking. It is available in two sizes: Super 64B and Nano 16B. The model's architecture consists of a Reasoner tower, a vision language model, and a Generator tower for video and action generation. According to NVIDIA, post-training targets only the Reasoner weights.
In the experiment, NVIDIA applied LoRA post-training to Cosmos 3 Nano using the Woven Traffic Safety (WTS) dataset from Toyota, which contains over 8,000 training and validation samples for a 4-way multiple-choice video question-answering task. The zero-shot baseline exact-match accuracy was 54.41%. After a single LoRA post-training run, accuracy reached 87.14%. Using TAO AutoML to sweep configurations, peak accuracy climbed to 93.35%.
NVIDIA says LoRA post-training required approximately seven times fewer GPU hours than full-parameter supervised fine-tuning for Cosmos 3 Nano. The post-training workflow employs NVIDIA TAO agent skills and the TAO library. TAO skills package task-specific knowledge for training, evaluating, optimizing, and serving models. These skills can be used with coding agents such as Codex and Claude, with setup instructions available in the NVIDIA-TAO/tao-skill-bank GitHub repository.