Source
@misc{sun_2026_vlajepa,
title={{VLA-JEPA}: Enhancing Vision-Language-Action Model with Latent World Model},
author={Jingwen Sun and Wenyao Zhang and Zekun Qi and Shaojie Ren and Zezhi Liu and Hanxin Zhu and Guangzhong Sun and Xin Jin and Zhibo Chen},
year={2026},
eprint={2602.10098},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2602.10098},
}
| (USTC) | arXiv |
TL;DR
…
Flash Reading
- Abstract: Pretraining VLA often learns the wrong thing: anchor to pixel variation rather than action-relevant state transitions. VLA-JEPA introduces leakage-free latent state prediction.
- Introduction: Four failure modes appear in latent-action pretraining: (1) Pixel-level objective bias (to appearance rather than action) [1]; (2) Real-world videos amplify noisy motion; (3) Future information leakage; (4) Multi-stage training is complex and fragile. The fix is JEPA, which replaces pixel reconstruction with latent state alignment.
- Related Works: Latent action learning: To leverage videos without action lablels, ILPO, LAPO, and Genie use latent action for video games. LAPA, IGOR, UniVLA, MotoGPT, Adaworld, CoMo, and StaMo extract motion tokens from frame transitions, and pretrain VLA to predict these latent actions before real-world mapping. To align the latent action with the real-world action, villa-x, XR-1, CLAP, and VITA propose to extract latent action from both robot and human videos and use a unified codebook. However, this may have future-frame leakage.
- Methodology: VLM backbone is Qwen3-VL (Qwen3 + SigLIP-2). The visual input is encoded by V-JEPA2 [2] and a time-causal attention is employed. After the encoder, the video state vectors from different camera views are concatenated. The latent state predictor (world model) takes the concatenated state vector and the latent action (last hidden feature) from the VLM to predict the next state vector. Time causal attention is also used in the latent world model, and for any time step, the model can only access the current and previous states.