Context

VLA-0 is, in my opinion, the simplest VLA model currently available, and perhaps the simplest a VLA can be. I reviewed the paper in a previous blog post. It uses the Qwen2.5 VLM as its backbone and predicts robot actions directly as text integers. What fascinated me was that, despite this simplicity, it achieves remarkably strong performance, outperforming several much more complex architectures.

This is exciting because simpler systems are generally more reliable, maintainable, and easier to understand. For me, another major advantage is that a straightforward design like VLA-0 is much easier to modify and experiment with. The authors also released both their code on GitHub and the trained model weights for the LIBERO benchmark on Hugging Face.

Before experimenting with modifications or new ideas, I first wanted to reproduce the results reported in the paper, both in simulation using LIBERO and in the real world. Successfully reproducing these results would validate my training setup, Python environments, robot, datasets, and evaluation pipeline, while also providing a reliable baseline against which I could compare future experiments.

First Attempts

I first downloaded the published LIBERO model and reproduced the evaluation results reported in the paper, confirming that my LIBERO evaluation pipeline was working correctly. The results matched those reported in the paper.

I then attempted to fine-tune the model on a 200-episode real-world dataset that I had previously collected using my SO101 robot. However, the resulting policy produced almost no meaningful movement. Since differences between the training and deployment environments may have contributed to the failure, I decided to postpone reproducing the real-world experiments and instead focus on retraining the LIBERO model, where the dataset and evaluation pipeline were already established.

Without access to the 8 A100 GPUs used in the paper, a full reproduction on my available hardware would have taken around two weeks. To speed up experimentation, I initially trained on a small subset of the LIBERO benchmark using LoRA. Although the training loss converged, evaluation achieved 0% success, with the robot making only slow, purposeless movements. Switching to full fine-tuning on the complete LIBERO Spatial benchmark produced almost identical results, leaving the cause of the failure unclear.

While waiting for access to a larger GPU cluster, I collected a new 400-episode real-world dataset closely matching the setup used in the VLA-0 paper and released it on Hugging Face. Fine-tuning the published VLA-0 model on this dataset using LoRA achieved encouraging real-world results, with success rates of approximately 10–20%, suggesting that the overall training pipeline was functioning correctly.

My Dataset

I collected a 400 episode dataset avilable to view on my Hugging Face. I aimed to recreate the dataset used by the VLA-0 authors so my reproduction would be comparable. They collected 100 teleoperated demos for 4 tasks. Whilst I did not have the exact same objects available, I matched their tasks as close as possible.

Their Task My Task Episodes
Pick up cupcake and place in bowl Pick up juggling ball and place in bowl 100
Pick up banana and place on plate Pick up sponge and place on cloth 100
Push the apple to the block Push the apple to the block 100
Reorient the block upright Reorient the block upright 100

I collected my dataset with 5 cameras, depth values, current and motor velocity values just so i had them in case i wanted to do something with that data, but during training i just used 2 cameras observations as they did in the paper.

World camera view (example demos of pick up juggling ball task).

Wrist camera view (example demos of reorient block task).

Full LIBERO Reproduction

Eventually, my supervisor secured me access to the Isambard-AI supercomputer, which hosts 5,448 NVIDIA GH200 Grace Hopper Superchips, over 20 pebibytes of storage, and delivers more than 21 exaFLOPS of AI compute performance. After setting up my fork of the repository and recreating my Python environments, I was finally able to perform a full fine-tuning run using the original LIBERO setup in an attempt to fully reproduce the results reported in the VLA-0 paper.

(results on epoch 8 of this on libero show good learning in all suites) (also try to eval on epoch 16) (also try to eval with longer action horizon usage / ensembling (no need to retrain for these results)) to be continued …

Full Real World Reproduction

Now that I had access to a large GPU cluster, I setout to recreate the training setup used by the authors as close as possible. My training configs can be found here and my dataset config here.

Noteable differences from the papers original configs:

  • LR_SCHED: cosine_anneal instead of none — The paper kept the learning rate constant throughout training. I used cosine annealing, which smoothly decays the learning rate following a cosine curve towards zero by the final epoch. This is a widely-used heuristic that tends to help convergence in the later stages of training.

  • clip_grad_norm: 1.0 instead of 0.0 — A value of 0.0 disables gradient clipping entirely. I clipped gradients to unit norm, which prevents occasional very large gradient updates from destabilising training — particularly important when fine-tuning a large vision-language model where gradient magnitudes can spike.

  • l2: 0.01 instead of 1e-10 — L2 regularisation (weight decay) penalises large weights during training. The paper’s value of 1e-10 is effectively zero — no regularisation. I used 0.01, which adds a modest regularisation pressure to reduce the risk of overfitting on a relatively small real-world dataset.

  • Per-GPU batch_size: 24 (global batch 96) instead of 8. The paper reports a global batch size of 192, which across 4 GPUs would require a per-GPU batch size of 48 — this was too large to fit in memory with a 3B parameter model. I used 24 per GPU (global 96), which fits comfortably on 4× H100 80GB GPUs with gradient checkpointing enabled.

  • grad_checkpoint: True — Gradient checkpointing trades compute for memory. Instead of storing all intermediate activations during the forward pass (which grows linearly with sequence length), the model recomputes them on-the-fly during the backward pass. This roughly halves GPU memory usage at the cost of ~30% longer training time, and was necessary to fit the model and the above batch size in GPU memory.

Here is the loss graph. It takes ~1 hour 50mins for each EPOCH on 4 H200 GPUs.

Training Loss — VLA0 Qwen2.5-VL-3B Real-World
Average cross-entropy loss per epoch · Epochs 0–44 complete (45 of 100)

I evaluated the model at 28 epochs on my SO101, which can be found on my HuggingFace. The results were very inspiring with an average 10% increase in success rate compared to the results reported by the paper. This is also exciting because at epoch 28 the loss was still steadily decreasing suggesting more performance might be available by further training. I included the reported success rates of SmolVLA and the original papers VLA-0 results alongside my results as a comparision. As you can see the results even show a great increase over SmolVLA which was pre trained on a large amount os SO100/SO101 data.

Inference took 1.56 seconds on average for an 8 action chunk on a single RTX6000 Pro GPU. I created my own robot_client.py based on the example client provided in the original repo alongside the original service.py as the server.

VLA-0 Results Reproduced

Evaluation rollouts for push apple to block task (x15 speed).

Evaluation rollouts for place sponge on cloth task (x15 speed).

Evaluation rollouts for put juggling ball in bowl task (x15 speed).

Evaluation rollouts for reorient block upright task (x15 speed).

Whilst there is no obvious reason that my results were better than the paper reported, there are a few possible explanations. Perhaps the most likley in my opinion would that I used a world and wirst camera. The paper used a left and right scene camera which might have hindered the models ability to be precise on grasping.

It is also important to note that the original authors did not provide their dataset so I can not be sure that our demonstartion collection was identical. For example the authors provided no details and how much they varied the positions of the objects in their dataset, or in their evaluation. For my dataset and evaluation I kept consistent if the objects were on the left or right of each other (ie apple always on the left of the block) but I varied their starting positions quite significantly around the reachable workspace.

The difference could be a result of the different learning parameters however I find this unlikley as the setup is very similar, and my evaluation was completed with less epochs. However again the authors did not provide their loss graph, final loss value or any other inidcation of how their model learnt so we can not be sure.

Conclusion

Overall, I was able to successfully reproduce the VLA-0 results in both simulation and the real world. Along the way I encountered several failed training runs, but these ultimately helped validate my training and evaluation pipeline and led to a closer understanding of the implementation. Most encouragingly, my real-world reproduction slightly outperformed the results reported in the original paper, demonstrating that VLA-0’s simple architecture can indeed achieve strong performance. With a reliable baseline now established, I can begin exploring modifications and new ideas with confidence.