Jocasta-diff-VLA

Jocasta-diff-VLA

A robot control policy that adapts Dream-VLA-7B to eight simulated Panda arm task families in robosuite, using a Flow Matching action head. For robotics researchers with a CUDA GPU.

Made by RobotLearningLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Jocasta-diff-VLA is a robot control policy built from Dream-VLA-7B. It targets eight simulated Panda arm task families in the robosuite simulator and pairs a diffusion language model backbone with a continuous Flow Matching action head. One jointly trained policy covers every family, and it succeeded in 332 of 512 evaluation episodes, or 64.8 percent.

Who it suits: Researchers and students who work on vision-language-action models for robot arms, and who have a CUDA GPU to run it.

Industry | Who uses it | How they use it | Example
Robotics research | VLA researchers | Reproduce or extend the adapted policy | Load the checkpoint and predict actions for a robosuite scene
Simulation | People testing Panda arm tasks | Use the policy as a baseline | Compare success rates across the eight task families
Academia | Students of action generation | Study the training setup and settings | Read how the LoRA and Flow Matching settings were chosen

You can skip it if you have no GPU, no interest in robosuite, or want a one-line load, since it needs a custom package.

Where to use it:

• A machine with one visible CUDA GPU and Python
• The robosuite simulator, with rendering on a separate physical GPU
• Research code that builds on the companion repository

Use case | Who | What they get back
Predict robot actions | Robotics researchers | A chunk of eight actions from two camera images, proprioception and a text instruction
Benchmark a transferred policy | Evaluation authors | Per-task success rates to compare against their own runs
Study the adaptation | Students | The training settings, from the base model to the LoRA setup

How to set it up: Start with what you need, then follow the steps. The base Dream-VLA weights are required separately, and the loader downloads them with the checkpoint when needed.

• Python and git
• One CUDA GPU for inference
• The companion code repository with its pinned submodules and patch

1. Clone the companion repository together with its submodules.
2. Enter the cloned folder.
3. Apply the patch to the Dream-VLX submodule.
4. Install the pinned requirements.

cd Diffusion-VLA-tuning
git -C third_party/Dream-VLX apply ../../patches/dream-vlx-panda.patch
python -m pip install -r requirements.lock -r requirements-multitask.txt

To fetch only this checkpoint's files yourself:

pip install -U huggingface_hub
hf download RobotLearningLab/Jocasta-diff-VLA

How to use it: Load the policy with the companion code, then ask it for actions. You pass two RGB camera images of 224 by 224, eight proprioception values and an instruction. Pass a Hub commit with revision to pin the exact files, or point base_model_path at a local copy of the base.

from load_policy import load_policy, predict_action

policy, processor, cfg = load_policy('RobotLearningLab/Jocasta-diff-VLA')
# primary and wrist: uint8 RGB arrays, each (224,224,3)
# proprio: float32 (8,), xyz + axis-angle orientation + two gripper joints
actions = predict_action(policy, processor, cfg, primary, wrist, proprio,
                         'pick up the red cube', seed=84907)
# actions.shape == (8,7)

Tool | What it does
load_policy | Builds the policy, processor and config from the model name
predict_action | Returns eight actions of seven values for the given images, state and instruction

Example: You give the policy the primary and wrist camera images, the eight proprioception values and the instruction "pick up the red cube", with seed 84907. It returns an array of eight actions with seven values each. The output still has to be turned into the controller commands the Panda arm expects, as the companion code describes.

For newcomers and experts: If you are new to this, think of it as research code for a simulated robot arm, not an app. A first small step is to read the companion repository and run the load call above on a machine with a CUDA GPU.

For experts, this is a LoRA adaptation with rank 32 and alpha 16 on all linear layers. It trained for 100,000 updates and samples 4 Flow Matching steps per chunk of 8 actions. The trained action head and projectors ship with the checkpoint, so it is not a plain language adapter. The original Dream-VLA scored 0 of 64 on the PickPlace scenes and this one scored 40 of 64. The two policies differ in inputs and action design, so the gap is not an isolated backbone effect. Results use one training seed and canonical instructions only.

Two camera images plus proprioception
Instruction text
Dream-VLA backbone with LoRA
Flow Matching head, 4 steps
Chunk of 8 actions

Good to know:

Title: Success rate per task family on the fixed development scenes
Lift: 100.0%
Push: 95.3%
Drawer: 76.6%
Stack: 67.2%
PickPlace: 62.5%
Door: 53.1%
NutAssembly: 50.0%
ToolHang: 14.1%

• Base weights: the original Dream-VLA base is needed separately and the loader fetches it
• Loading: use the companion loader, because generic model loading does not rebuild the trained action policy
• Data: raw demonstration data is not included, and no optimizer state is shared
• Evaluation: one training seed, 64 fixed scenes per family, and no held-out-language test
• Rendering: in simulation, give model inference and rendering their own physical GPUs
• License: Apache 2.0
• Cost to run: no API key or paid service named, but it needs a CUDA GPU
• Data it reaches: downloads the checkpoint and base weights from Hugging Face, nothing else is named
• Not for: loading with AutoModel, or as a result on held-out instructions

On Hugging Face, 13 November 2026

• Task: robotics
• Library: pytorch
• Formats: Safetensors
• Downloads in the last 30 days: 36
• Likes: 33
• License: apache-2.0
• Languages: en
• Based on: Dream-org/Dream-VLA-7B
• Tags: pytorch, safetensors, vision-language-action, robotics, robosuite, lora, flow-matching, diffusion-language-model, simulation

Links

• Model: https://huggingface.co/RobotLearningLab/Jocasta-diff-VLA
• Files: https://huggingface.co/RobotLearningLab/Jocasta-diff-VLA/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download RobotLearningLab/Jocasta-diff-VLA

Then follow the usage in its model card. Nothing in this kit runs by itself.