Llama-V-JEPA-LM-Qwen3-4B

Llama-V-JEPA-LM-Qwen3-4B

Answers questions about video clips with a 5.1B model that joins a frozen V-JEPA 2.1 video encoder to Qwen3-4B-Instruct-2507, for people who need video understanding on a GPU with 16 GB or more.

Made by kkt94 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a video-language model, so video and text go in and text comes out. Its video encoder, V-JEPA 2.1, stays frozen and passes its output across a small bridge to Qwen3-4B-Instruct-2507, the text side of the model. A clip of 32 frames becomes 1,024 visual tokens, and training drew on 4.60M public samples.

Who it suits: Researchers and engineers who want video question answering from a model of about 5B parameters will find it fits. It also suits students who want to read how a video encoder joins a language model.

Industry | Who uses it | How they use it | Example
Research | Video-language researchers | Compare a mid-size video model with other open models of its size | Testing how well it reads direction, speed and order
Robotics and simulation | Engineers | Ask how a scene changes from frame to frame | Asking which way an object moved
Media | Video analysts | Put questions to short clips and read the text answers | Asking what happens in a clip
Education | Students | Study how a world-model encoder joins a language model | Reading the architecture notes in the card

If you do not work with video, you can skip it. Its own licence is Apache-2.0, but its training data brings a Llama 3.2 condition.

Where to use it:

• Python scripts, using the transformers library that Hugging Face lists for it
• Windows, macOS or Linux, on a machine with a CUDA GPU
• A GPU with 16 GB or more, which the card names for running it

Use case | Who | What they get back
Describe a video | Anyone with a clip | A free-form text description
Video question answering | Researchers | A text answer to a question about a clip
Motion reading | Engineers | An answer about direction, speed, order or state

How to set it up: Start with Python 3.10 or later. PyTorch must be a CUDA build suited to your graphics driver. The card says one clip peaked at 10.0 GiB of memory in its own test. The weights are about 10.1 GB in BF16, or about 2.5 GB at 4-bit as an estimate.

• Python 3.10 or newer
• A CUDA build of PyTorch
• A GPU with 16 GB or more

1. Install PyTorch.
2. Install the other packages the card lists.
3. Load the model in Python and move it to the GPU.

pip install torch
pip install "transformers>=5" safetensors tokenizers numpy jinja2 eva-decord huggingface_hub

How to use it: Load the model with the AutoModel class from transformers, then give it a video file and a question. It answers in text. The exact call is on the model page.

Example: You give the model a short clip and ask what happens in it. It answers in written text.

For newcomers and experts: A video-language model answers questions about video. A good first step is to run the quick start on one short clip of your own and read the description it writes.

The encoder is a frozen ViT-g with 1.01B parameters. The bridge projects its features onto 2,048 fixed PCA directions, then uses a trainable MLP of 9.8M parameters. The language model is fully fine-tuned at 4.02B, for 5.06B in total. Training uses counterfactual supervision, where pixel changes to public videos create variants whose answers are fixed by construction.

A clip of 32 frames at 384 by 384
The frozen V-JEPA 2.1 encoder reads it
The spectral bridge makes 1,024 visual tokens
Qwen3-4B-Instruct-2507 reads tokens and question
The model writes a text answer

Title: Average over 12 video scores, as the card reports
Llama-V-JEPA-LM-Qwen3-4B: 72.14 points
Molmo2-4B: 61.08 points

Good to know:

• License: Apache-2.0, shown as apache-2 on Hugging Face
• The V-JEPA 2 encoder is MIT, and the FGQA training data follows the Llama 3.2 Community License, which is why the name starts with Llama
• Cost to run: none named
• Data it reaches: the video file you pass in, and the source names no web or account access
• Not for: none named
• English is the only listed language

On Hugging Face, 13 November 2026

• Task: video text to text
• Library: transformers
• Parameters: 5.1B
• Weights: about 10.1 GB in BF16; about 2.5 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 78
• Likes: 7
• License: apache-2.0
• Languages: en
• Based on: Qwen/Qwen3-4B-Instruct-2507
• Tags: transformers, safetensors, jepa_llm, feature-extraction, video, video-question-answering, video-language-model, multimodal, v-jepa, world-model, qwen3, video-text-to-text

Links

• Model: https://huggingface.co/kkt94/Llama-V-JEPA-LM-Qwen3-4B
• Files: https://huggingface.co/kkt94/Llama-V-JEPA-LM-Qwen3-4B/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download kkt94/Llama-V-JEPA-LM-Qwen3-4B

Then follow the usage in its model card. Nothing in this kit runs by itself.