Llama-V-JEPA-LM-Llama3.2-3B

Llama-V-JEPA-LM-Llama3.2-3B

Answers questions about video clips with a 4.2B model that joins a frozen V-JEPA 2.1 video encoder to Llama-3.2-3B-Instruct, for people who need video understanding on a single GPU of 12 GB or more.

Made by kkt94 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a video-language model, so video and text go in and text comes out. A frozen V-JEPA 2.1 video encoder passes what it sees to Meta's Llama-3.2-3B-Instruct through a small bridge. A clip of 32 frames becomes 1,024 visual tokens, and training drew on 4.53M public samples.

Who it suits: Researchers and engineers who want video question answering from a model of about 4B parameters will find it fits. It also suits students who want to read how a video encoder joins a language model.

Industry | Who uses it | How they use it | Example
Research | Video-language researchers | Compare a mid-size video model with other open models of its size | Testing how well it reads direction, speed and order
Robotics and simulation | Engineers | Ask how a scene changes from frame to frame | Asking which way an object moved
Media | Video analysts | Put questions to short clips and read the text answers | Asking what happens in a clip
Education | Students | Study how a world-model encoder joins a language model | Reading the architecture notes in the card

If you do not work with video, you can skip it. It also carries Meta's licence terms, which you should read first.

Where to use it:

• Python scripts, using the transformers library that Hugging Face lists for it
• Windows, macOS or Linux, on a machine with a CUDA GPU
• A GPU with 12 GB or more, which the card names for running it

Use case | Who | What they get back
Describe a video | Anyone with a clip | A free-form text description
Pick an option | Researchers | The index of the chosen option and a score for each option
Motion reading | Engineers | An answer about direction, speed, order or state

How to set it up: Start with Python 3.10 or later. PyTorch must be a CUDA build suited to your graphics driver. The card says one clip peaked at 8.5 GiB of memory in its own test. The weights are about 8.5 GB in BF16, or about 2.1 GB at 4-bit as an estimate.

• Python 3.10 or newer
• A CUDA build of PyTorch
• A GPU with 12 GB or more
• A read of the Llama 3.2 licence and acceptable use policy

1. Install PyTorch.
2. Install the other packages the card lists.
3. Load the model in Python and move it to the GPU.

pip install torch
pip install "transformers>=5" safetensors tokenizers numpy jinja2 eva-decord huggingface_hub

How to use it: Load the model, then call its chat method with the path to a video file and a question. It answers in free-form text using greedy decoding. Its choose method also takes a list of options.

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "kkt94/Llama-V-JEPA-LM-Llama3.2-3B",
    trust_remote_code=True, dtype=torch.bfloat16,
).to("cuda").eval()

# free-form answer (greedy decoding)
print(model.chat("my_video.mp4", "Describe what happens in the video."))

# multiple choice -> (index of the chosen option, one score per option)
idx, scores = model.choose("my_video.mp4", "Which way does the ball roll?", ["to the left", "to the right"])
print(idx, scores)

Tool | What it does
model.chat | Takes a video path and a question and returns a text answer
model.choose | Takes a video path, a question and options, and returns the chosen index and a score per option
AutoModel.from_pretrained | Loads the model in BF16, with the repository's own code allowed to run

Example: You pass my_video.mp4 with a question about which way the ball rolls and two options, to the left and to the right. The model returns the index of the option it picks and a score for each.

For newcomers and experts: A video-language model answers questions about video. A good first step is to run the quick start on one short clip of your own and read the description it writes.

The encoder is a frozen ViT-g with 1.01B parameters. The bridge projects its features onto 2,048 fixed PCA directions, then uses a trainable MLP of 11.0M parameters. The language model is fully fine-tuned at 3.21B, for 4.25B in total. Training uses counterfactual supervision, where pixel changes to public videos create variants whose answers are fixed by construction.

A clip of 32 frames at 384 by 384
The frozen V-JEPA 2.1 encoder reads it
The spectral bridge makes 1,024 visual tokens
Llama-3.2-3B-Instruct reads tokens and question
The model writes a text answer

Title: Average over 12 video scores, as the card reports
Llama-V-JEPA-LM-Llama3.2-3B: 68.97 points
Molmo2-4B: 61.08 points

Good to know:

• License: Llama 3.2 Community License, shown as llama3 on Hugging Face
• The V-JEPA 2 encoder is MIT
• Cost to run: none named
• Data it reaches: the video file you pass in, and the source names no web or account access
• The loading code sets trust_remote_code, so the repository's own code runs on your machine
• Not for: uses that Meta's Llama 3.2 Acceptable Use Policy forbids
• English is the only listed language

On Hugging Face, 13 November 2026

• Task: video text to text
• Library: transformers
• Parameters: 4.2B
• Weights: about 8.5 GB in BF16; about 2.1 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 109
• Likes: 7
• License: llama3.2
• Languages: en
• Based on: meta-llama/Llama-3.2-3B-Instruct
• Tags: transformers, safetensors, jepa_llm, feature-extraction, video, video-question-answering, video-language-model, multimodal, v-jepa, world-model, llama, llama-3

Links

• Model: https://huggingface.co/kkt94/Llama-V-JEPA-LM-Llama3.2-3B
• Files: https://huggingface.co/kkt94/Llama-V-JEPA-LM-Llama3.2-3B/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download kkt94/Llama-V-JEPA-LM-Llama3.2-3B

Then follow the usage in its model card. Nothing in this kit runs by itself.