Molmo2-8B

Molmo2-8B

Answers questions about images, video and several images at once, and can point to things it finds. Made for developers who want an open 8B vision-language model for video questions, counting and captions.

Made by allenai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Molmo2-8B is an open vision-language model from the Allen Institute for AI. It understands single images, several images and video, and it can ground answers by pointing to places in them. It is built on Qwen3-8B with SigLIP 2 as its vision backbone, and it trains on public third-party datasets.

Who it suits: It suits developers and researchers who need an open model to answer questions about video and images, count objects or caption clips.

Industry | Who uses it | How they use it | Example
Media | Video analysts | Ask what appears in a clip | Ask which animal shows up in a video
Software | App developers | Add image and video understanding to a product | Caption short clips for a library
Robotics and vision | Vision engineers | Locate objects by points in video frames | Get point coordinates for a tracked object
Research | Multimodal researchers | Test an open vision-language model on counting and captioning | Compare answers on short videos

If you only need text chat, or you cannot run an 8B model with a GPU, you can skip it.

Where to use it: It runs in Python through the transformers library on your own computer, with the card's pinned versions.

• A Python 3.11 environment, set up with conda
• Python scripts using transformers and the molmo_utils helper
• Windows, macOS or Linux machines with enough memory for the weights

Use case | Who | What they get back
Video question answering | Developers | A text answer about what happens in a video
Pointing in video | Vision teams | Point coordinates as time, x and y triplets
Counting and captioning | Researchers | Counts and short descriptions from images and clips

How to set it up: You need a conda environment with Python 3.11, transformers pinned to 4.57.1, and a few vision packages. The model loads with trust_remote_code turned on.

1. Install the Hugging Face hub tools and download the model files.
2. Create and activate the conda environment.
3. Install transformers and the vision packages.
4. Load the processor and the model.

pip install -U huggingface_hub
hf download allenai/Molmo2-8B

conda create --name transformers4571 python=3.11
conda activate transformers4571
pip install transformers==4.57.1
pip install torch pillow einops torchvision accelerate decord2 molmo_utils

How to use it: You load the processor and model, build a message with your question and a video or image, apply the chat template, then generate and decode. This first block loads both parts.

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model_id="allenai/Molmo2-8B"

# load the processor
processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
    device_map="auto"
)

# load the model
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
    device_map="auto"
)

After you build messages with a text question and a video, run the model and print the reply.

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
)

inputs = {k: v.to(model.device) for k, v in inputs.items()}

# generate output
with torch.inference_mode():
    generated_ids = model.generate(**inputs, max_new_tokens=2048)

# only get generated tokens; decode them to text
generated_tokens = generated_ids[0, inputs['input_ids'].size(1):]
generated_text = processor.tokenizer.decode(generated_tokens, skip_special_tokens=True)

# print the generated text
print(generated_text)

Tool | What it does
AutoProcessor | Prepares text, images and video for the model
apply_chat_template | Turns your messages into model inputs
process_vision_info | Prepares vision inputs for the pointing example
extract_video_points | Reads point coordinates out of the model's text

Example: You send a video with the question "Which animal appears in the video?" and the card's sample answer is a short sentence saying that penguins appear in it. A pointing request instead returns coordinates you can read as points over time.

For newcomers and experts: A vision-language model reads pictures or video along with your question and answers in text. A first try is the video question example with your own short clip.

The pointing format assumes coordinates scaled by 1000, which the card's helper code converts to pixel positions. The card claims an edge over similar open models at short videos, counting and captioning, with solid results on long videos. Training code and other artifacts are promised for later.

Load the processor and model
Build a message with a video and a question
Apply the chat template
Generate and decode the reply

Good to know: Ai2 releases the Molmo2 datasets openly and says more artifacts will follow.

• Setup: the card pins transformers to 4.57.1 in a fresh conda environment.
• Remote code: loading uses trust_remote_code, which runs custom code from the repository.
• Size: about 34.6 GB in F32, and about 4.3 GB at 4-bit as an estimate.
• License: Apache 2.0.
• Cost to run: none named. It needs your own GPU.
• Data it reaches: the images and videos you give it, and the card's sample loads a demo video from the web.
• Not for: the card names no exclusions.

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 8.7B
• Weights: about 34.6 GB in F32; about 4.3 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 45,213
• Likes: 201
• License: apache-2.0
• Languages: en
• Based on: Qwen/Qwen3-8B
• Quantized versions: 4
• Fine-tunes: 6
• Tags: transformers, safetensors, molmo2, image-text-to-text, multimodal, olmo, molmo, conversational

Links

• Model: https://huggingface.co/allenai/Molmo2-8B
• Files: https://huggingface.co/allenai/Molmo2-8B/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download allenai/Molmo2-8B

Then follow the usage in its model card. Nothing in this kit runs by itself.