SmolVLM2-2.2B-Instruct

SmolVLM2-2.2B-Instruct

Answers questions about videos, images and text together, with a 2.2B model that needs about 5.2 GB of GPU memory for video. For developers building visual assistants for on-device use.

Made by HuggingFaceTB on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: SmolVLM2-2.2B-Instruct is a compact multimodal model from Hugging Face. You give it text with a video, one image or several images, and it writes text back. It can caption, answer questions about what it sees, compare pictures and read text out of images. It does not generate images or video.

Who it suits: It suits developers who want to describe or question visual content with a small model, and who care about memory use on their own hardware.

Industry | Who uses it | How they use it | Example
Software | App developers | Add visual question answering to an app | Asking what is in a photo a user uploads
Media | Video teams | Describe or summarize clips automatically | Drafting a short summary of a recording
Accessibility | Tool builders | Turn images into spoken or written descriptions | Captioning pictures for screen readers
Research | Multimodal researchers | Fine-tune a small vision-language model | Adapting it to a document type

Skip it if you want to make images or video, since the card says the model does not support generating either.

Where to use it:

• Python with the transformers library and a CUDA GPU, as the card's setup does
• Video analysis, which needs the decord package
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Image description | Developers | A written description of one or more images
Video questions | Media teams | Text answers about what happens in a clip
Text reading | Document tools | Text transcribed from an image

How to set it up: You need these first.

• Python with pip and a CUDA GPU
• The num2words and flash-attn packages and a recent transformers release, as the card asks
• The decord package for video input
• About 5.2 GB of GPU memory for video, and about 9 GB of weights in F32

1. Install the packages the card lists.
2. Load the processor and the model in bfloat16 on the GPU.
3. Build a message with your media and a text question.
4. Apply the chat template and generate.

pip install -U huggingface_hub
hf download HuggingFaceTB/SmolVLM2-2.2B-Instruct

How to use it: Load the model once, then pass messages that mix a video or image with your question. The card shows image and video examples that decode the reply after generate. This one asks for a detailed description of a video file.

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model_path = "HuggingFaceTB/SmolVLM2-2.2B-Instruct"
processor = AutoProcessor.from_pretrained(model_path)
model = AutoModelForImageTextToText.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    _attn_implementation="flash_attention_2"
).to("cuda")

messages = [
    {
        "role": "user",
        "content": [
            {"type": "video", "path": "path_to_video.mp4"},
            {"type": "text", "text": "Describe this video in detail"}
        ]
    },
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, dtype=torch.bfloat16)

generated_ids = model.generate(**inputs, do_sample=False, max_new_tokens=64)
generated_texts = processor.batch_decode(
    generated_ids,
    skip_special_tokens=True,
)

print(generated_texts[0])

Tool | What it does
AutoProcessor | Prepares text, images and video for the model
AutoModelForImageTextToText | Loads the model weights
apply_chat_template | Builds the model's input from your messages
model.generate | Writes the reply, here up to 64 new tokens

Example: Point the video message at a clip and ask for a detailed description. The model returns a short written description of what happens, capped at 64 new tokens in the card's script, so raise the limit if you want more detail.

For newcomers and experts: If you are new, this model looks at pictures or video and answers in words. Start with one image and the question about what it shows, before trying a video, since video needs extra setup.

If you know the field, the architecture is based on Idefics3, and the model is a successor to SmolVLM 2.2B. The card's vision table shows gains on Mathvista, MMMU, OCRBench and MMStar, but lower scores on AI2D and ChartQA than the older model. Video scores climb with size: 52.1 on Video-MME for 2.2B, against 42.2 for 500M and 33.7 for 256M.

Video or images plus a text question
Processor builds the model input
SmolVLM2 reads media and text together
Model generates tokens
Text answer is decoded

Good to know:

• Language: English only
• Size: 2.2B parameters in Safetensors, with 500M and 256M siblings in the family
• Fine-tuning: the card points to a tutorial for adapting it to a task
• Community: other people have made 20 quantized and 48 fine-tuned versions

Title: Video-MME score by model size, from the card
2.2B: 52.1 points
500M: 42.2 points
256M: 33.7 points

• License: Apache 2.0
• Cost to run: none named, the source lists no API key, account or paid service
• Data it reaches: the images and videos you pass to your own script, and the source names no web or account access
• Not for: generating images or video, since the card says it supports neither

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 2.2B
• Weights: about 9 GB in F32; about 1.1 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 68,571
• Likes: 340
• License: apache-2.0
• Languages: en
• Based on: HuggingFaceTB/SmolVLM-Instruct
• Quantized versions: 20
• Fine-tunes: 48
• Tags: transformers, safetensors, smolvlm, image-text-to-text, video-text-to-text, conversational

Links

• Model: https://huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct
• Files: https://huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download HuggingFaceTB/SmolVLM2-2.2B-Instruct

Then follow the usage in its model card. Nothing in this kit runs by itself.