SmolLM2-135M-Instruct

SmolLM2-135M-Instruct

Answers chat-style questions and handles rewriting and summaries with a 135M-parameter instruction-tuned model small enough for on-device use. For developers who need a tiny chat model in Python or JavaScript.

Made by HuggingFaceTB on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: SmolLM2-135M-Instruct is the chat-tuned version of the 135M model in the SmolLM2 family. The team built it from the base model with supervised fine-tuning on public and curated datasets, then applied Direct Preference Optimization. It takes messages in a chat format and replies in English.

Who it suits: It suits developers who want a chat model small enough to run in a browser, on a laptop CPU or inside an app, and who accept modest answer quality.

Industry | Who uses it | How they use it | Example
Software | App and web developers | Add a tiny assistant that runs on the user's device | A help bubble inside a web page
Education | Teachers and students | Show how a chat model is loaded and prompted | A classroom demo of chat templates
Research | ML researchers | Use a small instruction model as a fine-tuning base | Adapting it to a narrow task
Testing | Engineers | Check chat pipelines with a model that loads fast | End-to-end tests of a prompt flow

Skip it if you need dependable facts or reasoning, since the card's score for grade-school maths is only 1.4.

Where to use it:

• Python scripts with the transformers library, on CPU or GPU
• The TRL command line, for chatting from a terminal
• JavaScript with Transformers.js, including in a browser
• Windows, macOS or Linux with Python or Node

Use case | Who | What they get back
Chat replies | Developers | A short answer to a message you send
Rewriting and summaries | Content tools | Reworded or condensed text, which the card lists as supported
Terminal chat | Hobbyists | An interactive session started with one command

How to set it up: You need these first.

• Python with pip, or Node with npm for the JavaScript route
• The transformers library, plus accelerate for several GPUs
• About 269 MB for the BF16 weights, or about 67 MB at 4-bit by estimate

1. Install the transformers library with pip.
2. Load the model by its Hugging Face name.
3. Build a messages list and apply the chat template.
4. Call generate and decode the output.

pip install transformers

pip install -U huggingface_hub
hf download HuggingFaceTB/SmolLM2-135M-Instruct

How to use it: Pass a list of role and content messages through the chat template, then generate. The card sets a low temperature of 0.2 with top-p 0.9 and 50 new tokens. For a quick terminal chat, install TRL and run its chat command.

from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "HuggingFaceTB/SmolLM2-135M-Instruct"

device = "cuda" # for GPU usage or "cpu" for CPU usage
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
# for multiple GPUs install accelerate and do `model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto")`
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

messages = [{"role": "user", "content": "What is gravity?"}]
input_text=tokenizer.apply_chat_template(messages, tokenize=False)
print(input_text)
inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
outputs = model.generate(inputs, max_new_tokens=50, temperature=0.2, top_p=0.9, do_sample=True)
print(tokenizer.decode(outputs[0]))

pip install trl
trl chat --model_name_or_path HuggingFaceTB/SmolLM2-135M-Instruct --device cpu

npm i @huggingface/transformers

import { pipeline } from "@huggingface/transformers";

// Create a text generation pipeline
const generator = await pipeline(
  "text-generation",
  "HuggingFaceTB/SmolLM2-135M-Instruct",
);

// Define the list of messages
const messages = [
  { role: "system", content: "You are a helpful assistant." },
  { role: "user", content: "What is the capital of France?" },
];

// Generate a response
const output = await generator(messages, { max_new_tokens: 128 });
console.log(output[0].generated_text.at(-1).content);

Tool | What it does
apply_chat_template | Turns your messages into the prompt format the model expects
model.generate | Produces the reply tokens
trl chat | Opens a chat session in the terminal
Transformers.js pipeline | Runs text generation from JavaScript

Example: In the JavaScript snippet, a system message says the assistant is helpful and the user asks for the capital of France. The reply names Paris, as the card's own comment shows. The terminal chat works the same way, one message at a time.

For newcomers and experts: If you are new, this is the version of SmolLM2 you can talk to. Start with the TRL chat command on CPU, ask one plain question and see how short the reply is. Do not expect long or deep answers from 135M parameters.

If you know the field, the card compares it with SmolLM-135M-Instruct on lighteval zero-shot tests. IFEval rises from 17.2 to 29.9 and MT-Bench from 16.8 to 19.8. The recipe is SFT then DPO on UltraFeedback. The function-calling support the card mentions applies to the 1.7B model, not this one.

SmolLM2-135M base model
Supervised fine-tuning on chat data
DPO on preference data
Chat template formats your messages
Model generates the reply

Good to know:

• Language: English only
• Size: 135M parameters in Safetensors
• Family: 360M and 1.7B siblings exist
• Community: other people have made 155 quantized and 411 fine-tuned versions

Title: Instruction benchmarks, SmolLM2 against the older SmolLM
IFEval, SmolLM2-135M-Instruct: 29.9 points
IFEval, SmolLM-135M-Instruct: 17.2 points
MT-Bench, SmolLM2-135M-Instruct: 19.8 points
MT-Bench, SmolLM-135M-Instruct: 16.8 points

• License: Apache 2.0
• Cost to run: none named, the source lists no API key, account or paid service
• Data it reaches: the messages you give your own script or browser page, and the source names no web or account access
• Not for: reliable facts, maths or function calling, since the card ties function calling to the 1.7B size

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 135M
• Weights: about 269 MB in BF16; about 67 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 841,106
• Likes: 474
• License: apache-2.0
• Languages: en
• Based on: HuggingFaceTB/SmolLM2-135M
• Quantized versions: 155
• Fine-tunes: 412
• Tags: transformers, tensorboard, onnx, safetensors, llama, text-generation, transformers.js, conversational

Links

• Model: https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct
• Files: https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download HuggingFaceTB/SmolLM2-135M-Instruct

Then follow the usage in its model card. Nothing in this kit runs by itself.