KLM 1.5

KLM 1.5

A 99M-parameter base language model trained from scratch on 100B tokens, with a hybrid attention design. Made for researchers and tinkerers who want a tiny open model to run, study or fine-tune.

Made by MinimaLabs on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: KLM 1.5 is a base language model with 99M parameters from MinimaLabs. Its architecture is new, and it learned from zero on 100B tokens, while KLM 1.1 kept training an older model. Because it is a base model, it continues text rather than following chat instructions. Its weights take about 396 MB in F32.

Who it suits: It suits researchers, students and hobbyists who want a tiny English language model for experiments, small-scale tests or fine-tuning on modest hardware.

Industry | Who uses it | How they use it | Example
Research | Language model researchers | Study a small model with mixed attention layers and an n-gram memory | Comparing it with other models under 150M parameters
Education | Students | Learn how small language models are built and trained | Reading the training stages next to the benchmark scores
Software | Developers | Use a tiny base model as a starting point for fine-tuning | Adapting it to a narrow text task
Hobby projects | Tinkerers | Run a tiny model on a laptop | Testing text continuation without a GPU

Skip it if you need a chat assistant or strong reasoning, since this is a small base model and its scores sit modestly above chance.

Where to use it:

• Python scripts using the transformers library
• Windows, macOS and Linux
• Small machines, since the weights are about 396 MB in F32

Use case | Who | What they get back
Text continuation | Developers | Text that continues your prompt
Research comparison | Researchers | A small reference model with published scores
Fine-tuning base | Developers | A compact starting point to adapt

How to set it up:

• Python, with the transformers library
• About 396 MB of disk space for the F32 weights, or about 49 MB at 4-bit as an estimate
• No GPU is required for a model this small

1. Install the Hugging Face tools.
2. Download the model files to your computer.
3. Load the model with transformers and give it a text prompt.

pip install -U huggingface_hub
hf download MinimaLabs/KLM-1.5

How to use it: You give the model the start of a text and it writes what comes next. The part of the card available here shows no run command, so check the full card for exact code. The card publishes zero-shot benchmark scores for comparison.

Tool | What it does
Gated DeltaNet-2 layers | Carry a fixed-size memory through the text
Full attention layers | Look at earlier tokens directly, one layer in four
n-gram memory | Looks up learned vectors for common word pairs and triples
Tokenizer | Uses 4,100 tokens and splits numbers into single digits

Example: You give the model the opening of a sentence. It writes a continuation, word by word. At this size the output can be rough, so use it to learn and test ideas rather than for finished text.

For newcomers and experts: A base model only continues text, and a 99M one is tiny next to chat models. A good first try is to download it and feed it a short opening line.

If you know the field, KLM 1.5 mixes Gated DeltaNet-2 and full attention at a 3:1 ratio, adds block attention residuals, and uses an exact n-gram memory of 5.4M parameters. Training had three stages: a superposition warm-up, stable training on 80B tokens, and a decay stage on 20B tokens with more math and Q&A.

Prompt text becomes tokens
Tokens get n-gram memory vectors
Hybrid layers process the sequence
Model predicts the next token
Repeat to continue the text

Title: ARC-Easy accuracy, zero-shot
KLM 1.5: 59.68%
KLM 1.1: 44.36%
GPT-X3-150M: 59.64%
SmolLM2-135M: 58.63%
LFM-2.5-230M-Base: 59.72%
void.1: 47.31%

Good to know:

• Size: 99M parameters, with a 4,100-token vocabulary that costs 2.1M parameters.
• Scores: the card reports an Intelligence Index of 30.26 on one leaderboard, first among models under 100M parameters.
• Training: three stages, using sequences of 2,048 tokens, 128 of them in each step.
• License: Apache 2.0.
• Cost to run: none named.
• Data it reaches: it downloads model files from Hugging Face and works on the text you give it.
• Not for: following chat instructions, since it is a base model.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 99M
• Weights: about 396 MB in F32; about 49 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 1,603
• Likes: 31
• License: apache-2.0
• Languages: en
• Tags: transformers, safetensors, text-generation, small-language-model, base, pretrained, hybrid, gated-deltanet

Links

• Model: https://huggingface.co/MinimaLabs/KLM-1.5
• Files: https://huggingface.co/MinimaLabs/KLM-1.5/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download MinimaLabs/KLM-1.5

Then follow the usage in its model card. Nothing in this kit runs by itself.