GPT-X3-150M

GPT-X3-150M

A 149M-parameter base text model from Axiomic Labs that continues the text you give it. The card reports an Intelligence Index of 33.05, ahead of SmolLM2-135M at 27.13. About 598 MB in F32.

Made by AxiomicLabs on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: GPT-X3-150M is the third base model from Axiomic Labs, trained from scratch with the team's own TrainWork tools. It uses the TX5 design, which mixes two kinds of attention layers. It has 149M parameters and works in English. As a base model it continues text rather than following chat instructions. The card says it tops the authors' own small model leaderboard.

Who it suits: It suits researchers, students and developers who want a tiny language model to study, fine tune or use in quick tests.

Industry | Who uses it | How they use it | Example
Research | Small model researchers | Compare it with other models under 150M parameters | Check its Intelligence Index against SmolLM2-135M
Education | Students learning about language models | Study a hybrid attention design at small scale | Read the layer layout of 24 layers in a 3 to 1 mix
Prototyping | Developers testing pipelines | Use a tiny base model for fast trials | Complete a short prompt with a 598 MB model

You can skip it if you want a chat assistant, since it is a base model and not tuned to follow instructions.

Where to use it:

• Python scripts that use the transformers library
• Fine tuning and evaluation setups for small models
• Windows, macOS and Linux with Python

Use case | Who | What they get back
Text completion | Developers | A continuation of the text they give it
Benchmark comparison | Researchers | Scores to set beside other small models
Architecture study | Students | A working example of a 3 to 1 hybrid stack

How to set it up: You need these first:

• Python on Windows, macOS or Linux
• The transformers library
• About 598 MB of disk space for the F32 weights

1. Install or update the Hugging Face download client.
2. Download the model files to your computer.
3. Load the model in your own transformers code.
4. Read the full model card for any extra loading options.

pip install -U huggingface_hub
hf download AxiomicLabs/GPT-X3-150M

How to use it: You load the model in Python and give it the start of a text to continue. The card text available here shows no code, and the architecture is custom, so check the full card for how to load it.

Tool | What it does
Intelligence Index | The card's combined score across four benchmark groups
Open SLM Leaderboard | The authors' own board, which the card's results follow
HellaSwag and PIQA | Common sense benchmarks that feed the index
ARC | Science question benchmarks, counted as one group
ArithMark-3 | An arithmetic benchmark with a lower weight in the index

Example: You open the card's table and compare the model's Intelligence Index of 33.05 with SmolLM2-135M at 27.13. The card says it got there on 75B training tokens, while SmolLM2-135M used 2T.

For newcomers and experts: A base model predicts what text comes next, so it works best when you give it the start of a sentence. A good first try is a short prompt and a few generated words.

For readers who know the field, the stack has 24 layers, 18 of them GDN2 and 6 of them gated GQA, in a 3 to 1 pattern. The vocabulary has 65,536 tokens from the LFM2.5 tokenizer. Training used a fixed nine source mix, a six token objective for the first 30 percent, and Muon with AdamW.

Download the model files
Load them with transformers
Give it the start of a text
It predicts the next tokens
Read the continuation

Good to know:

• Claims: the top ranking and the first under 150M claim come from the authors' own board and card.
• Index: the score corrects four benchmark groups for lucky guesses and counts the arithmetic one at 0.65, against 1.00 for the other three.
• Training: about 75B compute tokens, with about 187.5B raw tokens in total.
• Context: the config lists a block size of 2048 tokens.
• Cut: the later config lines are cut off in the text available here.
• Counts: Hugging Face lists 149M parameters, 1 quantized and 1 fine tuned version from others.

Title: Intelligence Index from the card's table, higher is better
GPT-X3-150M: 33.05
LFM2.5-350M-Base: 30.50
Boris-2-Preview-1: 27.91
cagliostro-v3.5: 27.49
SmolLM2-135M: 27.13
GPT-2 Medium: 19.68
GPT-2 (124M): 13.58

• License: Apache 2.0.
• Cost to run: none named. The weights take about 598 MB in F32.
• Data it reaches: the text you give it on your own machine. The text available here names no network use.
• Not for: chat or instruction following, since the card calls it a base model.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 149M
• Weights: about 598 MB in F32; about 75 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 3,000
• Likes: 104
• License: apache-2.0
• Languages: en
• Quantized versions: 1
• Fine-tunes: 1
• Tags: transformers, safetensors, gptx3, text-generation, gptx, gdn2, hybrid, math, language-model, gpt-x3, gpt-x3-150m, open_slm_leaderboard

Links

• Model: https://huggingface.co/AxiomicLabs/GPT-X3-150M
• Files: https://huggingface.co/AxiomicLabs/GPT-X3-150M/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download AxiomicLabs/GPT-X3-150M

Then follow the usage in its model card. Nothing in this kit runs by itself.