GPT-X3-150M ONNX

GPT-X3-150M ONNX

Lets you run GPT-X3-150M on ONNX Runtime at five precision levels and compare their output and speed, with streaming tokens on a CPU. For developers weighing model size against quality.

Made by ahmadw on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: GPT-X3-150M ONNX is a hands-on demo of ONNX versions of GPT-X3-150M, a small model from Axiomic Labs. You can switch between five precision builds and compare what each one writes and how fast it runs. Tokens stream live from ONNX Runtime on a CPU, and you can open the page in a browser with nothing to install.

Who it suits: It suits developers and researchers who want to see what lower precision does to a small language model's speed and output.

Industry | Who uses it | How they use it | Example
Software | Developers | Choose a precision for CPU deployment | Compare FP32 with INT4 on the same prompt
AI research | Researchers | Look at the cost of quantization | Read the benchmark list for each build
Edge computing | Device makers | Check what runs on modest hardware | Watch the decode speed on a CPU

If you want a large chat model, you can skip it, since this showcase is about a small one.

Where to use it:

• A web browser, on the Space's page on Hugging Face
• Windows, macOS or Linux with Docker, from a local build of the Space's files

Use case | Who | What they get back
Compare builds | Developers | Output and speed for each precision level
Measure speed | Developers | Time to first token and decode speed for every run
Read the benchmarks | Researchers | Per-build figures for file size, perplexity and how closely each build matches FP32

How to set it up:

• A web browser, which is all the online page needs
• Docker and Python with pip, for a local copy

1. Open the Space's page on Hugging Face to use it online. Nothing needs installing.
2. For a local copy, install the Hugging Face hub package and download the Space's files.
3. Move into the downloaded folder.
4. Build the Docker image, then run it with port 7860 published.

pip install -U huggingface_hub
hf download ahmadw/GPT-X3-150M-ONNX-demo --repo-type space --local-dir GPT-X3-150M-ONNX-demo
cd GPT-X3-150M-ONNX-demo
docker build -t gpt-x3-150m-onnx-demo .
docker run -it -p 7860:7860 gpt-x3-150m-onnx-demo

The app downloads the models from the model repository when it starts.

How to use it: Open the page, or port 7860 on your machine once the container runs, pick a precision build and run a prompt. Then switch builds and compare.

Tool | What it does
Precision switch | Picks one of five builds, from full FP32 down to a mixed INT4 one
Live stream | Shows tokens as ONNX Runtime produces them on the CPU
Run statistics | Shows time to first token and decode speed for each run
Benchmark list | Lists measured quality and size figures for every build

Example: Run one prompt on the FP32 build, then run the same prompt on the INT4 build. You compare the two outputs, and the page shows how fast each one started and decoded.

For newcomers and experts: Open the Space page and run one prompt on the FP32 build first. Then switch to INT4 to see what a smaller build changes.

A compact FastAPI backend in app.py and engine.py sends the page's hand-built static front end to your browser. Every build is scored against the FP32 one on file size and on quality measures such as perplexity and KL divergence, so you can weigh quality loss against speed gain.

The app downloads the ONNX builds at startup
You pick a precision and send a prompt
ONNX Runtime generates tokens on the CPU
The page streams them with speed figures

Good to know: The online copy runs on basic CPU hardware, so speeds will differ on other machines.

• License: Apache 2.0
• Cost to run: none named, apart from Docker for a local build
• Data it reaches: the prompts you type, and the ahmadw/GPT-X3-150M-ONNX and AxiomicLabs/GPT-X3-150M models

On Hugging Face, 13 November 2026

• Likes: 5
• Built with: docker
• License: apache-2.0

Links

• Space: https://huggingface.co/spaces/ahmadw/GPT-X3-150M-ONNX-demo
• Files: https://huggingface.co/spaces/ahmadw/GPT-X3-150M-ONNX-demo/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.