Axion-1.0-35B-GGUF

Axion-1.0-35B-GGUF

Provides GGUF files of Axion-1.0-35B, a 35B mixture-of-experts model with about 3B active parameters aimed at scientific reasoning, information seeking and tool use. For local use with llama.cpp.

Made by prithivMLmods on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF files of Axion-1.0-35B, a rebuild of Axion 1.0 under the Apache-2.0 license, made by following the original paper's published recipe. It is a 35B mixture-of-experts model with about 3B active parameters.

The card says it targets scientific reasoning, information seeking and problem solving with tools. The files come in several quantization sizes, from full BF16 down to Q3 types, plus one multimodal projection file.

Who it suits: It suits people who want to run an agent-style reasoning model on their own computer with llama.cpp and who need to match a file size to their memory.

Industry | Who uses it | How they use it | Example
Research | Scientists and students | Work through reasoning problems locally | Solve a rate and time word problem
Software | Developers | Test an agent style model with tool use | Write a linked list routine in C
Local AI | Enthusiasts | Run a 35B model with a size that fits their GPU | Pick the Q4_K_M file
Hardware testing | GPU owners | Try the MXFP4_MOE file on Blackwell cards | Measure generation speed

If you want a small model for a phone or a low-memory laptop, you can skip it, since even the smaller files are above 16 GB.

Where to use it: The files run in llama.cpp on Windows, macOS or Linux. The card tested the MXFP4_MOE file on an NVIDIA Blackwell GPU with 96 GB of memory.

• llama-cli from llama.cpp
• NVIDIA GPUs, with MXFP4_MOE tuned for Blackwell cards
• Standard desktop hardware, with the Q4 to Q6 files

Use case | Who | What they get back
Reasoning questions | Students | Worked text answers
Code snippets | Developers | Small programs written as text
Blackwell testing | GPU owners | Fast generation with the MXFP4_MOE file

How to set it up: You need Python with pip to fetch the files, and a built llama.cpp. Pick one quant file that fits your memory, since the full repository is far larger than a single file.

• Python with pip
• llama.cpp built on your machine
• Free disk space for the quant you choose

1. Install the Hugging Face hub package.
2. Download the repository files, or just the quant you want.
3. Run llama-cli with that file.

pip install -U huggingface_hub
hf download prithivMLmods/Axion-1.0-35B-GGUF

How to use it: You run llama-cli with the model path, the number of GPU layers, the context size and your prompt. The card gives this command for the MXFP4_MOE file.

./build/bin/llama-cli \
  -m /path/to/models/Axion-1.0-35B.MXFP4_MOE.gguf \
  -ngl 99 \
  -c 16384 \
  --cache-type-k q8_0 \
  --seed 42 \
  -n 8192 \
  -st \
  -p "Your prompt here"

File | Size | When to pick it
BF16 | 69.4 GB | You want the full weights and the best quality
MXFP4_MOE | 19.8 GB | You own an NVIDIA Blackwell card
Q3_K_L | 18.1 GB | Memory is tight and lower quality is fine
Q3_K_M | 16.8 GB | The smallest text file, with lower quality
Q4_K_M | 21.2 GB | The balanced choice for most uses
Q5_K_M | 24.7 GB | You want higher quality on standard hardware
Q6_K | 28.5 GB | You want near-full quality
mmproj-bf16 | 903 MB | You need the vision projection file

Example: You run the command with a prompt asking how long two pipes take to fill a tank together. The model replies with a written working and an answer, and the card used prompts of this kind in its tests.

For newcomers and experts: If you are new, pick the Q4_K_M file and run one short question first. The card calls it the balanced choice for most uses.

If you know the field, the model is a sparse mixture-of-experts with about 3B active parameters. The card notes it has no built-in speculative decoding. The MXFP4_MOE file uses the FP4 tensor cores on Blackwell hardware.

Pick a quant file that fits memory
Run llama-cli with the file
Send your prompt
Read the text answer

Good to know: The card's test used one RTX PRO 6000 Blackwell card, 16,384 tokens of context and full GPU offload. Your speed will differ with other hardware, prompts and builds.

Measure | Result on that test
Prompt processing | 238.50 tokens per second
Generation | 211.70 tokens per second
Peak GPU memory | 19.07 GiB

• License: apache-2.0
• Cost to run: none named
• Data it reaches: files you download, and the prompts you type
• Not for: a quick start on small hardware, since the smallest text file is 16.8 GB

On Hugging Face, 15 November 2026

• Task: text generation
• Library: transformers
• Parameters: 34.7B
• Weights: about 17.3 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 63,538
• Likes: 4
• License: apache-2.0
• Languages: en
• Based on: Axion1-0/Axion-1.0-35B
• Tags: transformers, gguf, agent, tool-use, scientific-reasoning, qwen3_5_moe, mxfp4, llama-cpp, nvfp4, nvidia, llama-cli, nvidia-blackwell

Links

• Model: https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF
• Files: https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download prithivMLmods/Axion-1.0-35B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.