Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF

Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF

Runs the 35B Axion-1.0 model locally in llama.cpp from one compressed GGUF file with mixed precision. A quantized release by IsValorum for people who want a large mixture-of-experts model on their own hardware.

Made by IsValorum on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a compressed GGUF version of Axion-1.0-35B, released by IsValorum as part of the APEX quantization series. Each part of the model is stored at the precision it needs, so the whole file is much smaller than the original. It targets llama.cpp and ships the text model only.

Who it suits: People who run models with llama.cpp and want a large mixture-of-experts model on their own computer, plus anyone studying how mixed-precision quantization is laid out. The files carry the Apache 2.0 license.

Industry | Who uses it | How they use it | Example
Software | Local model users | Run a 35B-class model offline through llama.cpp | Chatting with the model in a terminal
Research | Quantization researchers | Study how each tensor group is stored | Reading the per-tensor precision map
Productivity | Power users | Keep a large model on a personal machine | Drafting and Q and A with a long context

Anyone who needs image input out of the box can skip it, since this repository holds the text file only.

Where to use it:

• llama.cpp, through its command-line tool and its server
• Windows, macOS or Linux with Python, for downloading the files
• A machine with enough memory for a file of 14,751,301,312 bytes

Use case | Who | What they get back
Terminal chat | Developers | Text replies from a prompt typed on the command line
Local server | App builders | A running server that uses the chat template in the file
Quantization study | Researchers | A per-tensor map of the precision used

How to set it up:

• Python and the huggingface_hub package, to download the files
• A current llama.cpp build that can load GGUF files
• Room for the file, about 14,751,301,312 bytes, while Hugging Face estimates 17.3 GB at 4-bit

1. Install Python and a llama.cpp build if you do not have them yet.
2. Install the huggingface_hub package with the first command below.
3. Download the model files with the second command.
4. Run the model with one of the llama.cpp commands in the next section.

pip install -U huggingface_hub
hf download IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF

How to use it: The card shows two ways to start the model. The first sends a one-word prompt from the terminal, and the second starts a server that applies the chat template.

llama-cli -m Axion-1.0-35B.APEX-I-MiniPlus-V2.1.gguf -p "Hello" -ngl 99 -c 32768
llama-server -m Axion-1.0-35B.APEX-I-MiniPlus-V2.1.gguf --jinja -ngl 99 -c 32768

Tool | What it does
llama-cli | Runs the model once from the terminal with a prompt
llama-server | Starts a server so other programs can talk to the model
-c 32768 | Sets the context size to 32768 tokens
--jinja | Applies the chat template that ships with the model

Example: You run the llama-cli line with the prompt Hello. The model loads, then prints its reply in the terminal.

For newcomers and experts: Quantization stores a model's numbers with fewer bits so it fits on ordinary hardware. A good first step is to download the file and run the llama-cli line as written.

The model has 40 layers, with 30 linear-attention or SSM layers and 10 full-attention layers. It routes over 256 experts plus 1 shared expert, with 8 active per token, and a context of 262,144 tokens. The output head sits at Q6_K and the token embeddings at Q4_K. Shared experts use Q5_K, and norms and routers stay in F32. Routed experts use Q3_K at the edges and IQ3_XXS in layers 10 to 29, guided by an importance matrix from mradermacher.

Start from the BF16 reference file
Apply an importance matrix
Store each tensor group at its own precision
Save one GGUF file
Run it in llama.cpp

Good to know:

• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the download fetches files from Hugging Face, and the source names nothing else
• Image input needs a separate mmproj file from the upstream GGUF repository
• Multi-token prediction is not in this file, so there is no draft model to speed up generation
• Hugging Face lists 34.7B parameters and a 4-bit estimate of about 17.3 GB

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: gguf
• Parameters: 34.7B
• Weights: about 17.3 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 686
• Likes: 4
• License: apache-2.0
• Based on: Axion1-0/Axion-1.0-35B
• Tags: gguf, llama.cpp, quantized, apex, apex-quant, apex-i, axion, imatrix, image-text-to-text, conversational

Links

• Model: https://huggingface.co/IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
• Files: https://huggingface.co/IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.