--- title: "Ornith-1.5-35B-A3B-GSQ-RCO-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/1ce4ee0b-4bd4-410d-a55a-e0e0f7c496c4" updated: "2026-11-12" --- # Ornith-1.5-35B-A3B-GSQ-RCO-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Shrinks the Ornith-1.5-35B-A3B text model to 3-bit and 3.5-bit GGUF files, about 13 and 15 GB instead of 69 GB, for people who run local models with llama.cpp. Ornith-1.5-35B-A3B-GSQ-RCO-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by pfeifferj ## About ### What it is These are compressed GGUF copies of Ornith-1.5-35B-A3B, a mixture-of-experts text model of about 35B parameters from the Qwen3.5 family. They come in 3-bit and 3.5-bit sizes, roughly 13 GB and 15 GB against 69 GB for the BF16 original. An independent community member made them with the published GSQ and RCO methods, and they are not an official release of the method authors. ### Who it suits It suits people who run local models with llama.cpp and want a large model squeezed into a smaller download. Who uses this model | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a large coding and chat model locally | Serve it with llama-server for an app | | Research | Quantization researchers | Study low-bit compression results | Compare the 3-bit and 3.5-bit builds against BF16 | | Self-hosting | Hobbyists | Fit a big model on modest hardware | Pick the 3-bit file when disk space is tight | If you need the exact quality of the BF16 original, you can skip it, because the compressed builds score lower on the card's tests. ### Where to use it - llama.cpp, through llama-server - Other programs that open GGUF files - Windows, macOS or Linux with Python, for downloading the files Jobs it does | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Anyone | Text replies from a model running on your machine | | Local API | Developers | A server that other programs can call | | Compression study | Researchers | Perplexity and MMLU-Pro figures for each size | ### How to set it up You need Python for the download and a llama.cpp build. The author tested with one specific upstream llama.cpp commit, so a recent build is the safe choice. - Python with pip - A working llama.cpp build that includes llama-server - About 15.2 GB of disk space for the 3.5-bit file, or 13.0 GB for the 3-bit one 1. Install huggingface\_hub. 2. Download the GGUF file you want. 3. Start llama-server with the model file and the settings below. ```bash pip install -U huggingface_hub ``` ```bash hf download pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF Ornith-1.5-35B-A3B-GSQ-RCO-3.5bit.gguf --local-dir . llama-server -m Ornith-1.5-35B-A3B-GSQ-RCO-3.5bit.gguf \ -ngl 999 -c 8192 -fa on -ctk f32 -ctv f32 --jinja \ --temp 0.6 --top-p 0.95 --top-k 20 \ --chat-template-kwargs '{"preserve_thinking":true}' ``` ### How to use it Once llama-server runs, chat with it or call it from your apps. The sampling values in the command are the ones Ornith recommends. Thinking is on by default, and you can turn it off by setting enable\_thinking to false in the chat template arguments. Main command pieces | Tool | What it does | | --- | --- | | hf download | Fetches one GGUF file into the current folder | | llama-server | Serves the model on your machine | | --temp, --top-p, --top-k | Set the sampling values the card recommends | | --chat-template-kwargs | Keeps the model's reasoning visible across turns | ### Example You download the 3.5-bit file and start the server with the command above. Then you send a reasoning question. The model thinks first and then replies in text, because thinking is on by default. ### For newcomers and experts Quantization stores a model's numbers in fewer bits so the files shrink. This model is already big, so the small files make it easier to run. A first small test is the 3-bit file with the command above. GSQ learns how each weight is rounded and scaled through a Gumbel-Softmax relaxation. RCO picks a quantization type for each tensor under an exact total size budget. The card compares against matched initializers and native imatrix builds. Against BF16, perplexity is 3.7% higher for the 3-bit build and 1.4% higher for the 3.5-bit one. Ornith-1.5-35B-A3B in BF16 → GSQ learns the rounding → RCO assigns types per tensor → 3-bit and 3.5-bit GGUF files ### Good to know On MMLU-Pro the card reports 47.60% for BF16, 43.70% for the 3-bit build and 43.20% for the 3.5-bit build. On MMLU-Pro, the matched initializer builds beat both GSQ builds. The files hold the text model only, with the tokenizer and chat template. File size by build | Label | Value | | --- | --- | | BF16 reference | 69.4 GB | | GSQ-RCO 3.5-bit | 15.2 GB | | GSQ-RCO 3-bit | 13.0 GB | - License: MIT. - Cost to run: none named. - Data it reaches: the files come from Hugging Face, and the model then runs on your own computer. - Not for: anyone expecting BF16 quality, or images, since these files hold the text model only. ## README ````markdown # Ornith-1.5-35B-A3B-GSQ-RCO-GGUF Shrinks the Ornith-1.5-35B-A3B text model to 3-bit and 3.5-bit GGUF files, about 13 and 15 GB instead of 69 GB, for people who run local models with llama.cpp. Made by pfeifferj on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: These are compressed GGUF copies of Ornith-1.5-35B-A3B, a mixture-of-experts text model of about 35B parameters from the Qwen3.5 family. They come in 3-bit and 3.5-bit sizes, roughly 13 GB and 15 GB against 69 GB for the BF16 original. An independent community member made them with the published GSQ and RCO methods, and they are not an official release of the method authors. Who it suits: It suits people who run local models with llama.cpp and want a large model squeezed into a smaller download. ```table Title: Who uses this model Industry | Who uses it | How they use it | Example Software | Developers | Run a large coding and chat model locally | Serve it with llama-server for an app Research | Quantization researchers | Study low-bit compression results | Compare the 3-bit and 3.5-bit builds against BF16 Self-hosting | Hobbyists | Fit a big model on modest hardware | Pick the 3-bit file when disk space is tight ``` If you need the exact quality of the BF16 original, you can skip it, because the compressed builds score lower on the card's tests. Where to use it: - llama.cpp, through llama-server - Other programs that open GGUF files - Windows, macOS or Linux with Python, for downloading the files ```table Title: Jobs it does Use case | Who | What they get back Local chat | Anyone | Text replies from a model running on your machine Local API | Developers | A server that other programs can call Compression study | Researchers | Perplexity and MMLU-Pro figures for each size ``` How to set it up: You need Python for the download and a llama.cpp build. The author tested with one specific upstream llama.cpp commit, so a recent build is the safe choice. - Python with pip - A working llama.cpp build that includes llama-server - About 15.2 GB of disk space for the 3.5-bit file, or 13.0 GB for the 3-bit one 1. Install huggingface_hub. 2. Download the GGUF file you want. 3. Start llama-server with the model file and the settings below. ```bash pip install -U huggingface_hub ``` ```bash hf download pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF Ornith-1.5-35B-A3B-GSQ-RCO-3.5bit.gguf --local-dir . llama-server -m Ornith-1.5-35B-A3B-GSQ-RCO-3.5bit.gguf \ -ngl 999 -c 8192 -fa on -ctk f32 -ctv f32 --jinja \ --temp 0.6 --top-p 0.95 --top-k 20 \ --chat-template-kwargs '{"preserve_thinking":true}' ``` How to use it: Once llama-server runs, chat with it or call it from your apps. The sampling values in the command are the ones Ornith recommends. Thinking is on by default, and you can turn it off by setting enable_thinking to false in the chat template arguments. ```table Title: Main command pieces Tool | What it does hf download | Fetches one GGUF file into the current folder llama-server | Serves the model on your machine --temp, --top-p, --top-k | Set the sampling values the card recommends --chat-template-kwargs | Keeps the model's reasoning visible across turns ``` Example: You download the 3.5-bit file and start the server with the command above. Then you send a reasoning question. The model thinks first and then replies in text, because thinking is on by default. For newcomers and experts: Quantization stores a model's numbers in fewer bits so the files shrink. This model is already big, so the small files make it easier to run. A first small test is the 3-bit file with the command above. GSQ learns how each weight is rounded and scaled through a Gumbel-Softmax relaxation. RCO picks a quantization type for each tensor under an exact total size budget. The card compares against matched initializers and native imatrix builds. Against BF16, perplexity is 3.7% higher for the 3-bit build and 1.4% higher for the 3.5-bit one. ```flow Ornith-1.5-35B-A3B in BF16 GSQ learns the rounding RCO assigns types per tensor 3-bit and 3.5-bit GGUF files ``` Good to know: On MMLU-Pro the card reports 47.60% for BF16, 43.70% for the 3-bit build and 43.20% for the 3.5-bit build. On MMLU-Pro, the matched initializer builds beat both GSQ builds. The files hold the text model only, with the tokenizer and chat template. ```chart Title: File size by build BF16 reference: 69.4 GB GSQ-RCO 3.5-bit: 15.2 GB GSQ-RCO 3-bit: 13.0 GB ``` - License: MIT. - Cost to run: none named. - Data it reaches: the files come from Hugging Face, and the model then runs on your own computer. - Not for: anyone expecting BF16 quality, or images, since these files hold the text model only. ## On Hugging Face, 13 November 2026 - Task: text generation - Library: gguf - Parameters: 34.7B - Weights: about 17.3 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 11,634 - Likes: 30 - License: mit - Based on: ornith-ai/Ornith-1.5-35B-A3B - Tags: gguf, quantization, mixed-precision, text-generation, conversational ## Links - Model: https://huggingface.co/pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF - Files: https://huggingface.co/pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF on Hugging Face](https://huggingface.co/pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Ornith-1.5-35B-A3B-GSQ-RCO-GGUF (this page) | 11,634 | 30 | 34.7B | mit | 2026-09-24 | | [DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF](https://mindket.com/p/0bd5db10-f872-40c7-9371-3225afab2a6d) | 3,635,772 | 655 | 197B | mit | 2026-10-28 | | [DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF](https://mindket.com/p/1350113a-de6f-4d62-96d6-599c30da33a2) | 342,280 | 225 | 197B | mit | 2026-11-02 | | [Qwen3.8-27B-SLQ-GGUF](https://mindket.com/p/c549f05b-1d25-49d4-868a-58981cf20769) | 13,478 | 60 | 27.3B | apache-2.0 | 2026-10-27 | | [Qwen3.8-Flash-Next-GSQ-RCO-GGUF](https://mindket.com/p/ee9ed4a6-39c8-4eee-8958-cce1cd508fa6) | 6,963,596 | 1,229 | 177B | apache-2.0 | 2026-09-29 | | [Qwen3.8-27B-GSQ-RCO-GGUF](https://mindket.com/p/d79e6c18-7ed6-4306-a01a-66a747f13e40) | 1,358,588 | 2,739 | 26.9B | apache-2.0 | 2026-09-02 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-09-24 - First published: 2026-09-24 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF on Hugging Face](https://huggingface.co/pfeifferj/Ornith-1.5-35B-A3B-GSQ-RCO-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/1ce4ee0b-4bd4-410d-a55a-e0e0f7c496c4) - [Its page on Mindket](https://mindket.com/p/1ce4ee0b-4bd4-410d-a55a-e0e0f7c496c4)