Nanbeige_Nanbeige4.3-33B-GGUF

Nanbeige_Nanbeige4.3-33B-GGUF

GGUF quantizations of Nanbeige4.3-33B, made by bartowski with an importance matrix, from about 21 GB to 66 GB. For running a 33B English and Chinese text model locally with llama.cpp.

Made by bartowski on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF quantizations of Nanbeige4.3-33B, a 33B text model by Nanbeige. Bartowski made the files with llama.cpp and an importance matrix. Different quant levels trade file size against quality. The page lists English and Chinese as languages.

Who it suits: People who run large language models on their own hardware with llama.cpp or a tool built on it. The model takes text only and does not use speculative decoding, according to the card.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a 33B model locally instead of a cloud service | Load a mid-size quant and test it on prompts
Local AI | Enthusiasts | Pick the quant that fits their memory | Take the file the card suggests when unsure
Research | Model evaluators | Compare quant sizes against quality | Run the same prompts on a large and a small quant
Bilingual work | Teams in English and Chinese | Use one model for both languages | Ask the same question in each language

You can skip it if you have limited memory. Even the file the card suggests is about 21 GB.

Where to use it:

• llama.cpp and tools built on it, on Windows, macOS and Linux
• Python with the Hugging Face tool, for downloading files
• Tools that read the ChatML-style prompt format shown below

Use case | Who | What they get back
Local chat | Enthusiasts | Text replies from a 33B model on your machine
Tool calling tests | Developers | Replies that follow the tool-call format the card shows
Quant comparison | Evaluators | A size and quality trade-off across many files

How to set it up: Pick one file rather than the whole repository, because the full set is far larger than any one file.

• Python and the Hugging Face tool for downloading
• A llama.cpp based program to load the file
• Memory for your chosen quant, from about 21 GB upward

1. Install the Hugging Face hub tool.
2. Choose a quant from the file list.
3. Download that file. The command below fetches the whole repository, so narrow it to one file.
4. Load the file in llama.cpp or a similar tool.

pip install -U huggingface_hub
hf download bartowski/Nanbeige_Nanbeige4.3-33B-GGUF

How to use it: The card gives a prompt format with a system block, a user block and an assistant turn that opens with a think tag. It also shows a variant that adds tool definitions for function calling.

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
<think>

Quant | Size and role
bf16 | 66.24 GB, the full-precision weights, split into parts
Q8_0 | 35.20 GB, the largest quant, rarely needed
Q6_K_L | 30.40 GB, a larger Q6_K, near-perfect quality
Q6_K | 29.02 GB, near-perfect quality and recommended
Q5_K_M | 25.07 GB, high quality and recommended
Q5_K_S | 23.66 GB, high quality and recommended
Q4_K_L | 22.34 GB, keeps more sensitive weights at higher precision
Q4_1 | 21.20 GB, a legacy format
IQ4_NL | 20.87 GB, like IQ4_XS but a little larger
Q4_K_M | 20.79 GB, the default pick for most uses

Example: You download the Q4_K_M file, load it with the prompt format above and ask for a short summary of a paragraph you paste in. The model thinks first, then replies with the summary.

For newcomers and experts: A quant is a smaller copy of a model with its numbers stored in fewer bits. If you are unsure, start with Q4_K_M, which the card calls a good mix of size and performance. Move up to Q6_K when you have memory to spare.

The quants use an importance matrix, and the source checkpoint is listed at 33B parameters. The Q4_K_L file keeps more of the sensitive weights at higher precision, and the card notes Q4_1 as a legacy format with better tokens per watt on Apple silicon. Bartowski made the files from the original model by Nanbeige.

Pick a quant that fits your memory
Download that one file
Load it in llama.cpp or a similar tool
Send a prompt in the format shown

Title: File size in GB
bf16: 66.24 GB
Q8_0: 35.20 GB
Q6_K: 29.02 GB
Q5_K_M: 25.07 GB
Q4_K_M: 20.79 GB

Good to know:

• The file list shown is cut off, so more quants exist than the ones above.
• The 4-bit weights are estimated at about 16.6 GB by the page's own record.
• Input is text only.
• License: apache-2.0.
• Cost to run: none named, you use your own hardware.
• Data it reaches: files downloaded from Hugging Face, and the text you give it.
• Not for: image input, since the model takes text only.

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 33.1B
• Weights: about 16.6 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 151,191
• Likes: 33
• License: apache-2.0
• Languages: en, zh
• Based on: Nanbeige/Nanbeige4.3-33B
• Fine-tunes: 1
• Tags: gguf, nanbeige, text-generation, imatrix, conversational

Links

• Model: https://huggingface.co/bartowski/Nanbeige_Nanbeige4.3-33B-GGUF
• Files: https://huggingface.co/bartowski/Nanbeige_Nanbeige4.3-33B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download bartowski/Nanbeige_Nanbeige4.3-33B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.