gugujiji_novel-humanize-4b-GGUF

gugujiji_novel-humanize-4b-GGUF

Quantized GGUF copies of the novel-humanize-4b Chinese text model, offered in sizes from full precision down to small files. For people who run a 4B-parameter writing model on their own computer.

Made by bartowski on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a set of GGUF files that bartowski made from the novel-humanize-4b text generation model by gugujiji. Each file compresses the original weights to a different degree, so you can balance file size against quality for the memory you have.

The files were made with llama.cpp and an importance matrix. The model reads text and writes text, and Hugging Face lists Chinese as its language. The card does not describe the training or the exact writing style, so treat the name as the only hint of its purpose.

Who it suits: People who read and write Chinese and want a small model on their own computer, and who are comfortable picking a file size and loading it in a GGUF-capable program.

Industry | Who uses it | How they use it | Example
Publishing | Chinese-language writers | Try a small local model for prose drafts | Writing a first pass of a story scene
Software | Developers | Run a 4B model inside an app they build | Testing Chinese text output offline
Research | Hobbyists | Compare quant sizes on their own hardware | Choosing the smallest file with acceptable output
Education | Students | Experiment with local language models | Running a model with no cloud account

Anyone who needs images or audio as input can skip it, since only text input is listed.

Where to use it:

• llama.cpp and apps built on it, since the files were made with that tool.
• Windows, macOS or Linux with Python, for fetching the files with the Hugging Face tool.
• Computers with modest memory, using the smaller files.

Use case | Who | What they get back
Local drafting | Writers | Chinese text generated on their own machine
Offline testing | Developers | A model they can call without a network
Size comparison | Tinkerers | A view of how file size changes quality

How to set it up: You need Python, the Hugging Face tool and a program that can load GGUF files.

• Python on Windows, macOS or Linux
• The huggingface_hub package
• A GGUF-capable runner such as llama.cpp
• Disk space for the file you pick, from about 2 GB at 4-bit up to 8.05GB for full precision

1. Install the Hugging Face tool.
2. Download the repository, or choose a single file.
3. Load that file in your runner.

pip install -U huggingface_hub
hf download bartowski/gugujiji_novel-humanize-4b-GGUF

How to use it: Load a file and send prompts in the chat format below, putting your own text where the placeholders are. The card also shows a longer variant for function definitions.

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

The card suggests the Q4_K_M file for most people. Here is how a few of the files compare.

Quant | Size | What the card says
bf16 | 8.05GB | Full precision weights
Q8_0 | 4.28GB | Top quality, rarely needed
Q6_K | 3.52GB | Near-perfect quality, recommended
Q5_K_M | 3.11GB | High quality, recommended
Q4_K_L | 2.80GB | Keeps sensitive weights at higher precision
Q4_K_M | 2.59GB | The usual pick for most people
Q4_1 | 2.63GB | Older format, better energy use on Apple chips

Example: Load the Q4_K_M file and send a short Chinese request in the user slot, such as asking for a few lines describing a rainy street. The model replies with text written in Chinese. How well it suits fiction is not stated in the card, so judge the output yourself.

For newcomers and experts: If you are new, a GGUF file is a single copy of a model that a local program can open. Download the Q4_K_M file, send one short Chinese request, and see whether the reply and the speed suit your computer.

For experts, these are llama.cpp quantizations built with an importance matrix, and the card lists speculative decoding as unsupported. The L variants keep more of the sensitive weights at higher precision, which adds some size. The parameter count is given as 4B for the source checkpoint.

gugujiji publishes novel-humanize-4b
bartowski quantizes it with llama.cpp
You download the file that fits your memory
A local runner loads the GGUF file
You send prompts in the chat format

Good to know: The points below cover limits and costs.

• Purpose: the card only describes the quantization, so what the model is best at is not documented there.
• License: Apache 2.0.
• Cost to run: no API key or paid service is named. You need enough RAM and disk for your chosen file.
• Data it reaches: it reads the text you send. Downloading the files needs a connection to Hugging Face.
• Not for: image or other non-text input, and speculative decoding.

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 4B
• Weights: about 2 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 7,774
• Likes: 5
• License: apache-2.0
• Languages: zh
• Based on: gugujiji/novel-humanize-4b
• Tags: gguf, chinese, novel, rewriting, text-generation, conversational

Links

• Model: https://huggingface.co/bartowski/gugujiji_novel-humanize-4b-GGUF
• Files: https://huggingface.co/bartowski/gugujiji_novel-humanize-4b-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download bartowski/gugujiji_novel-humanize-4b-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.