MNN-Pocket-0.3B-GGUF

MNN-Pocket-0.3B-GGUF

Runs a compact 0.3B Chinese and English language model from GGUF files of 192 to 726 MB, made for phones, laptops and other on-device use with llama.cpp, LM Studio or ollama.

Made by taobao-mnn on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: MNN-Pocket-0.3B-GGUF is a GGUF export of MNN-Pocket-0.3B, a roughly 300M-parameter language model from taobao-mnn. The family is built together with the MNN inference stack for phones, tablets, laptops and smart glasses. The repository has three files, F16 at 726 MB, Q8_0 at 387 MB and Q4_1 at 192 MB, for llama.cpp and runtimes such as LM Studio and ollama.

Who it suits: People who want a tiny chat model for on-device assistants, data extraction or light agent pipelines, in Chinese or English.

Industry | Who uses it | How they use it | Example
Mobile apps | App developers | Put a small assistant on the device | A chat helper inside a phone app
Data work | Analysts | Pull structured fields out of text | Extracting names and dates from notes
Agent building | Pipeline developers | Run a light model in a tool chain | A small step that routes requests
Hobby projects | Tinkerers | Try a model on a low-memory machine | Chat on an older laptop

If you need advanced math or long coding tasks, you can skip it, because the card does not recommend it for reasoning-heavy work.

Where to use it:

• llama.cpp, with its llama-cli and llama-server programs
• LM Studio and ollama
• Windows, macOS or Linux, after you fetch the files with Python

Use case | Who | What they get back
On-device chat | App builders | Short answers from a tiny model
Data extraction | Analysts | Fields pulled from text
Local server | Developers | An OpenAI-compatible endpoint on your own machine

How to set it up:

• Python and pip, to fetch the files
• llama.cpp, a recent build so the chat template applies by itself
• A few hundred MB of disk space, depending on the file you pick

1. Install the Hugging Face hub tools.
2. Download the model files.
3. Run one file with llama-cli, always naming the file, because the repository has three.

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B-GGUF

How to use it: Run llama-cli with the file you want. The card recommends the Q4_1 file for on-device use. You can also start a local server.

llama-cli -hf taobao-mnn/MNN-Pocket-0.3B-GGUF \
    --hf-file MNN-Pocket-0.3B-Instruct-Q4_1.gguf \
    --temp 0.8 --top-p 0.8 --top-k 20

llama-server -hf taobao-mnn/MNN-Pocket-0.3B-GGUF --hf-file MNN-Pocket-0.3B-Instruct-Q4_1.gguf \
    -c 4096 --port 8080

Without a file name, llama.cpp picks the Q8_0 file, which is not the Q4_1 build. A short tag selects a file too.

llama-cli -hf taobao-mnn/MNN-Pocket-0.3B-GGUF:Q4_1

Tool | What it does
llama-cli | Chats with the chosen file in a terminal
llama-server | Starts a local server with an OpenAI-compatible endpoint
hf-file option | Picks one of the three files by name

Example: You run llama-cli with the Q4_1 file and the prompt "What is MNN?". The model answers in the terminal in a few sentences.

For newcomers and experts: If you are new to GGUF, run the Q4_1 command and ask one short question. It is the smallest file and is the one the card suggests for phones and similar devices.

The Q4_1 file was trained with quantization in the loop, so it keeps near-BF16 accuracy, 49.59 against 50.00 on the card's 11-benchmark average. The model is a decoder-only transformer with 28 layers, a 32K context and a 65,536-token vocabulary. A ChatML-style chat template is stored in the file.

Pick a file, Q4_1 for small devices
Run llama-cli or llama-server
The chat template applies itself
Chat or call the local endpoint

Good to know:

• Size range: the three files run from 192 MB to 726 MB
• Limits: like other models under a billion parameters, it is not meant for reasoning-heavy work

Title: File size of each version in MB
Q4_1: 192 MB
Q8_0: 387 MB
F16: 726 MB

• License: apache-2.0
• Cost to run: no API key or paid service is named, only your own computer
• Data it reaches: downloading fetches model files, and chats stay on your machine
• Not for: advanced math and long coding tasks, which the card says to avoid

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 362M
• Weights: about 181 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 12,467
• Likes: 20
• License: apache-2.0
• Languages: en, zh
• Based on: taobao-mnn/MNN-Pocket-0.3B
• Tags: gguf, mnn-pocket, on-device, llama-cpp, text-generation, conversational

Links

• Model: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-GGUF
• Files: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.