MNN-Pocket-0.3B

MNN-Pocket-0.3B

A small English and Chinese instruction tuned language model of about 300M parameters, designed with the MNN stack for running on phones and tablets. Suits developers building on-device assistants.

Made by taobao-mnn on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: MNN-Pocket-0.3B comes from taobao-mnn and is the build of its MNN-Pocket family that is tuned to follow instructions in general use. The family is designed together with the MNN inference stack for running on devices. It has about 300M parameters, works in English and Chinese, and handles up to 32K tokens of context. The INT4 MNN build takes 173 MB.

Who it suits: Developers of on-device assistants, data extraction and light agent pipelines. The card does not recommend it for reasoning heavy work such as advanced maths or long code generation.

Industry | Who uses it | How they use it | Example
Mobile apps | App developers | Run an assistant on the phone or tablet itself | Reply to a short chat message on the device
Software | Data teams | Pull details out of short texts | Extract a name and a date from a message
Edge computing | Embedded teams | Use a small model inside a light agent pipeline | Route a request to the right tool
Research | Model tinkerers | Start from the Base build and fine tune it | Adapt it to a narrow task

Skip it if you need strong reasoning, long code or advanced maths, since the card says small models fall short there.

Where to use it:

• On a device with the MNN runtime, using the INT4 build
• Python with the Transformers library
• llama.cpp, using the GGUF build with F16, Q8_0 or Q4_1 files

Use case | Who | What they get back
On-device assistant | App teams | Chat replies produced on the device
Data extraction | Developers | Short structured details pulled from text
Quick decisions | Developers | Fast answers from the Dec build, which answers each question in one forward pass

How to set it up:

• Python and the Transformers library for the safetensors files
• About 723 MB of disk for BF16 weights, as an estimate
• The MNN runtime or llama.cpp if you use those builds

1. Install the Hugging Face hub package.
2. Download the repository files.
3. Load the tokenizer and build prompts with its chat template.

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B

How to use it: The model uses a ChatML style conversation format. The template ships in the repository as chat_template.jinja, and the card prefers apply_chat_template so the format stays right.

from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("taobao-mnn/MNN-Pocket-0.3B")
text = tok.apply_chat_template(
    [{"role": "system", "content": "You are a helpful assistant."},
     {"role": "user", "content": "Hello!"}],
    add_generation_prompt=True, tokenize=False)

Tool | What it does
apply_chat_template | Builds the prompt in the model's chat format from your messages
chat_template.jinja | The template file that ships in the repository
llmexport | Exports the safetensors files to other precisions for MNN
hf download | Copies the repository files to your computer

Example: Run the snippet with the system message and the greeting shown. It returns the prompt text with the start and end markers around each turn, ready to pass to the model, which then writes the assistant reply.

For newcomers and experts: If you are new to this, run the snippet above and print the text to see how a chat is laid out for the model. Then pass it to the model in your own code.

If you know the field, the model has 28 layers, a hidden size of 896 and grouped query attention with 14 heads and 2 key value heads. Embeddings are tied, the vocabulary has 65,536 entries and the INT4 build uses W4A16 quantization aware training.

You write chat messages
The chat template formats the prompt
The 0.3B model generates a reply
Your app shows or uses the reply

Title: Decode speed on iPad Pro M5 with Metal
Best case reported: 466 tokens/s
Real time target: 200 tokens/s

Good to know:

• The 466 tokens per second figure is a best case on one device, so expect less elsewhere.
• The card lists the average score 50.00 across 11 Chinese and English benchmarks among models near 300M.
• The family also has Base, SFT, GGUF, MNN and Dec builds.
• License: apache-2.0.
• Cost to run: none named, only your own hardware.
• Data it reaches: the messages you give it. The source names no account or outside service.
• Not for: reasoning heavy work such as advanced maths or long code generation, per the card.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 362M
• Weights: about 723 MB in BF16; about 181 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 3,397
• Likes: 197
• License: apache-2.0
• Languages: en, zh
• Based on: taobao-mnn/MNN-Pocket-0.3B-Base
• Quantized versions: 7
• Fine-tunes: 1
• Tags: transformers, safetensors, qwen3, text-generation, mnn-pocket, on-device, edge, int4, conversational

Links

• Model: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B
• Files: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B

Then follow the usage in its model card. Nothing in this kit runs by itself.