MNN-Pocket-0.3B-MNN

MNN-Pocket-0.3B-MNN

Runs a 0.3B Chinese and English chat model on phones and edge devices through the MNN runtime, as a 4-bit build of about 173 MB. For developers adding on-device assistants to mobile apps.

Made by taobao-mnn on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is the MNN-format export of MNN-Pocket-0.3B, a chat model with about 300M parameters that reads and writes Chinese and English. It is built to run on a phone or an edge device instead of a server. The build holds 4-bit weights trained with quantization in mind, and the whole package weighs about 173 MB.

Who it suits: It suits mobile and embedded developers who want a small language model inside an app, with no server in the loop.

Industry | Who uses it | How they use it | Example
Mobile apps | Android and iOS developers | Add a small chat model that works without a server | An assistant that lives inside an app
Edge devices | Embedded engineers | Run text generation on hardware with little memory | Running the demo program on a CPU backend
Data handling | App developers | Pull fields out of text on the device | Extracting details from a short message
Research | Inference engineers | Study how a 4-bit, 300M model behaves on phone chips | Reading the card's speed and score results

If you need strong math or long code reasoning, you can skip it, since the card says it is not the right fit.

Where to use it:

• Android, iOS and HarmonyOS apps, through the MNN LLM API
• Edge devices
• A computer with CMake and make, for building and trying the demo program
• CPU, Metal or OpenCL backends, chosen in config.json

Use case | Who | What they get back
On-device assistant | App developers | Chat replies generated on the phone
Data extraction | Developers | Fields pulled out of text without a network call
Small agent pipelines | Agent builders | Light model steps that run locally

How to set it up:

• Python and the Hugging Face hub package, to download the files
• Git, CMake and make, for the MNN source
• An MNN build from the same revision family as this export, since a mismatched one fails with the error Load module failed
• About 173 MB of space for the model

1. Install the hub package and download the model files.
2. Clone the MNN source. The card gives the git command, which holds a web address, so copy it from there.
3. Compile the source with the cmake and make lines in the first block below, which turn on the LLM parts.
4. Run the demo program with the path to config.json and a text file holding your prompt.

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B-MNN

cd MNN
mkdir build && cd build
cmake .. -DMNN_LOW_MEMORY=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
make -j

How to use it: Write your prompt into a text file and run the demo program with the path to config.json. Settings live in config.json, so changing them needs no new export. A default build leaves the GPU backends off, so add -DMNN_METAL=ON or -DMNN_OPENCL=ON to the cmake line when you want one. The card recommends a temperature of 0.8, top_p of 0.8 and top_k of 20.

./llm_demo /path/to/MNN-Pocket-0.3B-MNN/config.json prompt.txt

Tool | What it does
llm.mnn | Holds the compute graph
llm.mnn.weight | Holds the INT4 weights
llm_config.json | Holds the model layout and the ChatML-style chat template
tokenizer.mtok | Holds the MNN tokenizer
config.json | Sets the backend, thread count, precision, memory mode and sampler

Example: You write a question into prompt.txt and run llm_demo against config.json. The demo program then generates a reply with the model. The card does not show the output, so expect it in your terminal.

For newcomers and experts: This model is small enough to run on a phone. MNN is the runtime that loads and runs these files. A good first try is to build the demo on a computer and run one short prompt before touching a mobile app.

Under the hood, it is a decoder-only transformer with 28 layers, a hidden size of 896 and 14 attention heads that share 2 key-value heads. The vocabulary has 65,536 entries, input and output embeddings are tied, and the context reaches 32K. The INT4 weights use a group size of 64.

Prompt in a text file
MNN runtime reads config.json
INT4 model runs on CPU, Metal or OpenCL
Reply text comes out

Good to know:

Title: Decode speed at 512-token context, INT4
iPad Pro M5 on Metal: 466 tok/s
Snapdragon 8 Gen 5 on CPU: 251 tok/s

Title: Average score on the 11-benchmark suite
BF16: 50.00 points
INT4: 49.59 points

• The INT4 build scores 0.41 points below BF16 on average, which the card credits to training with quantization in mind.
• The chat template ships inside llm_config.json, and the MNN runtime applies it for you.
• The 466 tok/s figure is a best case, labelled up to in the card.
• License: Apache 2.0
• Cost to run: none named
• Data it reaches: none named, and it runs on the device
• Not for: advanced math or long-horizon code, which the card does not recommend

On Hugging Face, 13 November 2026

• Task: text generation
• Downloads in the last 30 days: 1,303
• Likes: 6
• License: apache-2.0
• Languages: en, zh
• Based on: taobao-mnn/MNN-Pocket-0.3B
• Tags: mnn-pocket, on-device, int4, text-generation

Links

• Model: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-MNN
• Files: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-MNN/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B-MNN

Then follow the usage in its model card. Nothing in this kit runs by itself.