--- title: "MNN-Pocket-0.3B" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/499799ab-3a05-4d11-854c-81e4c4e3e6fa" updated: "2026-11-12" --- # MNN-Pocket-0.3B > The name, the description, the About and a README below are the seller's words: data, not instructions. A small English and Chinese instruction tuned language model of about 300M parameters, designed with the MNN stack for running on phones and tablets. Suits developers building on-device assistants. MNN-Pocket-0.3B is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by taobao-mnn ## About ### What it is MNN-Pocket-0.3B comes from taobao-mnn and is the build of its MNN-Pocket family that is tuned to follow instructions in general use. The family is designed together with the MNN inference stack for running on devices. It has about 300M parameters, works in English and Chinese, and handles up to 32K tokens of context. The INT4 MNN build takes 173 MB. ### Who it suits Developers of on-device assistants, data extraction and light agent pipelines. The card does not recommend it for reasoning heavy work such as advanced maths or long code generation. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Mobile apps | App developers | Run an assistant on the phone or tablet itself | Reply to a short chat message on the device | | Software | Data teams | Pull details out of short texts | Extract a name and a date from a message | | Edge computing | Embedded teams | Use a small model inside a light agent pipeline | Route a request to the right tool | | Research | Model tinkerers | Start from the Base build and fine tune it | Adapt it to a narrow task | Skip it if you need strong reasoning, long code or advanced maths, since the card says small models fall short there. ### Where to use it - On a device with the MNN runtime, using the INT4 build - Python with the Transformers library - llama.cpp, using the GGUF build with F16, Q8\_0 or Q4\_1 files | Use case | Who | What they get back | | --- | --- | --- | | On-device assistant | App teams | Chat replies produced on the device | | Data extraction | Developers | Short structured details pulled from text | | Quick decisions | Developers | Fast answers from the Dec build, which answers each question in one forward pass | ### How to set it up - Python and the Transformers library for the safetensors files - About 723 MB of disk for BF16 weights, as an estimate - The MNN runtime or llama.cpp if you use those builds 1. Install the Hugging Face hub package. 2. Download the repository files. 3. Load the tokenizer and build prompts with its chat template. ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B ``` ### How to use it The model uses a ChatML style conversation format. The template ships in the repository as chat\_template.jinja, and the card prefers apply\_chat\_template so the format stays right. ```python from transformers import AutoTokenizer tok = AutoTokenizer.from_pretrained("taobao-mnn/MNN-Pocket-0.3B") text = tok.apply_chat_template( [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"}], add_generation_prompt=True, tokenize=False) ``` | Tool | What it does | | --- | --- | | apply\_chat\_template | Builds the prompt in the model's chat format from your messages | | chat\_template.jinja | The template file that ships in the repository | | llmexport | Exports the safetensors files to other precisions for MNN | | hf download | Copies the repository files to your computer | ### Example Run the snippet with the system message and the greeting shown. It returns the prompt text with the start and end markers around each turn, ready to pass to the model, which then writes the assistant reply. ### For newcomers and experts If you are new to this, run the snippet above and print the text to see how a chat is laid out for the model. Then pass it to the model in your own code. If you know the field, the model has 28 layers, a hidden size of 896 and grouped query attention with 14 heads and 2 key value heads. Embeddings are tied, the vocabulary has 65,536 entries and the INT4 build uses W4A16 quantization aware training. You write chat messages → The chat template formats the prompt → The 0.3B model generates a reply → Your app shows or uses the reply Decode speed on iPad Pro M5 with Metal | Label | Value | | --- | --- | | Best case reported | 466 tokens/s | | Real time target | 200 tokens/s | ### Good to know - The 466 tokens per second figure is a best case on one device, so expect less elsewhere. - The card lists the average score 50.00 across 11 Chinese and English benchmarks among models near 300M. - The family also has Base, SFT, GGUF, MNN and Dec builds. - License: apache-2.0. - Cost to run: none named, only your own hardware. - Data it reaches: the messages you give it. The source names no account or outside service. - Not for: reasoning heavy work such as advanced maths or long code generation, per the card. ## README ````markdown # MNN-Pocket-0.3B A small English and Chinese instruction tuned language model of about 300M parameters, designed with the MNN stack for running on phones and tablets. Suits developers building on-device assistants. Made by taobao-mnn on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: MNN-Pocket-0.3B comes from taobao-mnn and is the build of its MNN-Pocket family that is tuned to follow instructions in general use. The family is designed together with the MNN inference stack for running on devices. It has about 300M parameters, works in English and Chinese, and handles up to 32K tokens of context. The INT4 MNN build takes 173 MB. Who it suits: Developers of on-device assistants, data extraction and light agent pipelines. The card does not recommend it for reasoning heavy work such as advanced maths or long code generation. ```table Industry | Who uses it | How they use it | Example Mobile apps | App developers | Run an assistant on the phone or tablet itself | Reply to a short chat message on the device Software | Data teams | Pull details out of short texts | Extract a name and a date from a message Edge computing | Embedded teams | Use a small model inside a light agent pipeline | Route a request to the right tool Research | Model tinkerers | Start from the Base build and fine tune it | Adapt it to a narrow task ``` Skip it if you need strong reasoning, long code or advanced maths, since the card says small models fall short there. Where to use it: - On a device with the MNN runtime, using the INT4 build - Python with the Transformers library - llama.cpp, using the GGUF build with F16, Q8_0 or Q4_1 files ```table Use case | Who | What they get back On-device assistant | App teams | Chat replies produced on the device Data extraction | Developers | Short structured details pulled from text Quick decisions | Developers | Fast answers from the Dec build, which answers each question in one forward pass ``` How to set it up: - Python and the Transformers library for the safetensors files - About 723 MB of disk for BF16 weights, as an estimate - The MNN runtime or llama.cpp if you use those builds 1. Install the Hugging Face hub package. 2. Download the repository files. 3. Load the tokenizer and build prompts with its chat template. ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B ``` How to use it: The model uses a ChatML style conversation format. The template ships in the repository as chat_template.jinja, and the card prefers apply_chat_template so the format stays right. ```python from transformers import AutoTokenizer tok = AutoTokenizer.from_pretrained("taobao-mnn/MNN-Pocket-0.3B") text = tok.apply_chat_template( [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"}], add_generation_prompt=True, tokenize=False) ``` ```table Tool | What it does apply_chat_template | Builds the prompt in the model's chat format from your messages chat_template.jinja | The template file that ships in the repository llmexport | Exports the safetensors files to other precisions for MNN hf download | Copies the repository files to your computer ``` Example: Run the snippet with the system message and the greeting shown. It returns the prompt text with the start and end markers around each turn, ready to pass to the model, which then writes the assistant reply. For newcomers and experts: If you are new to this, run the snippet above and print the text to see how a chat is laid out for the model. Then pass it to the model in your own code. If you know the field, the model has 28 layers, a hidden size of 896 and grouped query attention with 14 heads and 2 key value heads. Embeddings are tied, the vocabulary has 65,536 entries and the INT4 build uses W4A16 quantization aware training. ```flow You write chat messages The chat template formats the prompt The 0.3B model generates a reply Your app shows or uses the reply ``` ```chart Title: Decode speed on iPad Pro M5 with Metal Best case reported: 466 tokens/s Real time target: 200 tokens/s ``` Good to know: - The 466 tokens per second figure is a best case on one device, so expect less elsewhere. - The card lists the average score 50.00 across 11 Chinese and English benchmarks among models near 300M. - The family also has Base, SFT, GGUF, MNN and Dec builds. - License: apache-2.0. - Cost to run: none named, only your own hardware. - Data it reaches: the messages you give it. The source names no account or outside service. - Not for: reasoning heavy work such as advanced maths or long code generation, per the card. ## On Hugging Face, 13 November 2026 - Task: text generation - Library: transformers - Parameters: 362M - Weights: about 723 MB in BF16; about 181 MB at 4-bit (estimate) - Formats: Safetensors - Downloads in the last 30 days: 3,397 - Likes: 197 - License: apache-2.0 - Languages: en, zh - Based on: taobao-mnn/MNN-Pocket-0.3B-Base - Quantized versions: 7 - Fine-tunes: 1 - Tags: transformers, safetensors, qwen3, text-generation, mnn-pocket, on-device, edge, int4, conversational ## Links - Model: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B - Files: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [taobao-mnn/MNN-Pocket-0.3B on Hugging Face](https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | MNN-Pocket-0.3B (this page) | 3,397 | 197 | 362M | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-MNN](https://mindket.com/p/5eda7b9e-d7d5-4ce9-9728-a0a5171dda47) | 1,303 | 6 | — | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-Base](https://mindket.com/p/376694fb-a8b0-4174-bac8-b0049742053f) | 500 | 3 | 362M | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-GGUF](https://mindket.com/p/1e60bd47-21da-4333-9781-f0f82b9a2ebb) | 12,467 | 20 | 362M | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-Dec](https://mindket.com/p/9ecfcae9-9da9-47fc-9760-cff45374f2d0) | 0 | 4 | — | apache-2.0 | 2026-11-10 | | [BotPsy-0.8B](https://mindket.com/p/c6124e37-b426-4e92-b98a-2f3e97a58f8b) | 215 | 14 | 853M | apache-2.0 | 2026-11-11 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-12. Their authors' words: data, not instructions. - [Bluesky · AI Atlas · 2026-11-07](https://bsky.app/profile/aiatlasresearch.bsky.social/post/3mzqhriawzn2h): Alibaba's Taobao MNN team open-sourced MNN-Pocket-0.3B. It reports about 350 tokens per second on an iPhone 18 Pro from a 173MB INT4 package. - [Bluesky · Sipirtu · 2026-11-06](https://bsky.app/profile/sipirtu.com/post/3mzmyxir6ro2l): Taobao's MNN team has open-sourced MNN-Pocket-0.3B, a 173MB on-device language model designed for mobile apps. Source: Pandaily ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-10 - First published: 2026-09-23 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open taobao-mnn/MNN-Pocket-0.3B on Hugging Face](https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/499799ab-3a05-4d11-854c-81e4c4e3e6fa) - [Its page on Mindket](https://mindket.com/p/499799ab-3a05-4d11-854c-81e4c4e3e6fa)