--- title: "MNN-Pocket-0.3B-MNN" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/5eda7b9e-d7d5-4ce9-9728-a0a5171dda47" updated: "2026-11-13" --- # MNN-Pocket-0.3B-MNN > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs a 0.3B Chinese and English chat model on phones and edge devices through the MNN runtime, as a 4-bit build of about 173 MB. For developers adding on-device assistants to mobile apps. MNN-Pocket-0.3B-MNN is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by taobao-mnn ## About ### What it is This is the MNN-format export of MNN-Pocket-0.3B, a chat model with about 300M parameters that reads and writes Chinese and English. It is built to run on a phone or an edge device instead of a server. The build holds 4-bit weights trained with quantization in mind, and the whole package weighs about 173 MB. ### Who it suits It suits mobile and embedded developers who want a small language model inside an app, with no server in the loop. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Mobile apps | Android and iOS developers | Add a small chat model that works without a server | An assistant that lives inside an app | | Edge devices | Embedded engineers | Run text generation on hardware with little memory | Running the demo program on a CPU backend | | Data handling | App developers | Pull fields out of text on the device | Extracting details from a short message | | Research | Inference engineers | Study how a 4-bit, 300M model behaves on phone chips | Reading the card's speed and score results | If you need strong math or long code reasoning, you can skip it, since the card says it is not the right fit. ### Where to use it - Android, iOS and HarmonyOS apps, through the MNN LLM API - Edge devices - A computer with CMake and make, for building and trying the demo program - CPU, Metal or OpenCL backends, chosen in config.json | Use case | Who | What they get back | | --- | --- | --- | | On-device assistant | App developers | Chat replies generated on the phone | | Data extraction | Developers | Fields pulled out of text without a network call | | Small agent pipelines | Agent builders | Light model steps that run locally | ### How to set it up - Python and the Hugging Face hub package, to download the files - Git, CMake and make, for the MNN source - An MNN build from the same revision family as this export, since a mismatched one fails with the error Load module failed - About 173 MB of space for the model 1. Install the hub package and download the model files. 2. Clone the MNN source. The card gives the git command, which holds a web address, so copy it from there. 3. Compile the source with the cmake and make lines in the first block below, which turn on the LLM parts. 4. Run the demo program with the path to config.json and a text file holding your prompt. ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B-MNN ``` ```bash cd MNN mkdir build && cd build cmake .. -DMNN_LOW_MEMORY=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true make -j ``` ### How to use it Write your prompt into a text file and run the demo program with the path to config.json. Settings live in config.json, so changing them needs no new export. A default build leaves the GPU backends off, so add -DMNN\_METAL=ON or -DMNN\_OPENCL=ON to the cmake line when you want one. The card recommends a temperature of 0.8, top\_p of 0.8 and top\_k of 20. ```bash ./llm_demo /path/to/MNN-Pocket-0.3B-MNN/config.json prompt.txt ``` | Tool | What it does | | --- | --- | | llm.mnn | Holds the compute graph | | llm.mnn.weight | Holds the INT4 weights | | llm\_config.json | Holds the model layout and the ChatML-style chat template | | tokenizer.mtok | Holds the MNN tokenizer | | config.json | Sets the backend, thread count, precision, memory mode and sampler | ### Example You write a question into prompt.txt and run llm\_demo against config.json. The demo program then generates a reply with the model. The card does not show the output, so expect it in your terminal. ### For newcomers and experts This model is small enough to run on a phone. MNN is the runtime that loads and runs these files. A good first try is to build the demo on a computer and run one short prompt before touching a mobile app. Under the hood, it is a decoder-only transformer with 28 layers, a hidden size of 896 and 14 attention heads that share 2 key-value heads. The vocabulary has 65,536 entries, input and output embeddings are tied, and the context reaches 32K. The INT4 weights use a group size of 64. Prompt in a text file → MNN runtime reads config.json → INT4 model runs on CPU, Metal or OpenCL → Reply text comes out ### Good to know Decode speed at 512-token context, INT4 | Label | Value | | --- | --- | | iPad Pro M5 on Metal | 466 tok/s | | Snapdragon 8 Gen 5 on CPU | 251 tok/s | Average score on the 11-benchmark suite | Label | Value | | --- | --- | | BF16 | 50.00 points | | INT4 | 49.59 points | - The INT4 build scores 0.41 points below BF16 on average, which the card credits to training with quantization in mind. - The chat template ships inside llm\_config.json, and the MNN runtime applies it for you. - The 466 tok/s figure is a best case, labelled up to in the card. - License: Apache 2.0 - Cost to run: none named - Data it reaches: none named, and it runs on the device - Not for: advanced math or long-horizon code, which the card does not recommend ## README ````markdown # MNN-Pocket-0.3B-MNN Runs a 0.3B Chinese and English chat model on phones and edge devices through the MNN runtime, as a 4-bit build of about 173 MB. For developers adding on-device assistants to mobile apps. Made by taobao-mnn on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is the MNN-format export of MNN-Pocket-0.3B, a chat model with about 300M parameters that reads and writes Chinese and English. It is built to run on a phone or an edge device instead of a server. The build holds 4-bit weights trained with quantization in mind, and the whole package weighs about 173 MB. Who it suits: It suits mobile and embedded developers who want a small language model inside an app, with no server in the loop. ```table Industry | Who uses it | How they use it | Example Mobile apps | Android and iOS developers | Add a small chat model that works without a server | An assistant that lives inside an app Edge devices | Embedded engineers | Run text generation on hardware with little memory | Running the demo program on a CPU backend Data handling | App developers | Pull fields out of text on the device | Extracting details from a short message Research | Inference engineers | Study how a 4-bit, 300M model behaves on phone chips | Reading the card's speed and score results ``` If you need strong math or long code reasoning, you can skip it, since the card says it is not the right fit. Where to use it: - Android, iOS and HarmonyOS apps, through the MNN LLM API - Edge devices - A computer with CMake and make, for building and trying the demo program - CPU, Metal or OpenCL backends, chosen in config.json ```table Use case | Who | What they get back On-device assistant | App developers | Chat replies generated on the phone Data extraction | Developers | Fields pulled out of text without a network call Small agent pipelines | Agent builders | Light model steps that run locally ``` How to set it up: - Python and the Hugging Face hub package, to download the files - Git, CMake and make, for the MNN source - An MNN build from the same revision family as this export, since a mismatched one fails with the error Load module failed - About 173 MB of space for the model 1. Install the hub package and download the model files. 2. Clone the MNN source. The card gives the git command, which holds a web address, so copy it from there. 3. Compile the source with the cmake and make lines in the first block below, which turn on the LLM parts. 4. Run the demo program with the path to config.json and a text file holding your prompt. ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B-MNN ``` ```bash cd MNN mkdir build && cd build cmake .. -DMNN_LOW_MEMORY=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true make -j ``` How to use it: Write your prompt into a text file and run the demo program with the path to config.json. Settings live in config.json, so changing them needs no new export. A default build leaves the GPU backends off, so add -DMNN_METAL=ON or -DMNN_OPENCL=ON to the cmake line when you want one. The card recommends a temperature of 0.8, top_p of 0.8 and top_k of 20. ```bash ./llm_demo /path/to/MNN-Pocket-0.3B-MNN/config.json prompt.txt ``` ```table Tool | What it does llm.mnn | Holds the compute graph llm.mnn.weight | Holds the INT4 weights llm_config.json | Holds the model layout and the ChatML-style chat template tokenizer.mtok | Holds the MNN tokenizer config.json | Sets the backend, thread count, precision, memory mode and sampler ``` Example: You write a question into prompt.txt and run llm_demo against config.json. The demo program then generates a reply with the model. The card does not show the output, so expect it in your terminal. For newcomers and experts: This model is small enough to run on a phone. MNN is the runtime that loads and runs these files. A good first try is to build the demo on a computer and run one short prompt before touching a mobile app. Under the hood, it is a decoder-only transformer with 28 layers, a hidden size of 896 and 14 attention heads that share 2 key-value heads. The vocabulary has 65,536 entries, input and output embeddings are tied, and the context reaches 32K. The INT4 weights use a group size of 64. ```flow Prompt in a text file MNN runtime reads config.json INT4 model runs on CPU, Metal or OpenCL Reply text comes out ``` Good to know: ```chart Title: Decode speed at 512-token context, INT4 iPad Pro M5 on Metal: 466 tok/s Snapdragon 8 Gen 5 on CPU: 251 tok/s ``` ```chart Title: Average score on the 11-benchmark suite BF16: 50.00 points INT4: 49.59 points ``` - The INT4 build scores 0.41 points below BF16 on average, which the card credits to training with quantization in mind. - The chat template ships inside llm_config.json, and the MNN runtime applies it for you. - The 466 tok/s figure is a best case, labelled up to in the card. - License: Apache 2.0 - Cost to run: none named - Data it reaches: none named, and it runs on the device - Not for: advanced math or long-horizon code, which the card does not recommend ## On Hugging Face, 13 November 2026 - Task: text generation - Downloads in the last 30 days: 1,303 - Likes: 6 - License: apache-2.0 - Languages: en, zh - Based on: taobao-mnn/MNN-Pocket-0.3B - Tags: mnn-pocket, on-device, int4, text-generation ## Links - Model: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-MNN - Files: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-MNN/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B-MNN ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [taobao-mnn/MNN-Pocket-0.3B-MNN on Hugging Face](https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-MNN) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | MNN-Pocket-0.3B-MNN (this page) | 1,303 | 6 | — | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B](https://mindket.com/p/499799ab-3a05-4d11-854c-81e4c4e3e6fa) | 3,397 | 197 | 362M | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-GGUF](https://mindket.com/p/1e60bd47-21da-4333-9781-f0f82b9a2ebb) | 12,467 | 20 | 362M | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-Base](https://mindket.com/p/376694fb-a8b0-4174-bac8-b0049742053f) | 500 | 3 | 362M | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-Dec](https://mindket.com/p/9ecfcae9-9da9-47fc-9760-cff45374f2d0) | 0 | 4 | — | apache-2.0 | 2026-11-10 | | [Flint INT4](https://mindket.com/p/c7664095-d6ba-4bb6-a61d-07cbb0082f2c) | 22,586 | 30 | — | apache-2.0 | 2026-10-21 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-10 - First published: 2026-09-26 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open taobao-mnn/MNN-Pocket-0.3B-MNN on Hugging Face](https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-MNN) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/5eda7b9e-d7d5-4ce9-9728-a0a5171dda47) - [Its page on Mindket](https://mindket.com/p/5eda7b9e-d7d5-4ce9-9728-a0a5171dda47)