--- title: "MNN-Pocket-0.3B-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/1e60bd47-21da-4333-9781-f0f82b9a2ebb" updated: "2026-11-12" --- # MNN-Pocket-0.3B-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs a compact 0.3B Chinese and English language model from GGUF files of 192 to 726 MB, made for phones, laptops and other on-device use with llama.cpp, LM Studio or ollama. MNN-Pocket-0.3B-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by taobao-mnn ## About ### What it is MNN-Pocket-0.3B-GGUF is a GGUF export of MNN-Pocket-0.3B, a roughly 300M-parameter language model from taobao-mnn. The family is built together with the MNN inference stack for phones, tablets, laptops and smart glasses. The repository has three files, F16 at 726 MB, Q8\_0 at 387 MB and Q4\_1 at 192 MB, for llama.cpp and runtimes such as LM Studio and ollama. ### Who it suits People who want a tiny chat model for on-device assistants, data extraction or light agent pipelines, in Chinese or English. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Mobile apps | App developers | Put a small assistant on the device | A chat helper inside a phone app | | Data work | Analysts | Pull structured fields out of text | Extracting names and dates from notes | | Agent building | Pipeline developers | Run a light model in a tool chain | A small step that routes requests | | Hobby projects | Tinkerers | Try a model on a low-memory machine | Chat on an older laptop | If you need advanced math or long coding tasks, you can skip it, because the card does not recommend it for reasoning-heavy work. ### Where to use it - llama.cpp, with its llama-cli and llama-server programs - LM Studio and ollama - Windows, macOS or Linux, after you fetch the files with Python | Use case | Who | What they get back | | --- | --- | --- | | On-device chat | App builders | Short answers from a tiny model | | Data extraction | Analysts | Fields pulled from text | | Local server | Developers | An OpenAI-compatible endpoint on your own machine | ### How to set it up - Python and pip, to fetch the files - llama.cpp, a recent build so the chat template applies by itself - A few hundred MB of disk space, depending on the file you pick 1. Install the Hugging Face hub tools. 2. Download the model files. 3. Run one file with llama-cli, always naming the file, because the repository has three. ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B-GGUF ``` ### How to use it Run llama-cli with the file you want. The card recommends the Q4\_1 file for on-device use. You can also start a local server. ```bash llama-cli -hf taobao-mnn/MNN-Pocket-0.3B-GGUF \ --hf-file MNN-Pocket-0.3B-Instruct-Q4_1.gguf \ --temp 0.8 --top-p 0.8 --top-k 20 ``` ```bash llama-server -hf taobao-mnn/MNN-Pocket-0.3B-GGUF --hf-file MNN-Pocket-0.3B-Instruct-Q4_1.gguf \ -c 4096 --port 8080 ``` Without a file name, llama.cpp picks the Q8\_0 file, which is not the Q4\_1 build. A short tag selects a file too. ```bash llama-cli -hf taobao-mnn/MNN-Pocket-0.3B-GGUF:Q4_1 ``` | Tool | What it does | | --- | --- | | llama-cli | Chats with the chosen file in a terminal | | llama-server | Starts a local server with an OpenAI-compatible endpoint | | hf-file option | Picks one of the three files by name | ### Example You run llama-cli with the Q4\_1 file and the prompt "What is MNN?". The model answers in the terminal in a few sentences. ### For newcomers and experts If you are new to GGUF, run the Q4\_1 command and ask one short question. It is the smallest file and is the one the card suggests for phones and similar devices. The Q4\_1 file was trained with quantization in the loop, so it keeps near-BF16 accuracy, 49.59 against 50.00 on the card's 11-benchmark average. The model is a decoder-only transformer with 28 layers, a 32K context and a 65,536-token vocabulary. A ChatML-style chat template is stored in the file. Pick a file, Q4\_1 for small devices → Run llama-cli or llama-server → The chat template applies itself → Chat or call the local endpoint ### Good to know - Size range: the three files run from 192 MB to 726 MB - Limits: like other models under a billion parameters, it is not meant for reasoning-heavy work File size of each version in MB | Label | Value | | --- | --- | | Q4\_1 | 192 MB | | Q8\_0 | 387 MB | | F16 | 726 MB | - License: apache-2.0 - Cost to run: no API key or paid service is named, only your own computer - Data it reaches: downloading fetches model files, and chats stay on your machine - Not for: advanced math and long coding tasks, which the card says to avoid ## README ````markdown # MNN-Pocket-0.3B-GGUF Runs a compact 0.3B Chinese and English language model from GGUF files of 192 to 726 MB, made for phones, laptops and other on-device use with llama.cpp, LM Studio or ollama. Made by taobao-mnn on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: MNN-Pocket-0.3B-GGUF is a GGUF export of MNN-Pocket-0.3B, a roughly 300M-parameter language model from taobao-mnn. The family is built together with the MNN inference stack for phones, tablets, laptops and smart glasses. The repository has three files, F16 at 726 MB, Q8_0 at 387 MB and Q4_1 at 192 MB, for llama.cpp and runtimes such as LM Studio and ollama. Who it suits: People who want a tiny chat model for on-device assistants, data extraction or light agent pipelines, in Chinese or English. ```table Industry | Who uses it | How they use it | Example Mobile apps | App developers | Put a small assistant on the device | A chat helper inside a phone app Data work | Analysts | Pull structured fields out of text | Extracting names and dates from notes Agent building | Pipeline developers | Run a light model in a tool chain | A small step that routes requests Hobby projects | Tinkerers | Try a model on a low-memory machine | Chat on an older laptop ``` If you need advanced math or long coding tasks, you can skip it, because the card does not recommend it for reasoning-heavy work. Where to use it: - llama.cpp, with its llama-cli and llama-server programs - LM Studio and ollama - Windows, macOS or Linux, after you fetch the files with Python ```table Use case | Who | What they get back On-device chat | App builders | Short answers from a tiny model Data extraction | Analysts | Fields pulled from text Local server | Developers | An OpenAI-compatible endpoint on your own machine ``` How to set it up: - Python and pip, to fetch the files - llama.cpp, a recent build so the chat template applies by itself - A few hundred MB of disk space, depending on the file you pick 1. Install the Hugging Face hub tools. 2. Download the model files. 3. Run one file with llama-cli, always naming the file, because the repository has three. ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B-GGUF ``` How to use it: Run llama-cli with the file you want. The card recommends the Q4_1 file for on-device use. You can also start a local server. ```bash llama-cli -hf taobao-mnn/MNN-Pocket-0.3B-GGUF \ --hf-file MNN-Pocket-0.3B-Instruct-Q4_1.gguf \ --temp 0.8 --top-p 0.8 --top-k 20 ``` ```bash llama-server -hf taobao-mnn/MNN-Pocket-0.3B-GGUF --hf-file MNN-Pocket-0.3B-Instruct-Q4_1.gguf \ -c 4096 --port 8080 ``` Without a file name, llama.cpp picks the Q8_0 file, which is not the Q4_1 build. A short tag selects a file too. ```bash llama-cli -hf taobao-mnn/MNN-Pocket-0.3B-GGUF:Q4_1 ``` ```table Tool | What it does llama-cli | Chats with the chosen file in a terminal llama-server | Starts a local server with an OpenAI-compatible endpoint hf-file option | Picks one of the three files by name ``` Example: You run llama-cli with the Q4_1 file and the prompt "What is MNN?". The model answers in the terminal in a few sentences. For newcomers and experts: If you are new to GGUF, run the Q4_1 command and ask one short question. It is the smallest file and is the one the card suggests for phones and similar devices. The Q4_1 file was trained with quantization in the loop, so it keeps near-BF16 accuracy, 49.59 against 50.00 on the card's 11-benchmark average. The model is a decoder-only transformer with 28 layers, a 32K context and a 65,536-token vocabulary. A ChatML-style chat template is stored in the file. ```flow Pick a file, Q4_1 for small devices Run llama-cli or llama-server The chat template applies itself Chat or call the local endpoint ``` Good to know: - Size range: the three files run from 192 MB to 726 MB - Limits: like other models under a billion parameters, it is not meant for reasoning-heavy work ```chart Title: File size of each version in MB Q4_1: 192 MB Q8_0: 387 MB F16: 726 MB ``` - License: apache-2.0 - Cost to run: no API key or paid service is named, only your own computer - Data it reaches: downloading fetches model files, and chats stay on your machine - Not for: advanced math and long coding tasks, which the card says to avoid ## On Hugging Face, 13 November 2026 - Task: text generation - Library: gguf - Parameters: 362M - Weights: about 181 MB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 12,467 - Likes: 20 - License: apache-2.0 - Languages: en, zh - Based on: taobao-mnn/MNN-Pocket-0.3B - Tags: gguf, mnn-pocket, on-device, llama-cpp, text-generation, conversational ## Links - Model: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-GGUF - Files: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download taobao-mnn/MNN-Pocket-0.3B-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [taobao-mnn/MNN-Pocket-0.3B-GGUF on Hugging Face](https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | MNN-Pocket-0.3B-GGUF (this page) | 12,467 | 20 | 362M | apache-2.0 | 2026-11-10 | | [Bonsai-27B-gguf](https://mindket.com/p/0f4c3b2e-9ef6-4680-896e-4319b270e9f8) | 179,818 | 945 | 26.9B | apache-2.0 | 2026-07-17 | | [Bonsai-1.7B-GGUF](https://mindket.com/p/976ed9c8-c475-4a88-a05b-e149421d5c28) | 173,963 | 100 | 1.7B | apache-2.0 | 2026-04-18 | | [MNN-Pocket-0.3B](https://mindket.com/p/499799ab-3a05-4d11-854c-81e4c4e3e6fa) | 3,397 | 197 | 362M | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-Base](https://mindket.com/p/376694fb-a8b0-4174-bac8-b0049742053f) | 500 | 3 | 362M | apache-2.0 | 2026-11-10 | | [MNN-Pocket-0.3B-MNN](https://mindket.com/p/5eda7b9e-d7d5-4ce9-9728-a0a5171dda47) | 1,303 | 6 | — | apache-2.0 | 2026-11-10 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-10 - First published: 2026-09-26 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open taobao-mnn/MNN-Pocket-0.3B-GGUF on Hugging Face](https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/1e60bd47-21da-4333-9781-f0f82b9a2ebb) - [Its page on Mindket](https://mindket.com/p/1e60bd47-21da-4333-9781-f0f82b9a2ebb)