MNN-Pocket-0.3B-Base

MNN-Pocket-0.3B-Base

A pre-trained base language model of about 300M parameters for Chinese and English, designed with the MNN inference engine for on-device use, as a start for further training or fine-tuning.

Made by taobao-mnn on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: MNN-Pocket-0.3B-Base is the raw, pre-trained form of MNN-Pocket-0.3B, a small language model with roughly 300M parameters. It was designed alongside the MNN inference engine so it can run on devices.

This checkpoint has no instruction tuning. It simply continues text, which makes it a place to begin more pre-training or fine-tuning. For chat or assistant use, the card sends you to the instruction-tuned MNN-Pocket-0.3B instead.

Who it suits: ML engineers and researchers who want a small Chinese and English base model to train further. It also suits app teams planning a custom on-device model.

Industry | Who uses it | How they use it | Example
Research | Language model researchers | Continue pre-training on their own text | Adapt the base to a specialist domain
Mobile and edge software | On-device app teams | Start a small custom model for a phone app | Fine-tune the base, then convert it for MNN
Education | Students | Study how a compact 28-layer model is built | Read the model details table and try plain completion

People who want a ready chat assistant can skip this checkpoint and take the instruction-tuned version.

Where to use it:

• Python with the Hugging Face transformers library
• On-device apps, through the MNN engine and the deployment builds listed on the card
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Plain text completion | Developers | A continuation of the text you give it
Continued pre-training | Researchers | A base ready to learn from new text
Fine-tuning | ML engineers | A starting model for chat or other tasks

How to set it up: You need a Python environment with the transformers library.

• Python and pip
• The transformers library
• Disk space for about 723 MB of BF16 weights

1. Install the huggingface_hub package if you want the files on disk.
2. Download the model files with the hf command.
3. Load the model and tokenizer with transformers, using the snippet below.

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B-Base

How to use it: Load the model and tokenizer, then give it text to continue. The card notes that this is a base checkpoint, so it does plain text completion with no chat formatting.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "taobao-mnn/MNN-Pocket-0.3B-Base"
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", dtype="bfloat16")
tokenizer = AutoTokenizer.from_pretrained(model_id)
# base checkpoint: plain text completion, no chat formatting

The card lists sibling repositories for other needs.

Variant | What it is
MNN-Pocket-0.3B | The instruction-tuned BF16 model, with the full model card
MNN-Pocket-0.3B-SFT | A supervised fine-tuning checkpoint
MNN-Pocket-0.3B-GGUF | A GGUF build for deployment
MNN-Pocket-0.3B-MNN | An INT4 build for the MNN engine
MNN-Pocket-0.3B-Dec | A non-autoregressive decision model, a LoRA adapter plus a pointer head

Example: You load the model with the snippet above and give it the start of a sentence, such as "The three main parts of a computer are". It continues the text. It does not answer like a chat assistant, since it has had no instruction tuning.

For newcomers and experts: Think of a base model as a text continuer. It has read a lot of text and predicts what comes next, but it has not been taught to follow requests. A good first try is the loading snippet and a half-written sentence.

Underneath, it is a decoder-only transformer trained with a multi-stage data recipe under a fixed budget of tokens per parameter. The card reports an average of 33.58 on an 11-benchmark base suite, ahead of three similar-sized base models. The trade-off is small size against the knowledge a larger model holds.

Property | Value
Layers | 28
Hidden size | 896
Attention | 14 heads, with 2 key-value heads (GQA)
Context length | 32K
Vocabulary | 65,536
Precision | bfloat16

Pre-train the base model
Continue training or fine-tune it
Convert for MNN or GGUF deployment
Run it on a device

Good to know: This is a base checkpoint, so use the instruction-tuned sibling when you want a chatbot.

• Size: the card says about 300M parameters, and Hugging Face's record counts 362M
• Weights: about 723 MB in BF16, and about 181 MB at 4-bit as an estimate
• Languages: Chinese and English
• License: Apache 2.0
• Cost to run: no API key or paid service is named
• Data it reaches: the text you give it and the model files you download from Hugging Face
• Not for: chat or assistant use, since this checkpoint has no instruction tuning

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 362M
• Weights: about 723 MB in BF16; about 181 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 500
• Likes: 3
• License: apache-2.0
• Languages: en, zh
• Quantized versions: 1
• Fine-tunes: 3
• Tags: safetensors, qwen3, mnn-pocket, on-device, base, text-generation, conversational

Links

• Model: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-Base
• Files: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-Base/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B-Base

Then follow the usage in its model card. Nothing in this kit runs by itself.