MNN-Pocket-0.3B-Dec

MNN-Pocket-0.3B-Dec

Reads a text state plus your questions and picks one of your options for each, with a probability per candidate, in a single pass. A decision model for developers, not a chat model.

Made by taobao-mnn on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a decision model, not a chatbot. You give it a text describing a situation plus a few questions. It answers each one by choosing among candidate options and scoring them, in one pass through the network. It is built on MNN-Pocket-0.3B, keeps that model's backbone and lm_head, and adds a separate LoRA adapter on top.

Who it suits: It suits developers who need a small model to route or label text inside an app, and machine learning engineers who want to study a decision head on a compact backbone.

Industry | Who uses it | How they use it | Example
Customer support | Support tool developers | Send each incoming message to the right team | A note about wrong size shoes goes to returns, not billing
Software | App developers | Add a labelling step that picks from options you define | Choose one department from a short list of criteria
Research | ML engineers | Study a pointer head on a small backbone with an adapter | Compare candidate scores from one forward pass

Anyone who only wants to chat or write text can skip it, since it does not generate replies.

Where to use it:

• Python scripts and services that use Transformers and PEFT
• A local CPU, since the card's example loads it on cpu
• Windows, macOS or Linux, where the files download with Python

Use case | Who | What they get back
Route a customer message | Support tool developers | The team that should handle it, with a probability per option
Label text against your own criteria | App developers | A choice for each question you define
Check how a decision head behaves | ML engineers | Scores over candidate positions from a single forward pass

How to set it up:

• Python with pip
• Disk space for about 723 MB of backbone plus a small adapter and head
• The package versions pinned in requirements.txt

1. Install the Hugging Face hub client.
2. Download the model folder to your computer.
3. Install the versions pinned in requirements.txt from that folder.
4. Keep the base folder inside the model directory, because the loader rejects any path that escapes it.

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B-Dec

How to use it: Load the folder with from_pretrained on the device you want, then call predict with a state and a dictionary of questions. Each question has a type, instructions and criteria that name the candidates.

from inference import PocketDecisionModel

model = PocketDecisionModel.from_pretrained(".", device="cpu", dtype="float32")
result = model.predict(
    state=(
        "The shoes arrived, but they are the wrong size. "
        "I want to send them back for an exchange. "
        "Payment and delivery were fine."
    ),
    questions={
        "department": {
            "type": "choice",
            "instructions": "Which team should handle this customer request?",
            "criteria": {
                "returns": "Returns, exchanges and refunds",
                "billing": "Charges, invoices and payment problems"
            }
        }
    }
)

Tool | What it does
PocketDecisionModel.from_pretrained | Loads the backbone, adapter and head from a folder, on the device and dtype you pick
predict | Takes a state and your questions and returns the decisions
inference.py | Holds the loader, the PocketDecisionModel API and a command line interface

Example: You pass a state saying the shoes arrived in the wrong size and the buyer wants an exchange, plus a department question that offers returns and billing. What comes back is a decision for the department question, together with a probability for each of the two candidates.

For newcomers and experts: Think of it as a sorter that reads a short note and picks a label, rather than something you talk to. Download the folder, run the card's example on a CPU, then swap in your own text and your own options.

The backbone is a decoder-only transformer with 28 layers, 14 attention heads and 2 key-value heads. A LoRA adapter at rank 16 sits unmerged beside it, and a float32 pointer head scores candidate positions with scaled dot-product. Because it is non-autoregressive, no token-by-token generation is needed. Requests can hold up to 32,768 tokens.

State and questions go in
Backbone plus LoRA adapter reads them
Pointer head scores each candidate
Decisions and probabilities come out

Good to know:

• Input limit: 32,768 tokens, and longer input raises an error instead of being cut
• Size: about 300M parameters in the backbone (723 MB), 28.7 MB of adapter and 1.8 MB of head
• Languages: Chinese and English
• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the model folder and the text you pass in, with no web or account access named
• Not for: chat or free-form text, since it only chooses among options you supply

On Hugging Face, 13 November 2026

• Library: peft
• Formats: Safetensors
• Downloads in the last 30 days: 0
• Likes: 4
• License: apache-2.0
• Languages: en, zh
• Based on: taobao-mnn/MNN-Pocket-0.3B
• Quantized versions: 2
• Tags: peft, safetensors, mnn-pocket, on-device, decision-model, non-autoregressive, lora

Links

• Model: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-Dec
• Files: https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B-Dec/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download taobao-mnn/MNN-Pocket-0.3B-Dec

Then follow the usage in its model card. Nothing in this kit runs by itself.