Bonsai-27B-GGUF
lmstudio-community · Kit
A GGUF conversion of the 27B Bonsai model for LM Studio, with a 3.80 GB Q1_0 language file and a 0.93 GB BF16 vision projector. Tested for text, vision and par…
New
Overview
The smallest base model in Qwen2.5, about 494M parameters with a 32,768-token context, meant for further training on your own data. For developers who want a tiny language model to build on.
Qwen2.5-0.5B is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on . Download it on this page, a zip of its files. Nothing in it runs by itself.
At a glance
- You need
- Python with pip.
- A recent transformers. Versions older than 4.37.0 fail with a KeyError for qwen2.
- Around 1 GB of space for the BF16 weights, or about a quarter of that at 4-bit, as estimated on the record.
- Runs in
- The transformers library in Python, with the Safetensors weights.
- Your own post-training setup, such as supervised fine-tuning, RLHF or continued pretraining.
- Windows, macOS or Linux: the download steps run on all three with Python.
- Start with
pip install -U huggingface_hub hf download Qwen/Qwen2.5-0.5B
Platform
Windows
macOS
Linux
About
What it is
Qwen2.5-0.5B is the smallest pretrained base model in the Qwen2.5 series from the Qwen team. The series runs from 0.5 to 72 billion parameters. This one is a causal language model with 24 layers, 14 query heads and 2 key and value heads, and it reads up to 32,768 tokens at once. It is a base model, so it has not been taught to hold a conversation.
Who it suits
Developers and researchers who want a light model to fine-tune, test quickly or keep on a small device.
| Industry | Who uses it | How they use it | Example |
|---|---|---|---|
| Software | ML engineers | Fine-tune a small model for one job | Train it to output JSON for your app |
| AI research | Researchers | Use a compact base for quick experiments | Try a new training idea in minutes |
| Education | Students | Learn language model training on modest hardware | Fine-tune it on a laptop or a free GPU |
Anyone who wants a model that chats well as soon as it downloads can skip it, since the card advises against using this base model for conversations.
Where to use it
- The transformers library in Python, with the Safetensors weights.
- Your own post-training setup, such as supervised fine-tuning, RLHF or continued pretraining.
- Windows, macOS or Linux: the download steps run on all three with Python.
| Use case | Who | What they get back |
|---|---|---|
| Fine-tuning | ML engineers | A small model adapted to your task |
| Structured output | App builders | A model tuned toward JSON or table-style answers |
| Research baselines | Researchers | A cheap reference model |
How to set it up
What you need first:
- Python with pip.
- A recent transformers. Versions older than 4.37.0 fail with a KeyError for qwen2.
- Around 1 GB of space for the BF16 weights, or about a quarter of that at 4-bit, as estimated on the record.
- Install the newest transformers.
- Install the Hugging Face hub package with pip.
- Download the repository with the hf command.
pip install -U huggingface_hub hf download Qwen/Qwen2.5-0.5B
How to use it
Load the model with transformers and train it further for your use. The card gives no code here, so use the Qwen documentation and the transformers guides for loading and training.
| Tool | What it does |
|---|---|
| SFT | Teaches the model from example answers |
| RLHF | Tunes it with human feedback |
| Continued pretraining | Keeps training on more raw text |
Example
You prepare a few thousand examples of messy text paired with clean JSON, then run supervised fine-tuning on this model. You end up with a small extractor that returns structured data from your own documents.
For newcomers and experts
A base model has only read text and learned to predict the next word. It has not been taught to follow instructions. Fine-tuning is the step that does that. A good first step is to fine-tune on a few dozen examples and compare its replies before and after.
The series claims gains over Qwen2 in knowledge, coding, math, following instructions, long text and structured output, with support for over 29 languages. Those claims are for the series, and the card gives no scores for this size. Its architecture uses RoPE, SwiGLU, RMSNorm, QKV bias in attention and tied embeddings, with 0.36B non-embedding parameters.
- Download the base model
- Prepare your own training data
- Post-train with SFT or RLHF
- Test the result on your task
Good to know
The record estimates how much space the weights take.
- Context: 32,768 tokens in full for this model.
- Versions: the record lists 118 quantized and 803 fine-tuned versions made by others.
- Language: English on the record, with the card naming over 29 languages for the series.
- License: apache-2.0
- Cost to run: no API key, account or paid service named.
- Data it reaches: files you download from Hugging Face and the text you train or run it on.
- Not for: use as a ready chat assistant without more training.
README and files
Qwen2.5-0.5B
The smallest base model in Qwen2.5, about 494M parameters with a 32,768-token context, meant for further training on your own data. For developers who want a tiny language model to build on.
Made by Qwen on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.
About
What it is: Qwen2.5-0.5B is the smallest pretrained base model in the Qwen2.5 series from the Qwen team. The series runs from 0.5 to 72 billion parameters. This one is a causal language model with 24 layers, 14 query heads and 2 key and value heads, and it reads up to 32,768 tokens at once. It is a base model, so it has not been taught to hold a conversation.
Who it suits: Developers and researchers who want a light model to fine-tune, test quickly or keep on a small device.
Industry | Who uses it | How they use it | Example
Software | ML engineers | Fine-tune a small model for one job | Train it to output JSON for your app
AI research | Researchers | Use a compact base for quick experiments | Try a new training idea in minutes
Education | Students | Learn language model training on modest hardware | Fine-tune it on a laptop or a free GPUAnyone who wants a model that chats well as soon as it downloads can skip it, since the card advises against using this base model for conversations.
Where to use it:
- The transformers library in Python, with the Safetensors weights.
- Your own post-training setup, such as supervised fine-tuning, RLHF or continued pretraining.
- Windows, macOS or Linux: the download steps run on all three with Python.
Use case | Who | What they get back
Fine-tuning | ML engineers | A small model adapted to your task
Structured output | App builders | A model tuned toward JSON or table-style answers
Research baselines | Researchers | A cheap reference modelHow to set it up: What you need first:
- Python with pip.
- A recent transformers. Versions older than 4.37.0 fail with a KeyError for qwen2.
- Around 1 GB of space for the BF16 weights, or about a quarter of that at 4-bit, as estimated on the record.
- Install the newest transformers.
- Install the Hugging Face hub package with pip.
- Download the repository with the hf command.
pip install -U huggingface_hub
hf download Qwen/Qwen2.5-0.5BHow to use it: Load the model with transformers and train it further for your use. The card gives no code here, so use the Qwen documentation and the transformers guides for loading and training.
Title: Training routes the card names
Tool | What it does
SFT | Teaches the model from example answers
RLHF | Tunes it with human feedback
Continued pretraining | Keeps training on more raw textExample: You prepare a few thousand examples of messy text paired with clean JSON, then run supervised fine-tuning on this model. You end up with a small extractor that returns structured data from your own documents.
For newcomers and experts: A base model has only read text and learned to predict the next word. It has not been taught to follow instructions. Fine-tuning is the step that does that. A good first step is to fine-tune on a few dozen examples and compare its replies before and after.
The series claims gains over Qwen2 in knowledge, coding, math, following instructions, long text and structured output, with support for over 29 languages. Those claims are for the series, and the card gives no scores for this size. Its architecture uses RoPE, SwiGLU, RMSNorm, QKV bias in attention and tied embeddings, with 0.36B non-embedding parameters.
Download the base model
Prepare your own training data
Post-train with SFT or RLHF
Test the result on your taskGood to know: The record estimates how much space the weights take.
Title: Weights size, estimate
BF16: 988 MB
4-bit: 247 MB- Context: 32,768 tokens in full for this model.
- Versions: the record lists 118 quantized and 803 fine-tuned versions made by others.
- Language: English on the record, with the card naming over 29 languages for the series.
- License: apache-2.0
- Cost to run: no API key, account or paid service named.
- Data it reaches: files you download from Hugging Face and the text you train or run it on.
- Not for: use as a ready chat assistant without more training.
On Hugging Face, 13 November 2026
- Task: text generation
- Library: transformers
- Parameters: 494M
- Weights: about 988 MB in BF16; about 247 MB at 4-bit (estimate)
- Formats: Safetensors
- Downloads in the last 30 days: 1,163,625
- Likes: 494
- License: apache-2.0
- Languages: en
- Quantized versions: 118
- Fine-tunes: 856
- Tags: transformers, safetensors, qwen2, text-generation, conversational
Links
- Model: https://huggingface.co/Qwen/Qwen2.5-0.5B
- Files: https://huggingface.co/Qwen/Qwen2.5-0.5B/tree/main
Get the model
Download its files with Hugging Face's command line:
pip install -U huggingface_hub
hf download Qwen/Qwen2.5-0.5BThen follow the usage in its model card. Nothing in this kit runs by itself.
README.md 5 KB
What’s new
Compare with similar
Listed Hugging Face models in the same category that share the most tags with this one.
| Product | Downloads, last 30 days | Likes | Parameters | License | Last change |
|---|---|---|---|---|---|
| Qwen2.5-0.5B This page | 1,163,625 | 494 | 494M | apache-2.0 | |
| Qwen2.5-7B-InstructShares qwen2 · text-generation · conversational | 6,364,238 | 2,494 | 7.6B | apache-2.0 | |
| Qwen2.5-1.5B-InstructShares qwen2 · text-generation · conversational | 5,181,515 | 941 | 1.5B | apache-2.0 | |
| Qwen2.5-0.5B-InstructShares qwen2 · text-generation · conversational | 4,292,272 | 743 | 494M | apache-2.0 | |
| Qwen2.5-Coder-14B-InstructShares qwen2 · text-generation · conversational | 1,860,327 | 214 | 14.8B | apache-2.0 | |
| Qwen2.5-7B-Instruct-AWQShares qwen2 · text-generation · conversational | 1,578,054 | 63 | 7.6B | apache-2.0 |
Reviews
What people say
Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search and on Trendshift. Mindket doesn’t write or check them. Open one to read it where it was posted.
spottedmarleyShow HN: IttyBittyAI – Qwen2.5:0.5B running on a 2017 Asus smartphoneRummaging through some boxes of old stuff, I found one of my old smart phones. It's an Asus Zenfone 4 Max (ZC554KL, Snapdragon 430, 8x Cortex-A53, 3 GB RAM, 32 GB storage) that I bought back in 2017 and so it's been sitting in a box for ~8 or 9 years. Still works! So I thought I'd give it a chance…2 pointsLuke OntarioSo. On the same benchmarks, Qwen2.5-0.5b scores 81%, and Drummer-0.6b scores 71%. That Qwen is a year and a half old, but this was trained with 1,659 times fewer tokens and even less electricity. To take it further requires a jump in computing that I do not want to invest in. Not sure what to do.7 likes · 3 repliesalebal123balShow HN: Dual YOLOv8n UAV Detection on RK3588S at 42 FPS Using NPUI built this while trying to understand how much of the RK3588S vision pipeline could be kept off the CPU. The main trick is not the YOLO model itself, but the pipeline structure: MIPI capture through the ISP, resize/color conversion through RGA, and YOLOv8n inference through all 3 NPU cores with…Zero WhiteSora AI — Voice Demo 🤖💬 I recently pushed the Improved Training Edition, upgrading the base model to Qwen/Qwen2.5-0.5B-Instruct and adding reliable Edge TTS voice generation. Check here: github.com/NivairaaWhit... #AI #BuildInPublic #MachineLearning #Dev #OpenSource #LocalLLM3 likesmolchanovartemI fine-tuned a 0.5B LLM to classify support tickets for $10/monthHi HN, I built a support ticket classifier using a fine-tuned Qwen2.5-0.5B model. It determines intent, category, urgency, sentiment, and routing — all in a single inference. *Why I built this:* A company needed to automate ticket routing but couldn't use cloud LLM APIs due to data privacy…@aichina.newsA 0.5B Qwen in ONNX under MIT is a sensible shape for Ryzen AI edge work — but the model card is a stub, with no benchmarks, prompt format or evidence it actually hits the NPU. Until AMD publishes a compatibility matrix, hard to see this beating a plain llama.cpp build on the same laptop.1 likeCheer2171Ollama Web SearchYour regular reminder that you don't need ollama to get a quick chat engine on the command line, you can just do this with pretty much any major model on huggingface: pip install transformers transformers chat Qwen/Qwen2.5-0.5B-Instruct@aichina.newsPre-split ONNX Qwen2.5-0.5B for Ryzen AI's NPU/GPU hybrid kills the export faff, and MIT makes embedding painless. But the card's a stub: no quantisation levels, no latency, no footprint. Can't back this for a Copilot+ build until AMD ships real numbers.robbruTiny-LLM – a course of serving LLM on Apple Silicon for systems engineersHey Nico, Very cool to hear your perspective in how you are using the small LLMs! I’ve been experimenting extensively with local LLM stacks on: • M1 Max (MLX native) • LM Studio (GLM, MLX, GGUFs) • Llama.cp (GGUFs) • n8n for orchestration + automation (multi-stage LLM workflows) My emerging use…@aichina.newsA 0.5B Qwen in int4 ONNX is the right shape for CPU inference — but the card is near-empty, no benchmarks, and the licence says only "other". Hard to justify shipping this over a llama.cpp quant you can actually audit. Anyone deployed it and got a real latency number?sunshine-oA hackable AI assistant using a single SQLite table and a handful of cron jobsThis is brilliant ! I am wondering, how powerful the AI model need to be to power this app? Would a selfhosted Llama-3.2-1B, Qwen2.5-0.5B or Qwen2.5-1.5B on a phone be enough?@aichina.newsA pre-converted 0.5B Qwen for Ascend NPUs under Apache 2.0 genuinely skips Huawei's toolchain. But no CANN version, no precision, no silicon list, and 4K context against upstream's 32K. Hard to see this saving time until the hardware list ships.
More from Mindket
See all from MindketDeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw
LibertAIDAI · Kit
Runs DeepSeek-V4.1-Flash-1017 as a 2.9 bits-per-weight EXL3 quantization, a 196 GiB copy for people serving large models on their own hardware. Perplexity stay…
New
MusicGen Small
facebook · Kit
Generates short music clips from a text description or an audio prompt, in the small checkpoint of facebook's MusicGen. It suits developers and music hobbyists…
New
Similar in Developer
See all in DeveloperLexnet
alpindale · Kit
Tags every character of C source code for syntax highlighting with a tiny neural network of about 984 weights, run through a small C program. For developers an…
New
FLUX.2 Klein LoRA Studio
prithivMLmods · Kit
Try a set of LoRAs for the FLUX.2 Klein image model in a Gradio app that runs in your browser or on your own computer. Made for people who make or test AI imag…
New
anaksor-ocr
Sokheng · Kit
Reads Khmer and English text from documents, receipts and photos of signs, using a detector for text lines and a recogniser for each line. Suits developers who…
New