Qwen2.5-Coder-14B-Instruct-GGUF

Qwen2.5-Coder-14B-Instruct-GGUF

Gives you the Qwen2.5-Coder 14B instruct model as GGUF files in eight quantization sizes, for coding help with llama.cpp. For developers who want a stronger local code assistant on a good GPU or a large-memory machine.

Made by Qwen on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds the instruction-tuned Qwen2.5-Coder 14B model in GGUF format, for llama.cpp. Qwen2.5-Coder is the code-focused Qwen family, formerly called CodeQwen, in sizes from 0.5B to 32B.

The card says the series improves code generation, code reasoning and code fixing over CodeQwen1.5, and was trained on 5.5 trillion tokens. It also targets code agents while keeping math and general skills.

Who it suits: It suits developers who want a mid-size code model on their own hardware and who can handle a file of several GB. A 4-bit copy is estimated at about 7.4 GB.

Industry | Who uses it | How they use it | Example
Software | Developers | Draft and fix code with a local model | Ask for a function and tests
Engineering | Teams | Keep source code on their own machines | Review a snippet privately
Agent building | Tool builders | Back a code agent with a local model | Run a repair loop
Local AI | Enthusiasts | Run a 14B coder on a strong GPU | Try different quant sizes

If you only have a small laptop, you can skip it and look at the smaller Qwen2.5-Coder sizes.

Where to use it: The files run in llama.cpp on Windows, macOS or Linux. The card suggests the conversation mode for a chatbot-style experience.

• llama-cli from llama.cpp
• A GPU with enough memory, or a machine with plenty of RAM
• Any tool that loads GGUF files

Use case | Who | What they get back
Code generation | Developers | Code written from a plain request
Code fixing | Engineers | A corrected snippet and a short explanation
Code agents | Builders | Text output for an agent loop

How to set it up: You need Python with pip and a working llama.cpp. Large quant files are split into parts, so you download all parts and may need to merge them before use.

• Python with pip
• llama.cpp installed
• Several GB of free disk space for your quant

1. Install the Hugging Face hub package.
2. Download all parts of the quant you want.
3. Merge the parts with llama-gguf-split.

pip install -U huggingface_hub

huggingface-cli download Qwen/Qwen2.5-Coder-14B-Instruct-GGUF --include "qwen2.5-coder-14b-instruct-q5_k_m*.gguf" --local-dir . --local-dir-use-symlinks False

./llama-gguf-split --merge qwen2.5-coder-14b-instruct-q5_k_m-00001-of-00002.gguf qwen2.5-coder-14b-instruct-q5_k_m.gguf

How to use it: You start llama-cli in conversation mode with the path to the merged GGUF file, then type your coding request. The command also sets the assistant's role, the GPU layers and the reply length.

./llama-cli -m <gguf-file-path> \
    -co -cnv -p "You are Qwen, created by Alibaba Cloud. You are a helpful assistant." \
    -fa -ngl 80 -n 512

Tool | What it does
huggingface-cli download | Fetches every part of the quant you name
llama-gguf-split --merge | Joins split parts into one GGUF file
llama-cli | Chats with the model in the terminal

Example: You merge the q5_k_m parts, start the chat command with the merged file and ask for a function that parses a date. The model replies in the terminal with code and a short note.

For newcomers and experts: If you are new, try a smaller quant or the 1.5B model first to check your setup, then move up to this one. The merge step is only needed for split files.

If you know the field, the model has 48 layers and grouped-query attention with 40 query heads and 8 key-value heads. Context is 32,768 tokens in the GGUF files, and the card says only vLLM supports YARN for longer inputs.

Download all parts of a quant
Merge them if they are split
Run llama-cli in conversation mode
Type a coding request

Good to know: For inputs up to 131,072 tokens, the card sends you to the non-GGUF models. The quant sizes run from a 2-bit q2_K up to an 8-bit q8_0.

• License: apache-2.0
• Cost to run: none named
• Data it reaches: files you download, and the code and messages you type
• Not for: long inputs beyond 32,768 tokens in GGUF, since YARN needs vLLM

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 14.8B
• Weights: about 7.4 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 202,738
• Likes: 280
• License: apache-2.0
• Languages: en
• Based on: Qwen/Qwen2.5-Coder-14B-Instruct
• Quantized versions: 1
• Tags: transformers, gguf, code, codeqwen, chat, qwen, qwen-coder, text-generation, conversational

Links

• Model: https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct-GGUF
• Files: https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Qwen/Qwen2.5-Coder-14B-Instruct-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.