AI projects from Hugging Face

Swift 1.5 Ornith 9B GGUF

ukisai · Kit

Runs Swift 1.5 Ornith 9B, a reasoning model tuned to think with fewer tokens, as GGUF quantizations for llama.cpp. For people who run a 9B reasoning model on t…

Ornith-1.5-9B-GGUF by AtomicChat

AtomicChat · Kit

GGUF builds of Ornith 1.5 9B, a model that reads images and text, in sizes from 2.81 GB to 17.9 GB plus a vision projector. Pick the file that fits your graphi…

Groot-19B-A3B-TTQ-GGUF

TokenTinkerer · Kit

Offers eleven GGUF sizes of the Groot-19B-A3B reasoning model for llama.cpp, from 7.88 to 18.59 GiB, plus a vision projector. Made for people who run the model…

Qwen3.8-27B-DFlash2-GGUF by z-lab

z-lab · Kit

A draft model in GGUF form that speeds up Qwen3.8-27B by proposing token blocks for the main model to verify. For people who run that model locally with llama.…

Qwen3.5-4B-MTP-GGUF

unsloth · Kit

Unsloth's GGUF files of Qwen3.5-4B with multi-token prediction, which lets llama.cpp use speculative decoding for roughly 1.5 to 2 times faster generation. For…

Qwen3-30B-A3B-GGUF by unsloth

unsloth · Kit

GGUF files of Qwen3-30B-A3B, quantized by unsloth, for running a local chat and reasoning model in llama.cpp, Ollama or Open WebUI, with a thinking mode you ca…

Clef-LoRA-for-Qwen3.8-27B-GGUF

trevest · Kit

Adds Cloudflare Clef's decision head to a Qwen3.8-27B chat model, so one llama.cpp server on a 24 GB GPU both answers typed questions and chats. It is a LoRA a…

Hy-MT2-1.8B-1.25Bit-GGUF

tencent · Kit

Translates between 33 languages on your own device with a 1.8B model squeezed to 1.25 bits, about 440 MB as GGUF files for llama.cpp. For developers adding off…

BeeLLM

schizophyllume · Kit

Offers a family of tiny GGUF language models that talk in bee buzzes, from buzz-only bees to mood-tagged ones and linked queen and worker hives. For hobbyists…

Sarvam-105B GGUF

sarvamai · Kit

The GGUF version of Sarvam-105B, a large mixture-of-experts model with a focus on Indian languages, split into 9 shards at 4-bit for llama.cpp. For people with…

MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF

sainez · Kit

Gives you a 9B coding and tool-calling model as one 3.64 GiB GGUF file, quantized at about 3.5 bits per weight for running on your own machine.

Unlimited-OCR-2-GGUF

RX5950XT · Kit

GGUF files of the Unlimited-OCR-2 document OCR model in BF16, Q8_0 and Q4_K_M, with a vision projector. They run only in a llama.cpp build with the repository'…

Qwen3-30B-A3B-GGUF

Qwen · Kit

A 30.5B mixture-of-experts chat model in GGUF format with only 3.3B parameters active per token, runnable in llama.cpp or ollama. It has thinking and plain mod…

Qwen3-0.6B-GGUF

Qwen · Kit

A small 0.6B chat model in GGUF format that runs locally in llama.cpp or ollama and can switch between thinking and plain replies. Suits people with modest har…

Qwen2.5-Coder-32B-Instruct-GGUF

Qwen · Kit

GGUF copy of a 32B code chat model that runs locally with llama.cpp. It writes, explains and fixes code from plain requests, for developers who want to run it…

pplx-decider-flash-gguf-v1.2

perplexity-ai · Kit

Sorts a request into the choices you set, from text and images, using a Q8_0 GGUF copy of a small decision model. For developers who can run a protocol v3 runt…

MiniCPM-OCR 1.0 GGUF

openbmb · Kit

Reads English and Chinese documents, including PDFs, into text, tables and formulas, in GGUF files for llama.cpp. For people who run MiniCPM-OCR 1.0 on their o…

SenseNova-6.8-Code2Video GSQ-RCO GGUF

OliviaRossi · Kit

Runs a 26.9B parameter SenseNova-6.8-Code2Video model as a mixed precision GGUF file, in a terminal chat or a local server. For people who want that model on t…

nomic-embed-text-v1.5-GGUF

nomic-ai · Kit

Turns sentences into vectors on your own machine through llama.cpp, using compact GGUF files, for search and RAG questions. For developers who want local text…

Qwen3.8-Flash-Next-DFlash2-GGUF

nazeshinjite · Kit

GGUF draft model that speeds up Qwen3.8-Flash-Next in llama.cpp through speculative decoding, in 334 MiB and 622 MiB files. It is not a standalone model, and i…

Muse Glimmer 30B GGUF by meta-models

meta-models · Kit

GGUF builds of a 30-billion-parameter agentic model for llama.cpp, with an optional image encoder and a speculative decoding drafter. They suit people who run…

dale-9b (GGUF)

manuelgscom · Kit

Turns a Spanish or English request into one Linux bash command, returned as a small JSON object. For terminal users and developers who want a local command hel…

askmdb-9b-GGUF

manuelgscom · Kit

Writes a read-only MongoDB aggregate pipeline from a plain Spanish or English question. A GGUF copy of a Qwen3.5-9B fine-tune that runs locally with llama.cpp,…

humanizer-GGUF

jialinyyzz · Kit

Rewrites AI-written drafts in English and Chinese so they read like a person wrote them, keeping numbers, dates, names and quotes. A 12B model in GGUF files fo…

Qwopus3.8-27B-Flash-GGUF

Jackrong · Kit

Cuts reasoning time and cost for long agent tasks, as a GGUF build of a Qwen3.8-27B fine-tune. Made for people who run agent loops on their own machine and wan…

Qwen3.8-Flash-Coder APEX-I-MiniPlus V2.1 GGUF

IsValorum · Kit

An English-only coding model in GGUF, a 42.6B mixture-of-experts quant of about 21.8 GB, for code completion, refactoring and agentic tool calls in llama.cpp o…

K2-Horizon-3.7B-GGUF

IFM · Kit

GGUF files of the 3.7B K2-Horizon text model in six precision levels, from BF16 to Q4_K_M, for running with llama.cpp. The model has a 512K-token context windo…

ibnsina-1.5b

ibnsina-llm · Kit

A small Persian language model for writing, summarizing, translating and chatting offline, trained from scratch mostly on Persian text. Runs through ollama, LM…

granite-4.2-8b GGUF

ibm-granite · Kit

Holds GGUF copies of IBM's Granite 4.2 8B model in several quantization levels, ready for llama.cpp. For people who want to run the model locally and read its…

Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF

hesamation · Kit

Runs a reasoning-tuned Qwen3.6-35B-A3B on your own machine as GGUF files in four quant sizes, trained on Claude Opus 4.6 style reasoning data. For people who w…

phonon-31m English for llama.cpp

gradium · Kit

Turns English text into speech from the command line or a local server, as a GGUF copy of the Phonon 31M model for llama.cpp. It suits developers who already r…

Qwen2.5-Coder-1.5B Q8_0 GGUF

ggml-org · Kit

Runs Qwen2.5-Coder 1.5B on your own machine as a Q8_0 GGUF file through llama.cpp, from the command line or as a local server. It suits developers who want a s…

Clef-GGUF

ggml-org · Kit

GGUF files of Cloudflare's Clef decision model, which takes image and text input. Served with llama.cpp tools and called through a decision API path. For devel…

theOG-50M

DrunkkToys · Kit

Writes a Python function from a plain English request. A 50M parameter code model trained from scratch on one laptop, small enough to run offline in Ollama, LM…

Qwen 3.6 35B-A3B REAP-48 v2 GGUF

crucible-labs · Kit

Runs a pruned and quantized Qwen3.6-35B-A3B in an 8.78 GiB GGUF file for llama.cpp, closer to the unquantized weights than the first release. For people with l…

t5-v1_1-xxl-encoder-gguf

city96 · Kit

A GGUF conversion of Google's T5 v1.1 XXL text encoder, for turning text into embeddings in llama.cpp or feeding image generation models through ComfyUI-GGUF.…

Paraphase-2B

chadstthomas · Kit

A small offline model that rewrites stiff, AI-sounding English email drafts into plain prose while keeping the facts. Runs on a 4 to 6 GB device through llama.…

Qwen3.5-9B GGUF (ShapeLearn)

byteshape · Kit

GGUF copies of Qwen3.5-9B, a model that reads text and images, compressed with ByteShape's ShapeLearn for llama.cpp. Variants for GPUs and CPUs, from about 3 G…

North-Mini-Code-1.0 GGUF

byteshape · Kit

GGUF copies of the North-Mini-Code-1.0 coding model, compressed with ByteShape's ShapeLearn for GPU use in llama.cpp. Four sizes from 12.1 GB to 21.5 GB, for d…

MiniCPM5-2B-GGUF by bartowski

bartowski · Kit

Quantized GGUF copies of the MiniCPM5-2B text model in many sizes, made with llama.cpp and an importance matrix. For people running a small English and Chinese…

google_gemma-4-31B-it-GGUF

bartowski · Kit

Quantized GGUF files of Google's Gemma 4 31B instruction model, from 17.15 GB to 61.41 GB, made by bartowski with an importance matrix. Reads images and text.…

Cloudflare clef GGUF

bartowski · Kit

Static GGUF quantizations of Cloudflare's clef decision model, which scores every answer option to typed questions about a state in one pass instead of writing…

Surya 2.5 GGUF

datalab-to · Kit

Reads text, tables and layout from document images and returns it as text. GGUF builds of the Surya 2.5 OCR model for llama.cpp, for people who run document OC…

Hy-MT2-7B-GGUF

tencent · Kit

Translates text among 33 languages and follows translation instructions, as a GGUF copy of Tencent's 7B model for llama.cpp. For developers who run translation…

Hy-MT2-1.8B-GGUF

tencent · Kit

Translates text among 33 languages and obeys translation instructions, as a 1.8B model in GGUF format for llama.cpp. For developers who want a small local tran…

Qwen3.8-27B Agention Precision GGUF

agentionai · Kit

Packs Qwen3.8-27B as llama.cpp GGUF files in eight sizes from 8.95 to 16.35 GiB, plus a vision projector. The author reports less drift from the BF16 model tha…

Qwen3.8-Flash-Next-125B-UltraLite-37GiB-GGUF

0xKitkat · Kit

Squeezes the Qwen3.8-Flash-Next mixture-of-experts model into about 37 GiB of text-only GGUF files for experiments. It needs a patched llama.cpp build that com…

OpenBonsai Qwen3.5-9B Ternary

webbrain-one · Kit

Runs Qwen3.5-9B squeezed to ternary weights in a roughly 2 GB GGUF file, for CPU use in two llama.cpp variants. A research preview for people who experiment wi…

Qwen3.5-0.8B-MTP-GGUF

unsloth · Kit

Runs the small Qwen3.5 0.8B vision-language model in llama.cpp with multi-token prediction for faster generation, as Unsloth's GGUF files. For developers testi…

gemma-4-31B-it-qat-GGUF

unsloth · Kit

Runs Google's Gemma 4 31B instruction model from Unsloth's 4-bit QAT GGUF files, with an included drafter for faster speculative decoding in llama.cpp. For peo…