AI projects from Hugging Face

Qwen2.5-7B-Instruct-GGUF by Qwen

Qwen · Kit

Qwen2.5-7B-Instruct as GGUF files in eight quantization sizes, to chat with locally through llama.cpp. For people who want a capable 7.6B model on their own co…

Meta-Llama-3.1-8B-Instruct-AWQ-INT4

hugging-quants · Kit

Runs Meta's Llama 3.1 8B Instruct chat model in 4-bit AWQ form, so it fits on a GPU with about 4 GiB of free memory for the weights. For developers who want a…

AsteroRank-Lepton-GGUF

Asteromorph · Kit

Ranks search results by relevance with a reranker for STEM and general search, shipped as GGUF files for llama.cpp. For developers and researchers who want loc…

Swift 1.5 Ornith 9B GGUF

ukisai · Kit

Runs Swift 1.5 Ornith 9B, a reasoning model tuned to think with fewer tokens, as GGUF quantizations for llama.cpp. For people who run a 9B reasoning model on t…

Groot-19B-A3B-TTQ-GGUF

TokenTinkerer · Kit

Offers eleven GGUF sizes of the Groot-19B-A3B reasoning model for llama.cpp, from 7.88 to 18.59 GiB, plus a vision projector. Made for people who run the model…

zeta-2.1

zed-industries · Kit

Predicts how the code around your cursor should be rewritten next, from the file context and your recent edits. A model from Zed Industries for people building…

Qwen3.8-27B-DFlash2-GGUF by z-lab

z-lab · Kit

A draft model in GGUF form that speeds up Qwen3.8-27B by proposing token blocks for the main model to verify. For people who run that model locally with llama.…

LocalAlign_llama3.1_8B

yuyangGong · Kit

A LoRA adapter for Llama-3.1-8B-Instruct that trains it to follow trusted instructions and ignore commands hidden in external data. A prompt injection defense…

Qwen3.5-4B-MTP-GGUF

unsloth · Kit

Unsloth's GGUF files of Qwen3.5-4B with multi-token prediction, which lets llama.cpp use speculative decoding for roughly 1.5 to 2 times faster generation. For…

Qwen3-30B-A3B-GGUF by unsloth

unsloth · Kit

GGUF files of Qwen3-30B-A3B, quantized by unsloth, for running a local chat and reasoning model in llama.cpp, Ollama or Open WebUI, with a thinking mode you ca…

Llama-3.2-3B-Instruct-GGUF by unsloth

unsloth · Kit

Chats, summarizes and answers questions with Meta's compact Llama 3.2 3B instruct model, in GGUF files by Unsloth from 2-bit up to 16-bit. Small enough for ord…

Clef-LoRA-for-Qwen3.8-27B-GGUF

trevest · Kit

Adds Cloudflare Clef's decision head to a Qwen3.8-27B chat model, so one llama.cpp server on a 24 GB GPU both answers typed questions and chats. It is a LoRA a…

Hy-MT2-1.8B-1.25Bit-GGUF

tencent · Kit

Translates between 33 languages on your own device with a 1.8B model squeezed to 1.25 bits, about 440 MB as GGUF files for llama.cpp. For developers adding off…

Sarvam-105B GGUF

sarvamai · Kit

The GGUF version of Sarvam-105B, a large mixture-of-experts model with a focus on Indian languages, split into 9 shards at 4-bit for llama.cpp. For people with…

Llama-3.1-8B-Stheno-v3.4

Sao10K · Kit

Writes stories and plays characters in multi-turn chat. An 8B Llama 3.1 fine-tune for creative writing and roleplay, aimed at hobbyists who run models on their…

Unlimited-OCR-2-GGUF

RX5950XT · Kit

GGUF files of the Unlimited-OCR-2 document OCR model in BF16, Q8_0 and Q4_K_M, with a vision projector. They run only in a llama.cpp build with the repository'…

Qwen3-30B-A3B-GGUF

Qwen · Kit

A 30.5B mixture-of-experts chat model in GGUF format with only 3.3B parameters active per token, runnable in llama.cpp or ollama. It has thinking and plain mod…

Qwen3-0.6B-GGUF

Qwen · Kit

A small 0.6B chat model in GGUF format that runs locally in llama.cpp or ollama and can switch between thinking and plain replies. Suits people with modest har…

Qwen2.5-Coder-32B-Instruct-GGUF

Qwen · Kit

GGUF copy of a 32B code chat model that runs locally with llama.cpp. It writes, explains and fixes code from plain requests, for developers who want to run it…

MiniCPM-OCR 1.0 GGUF

openbmb · Kit

Reads English and Chinese documents, including PDFs, into text, tables and formulas, in GGUF files for llama.cpp. For people who run MiniCPM-OCR 1.0 on their o…

nomic-embed-text-v1.5-GGUF

nomic-ai · Kit

Turns sentences into vectors on your own machine through llama.cpp, using compact GGUF files, for search and RAG questions. For developers who want local text…

Qwen3.8-Flash-Next-DFlash2-GGUF

nazeshinjite · Kit

GGUF draft model that speeds up Qwen3.8-Flash-Next in llama.cpp through speculative decoding, in 334 MiB and 622 MiB files. It is not a standalone model, and i…

Nanbeige4.3-33B-INT4

Nanbeige · Kit

INT4 quantized version of Nanbeige4.3-33B, a 33B model for coding and professional agent tasks, made with quantization-aware training. For developers who want…

Agaram 2.5B

murai-labs · Kit

Chats in Tamil script, romanized Tamil and English, and translates between Tamil and English. A 2.5B model built from scratch by Murai Labs, for Tamil speakers…

Muse Glimmer 30B GGUF by meta-models

meta-models · Kit

GGUF builds of a 30-billion-parameter agentic model for llama.cpp, with an optional image encoder and a speculative decoding drafter. They suit people who run…

Maya-1

maya-research · Kit

An open-weight text-to-speech model where you describe the voice in plain words and add inline emotion cues. It suits developers and creators who need expressi…

askmdb-9b-GGUF

manuelgscom · Kit

Writes a read-only MongoDB aggregate pipeline from a plain Spanish or English question. A GGUF copy of a Qwen3.5-9B fine-tune that runs locally with llama.cpp,…

ColGrep-Agent-2B

lightonai · Kit

A 2.5B-parameter model that finds the files and line ranges relevant to a bug report or question about a code repository, by searching and reading the repo. Ru…

tinyllamas

karpathy · Kit

Provides a series of small language models in the Llama 2 design, trained on the TinyStories dataset and made for Karpathy's llama2.c project. For developers w…

Qwen3.8-Flash-Coder APEX-I-MiniPlus V2.1 GGUF

IsValorum · Kit

An English-only coding model in GGUF, a 42.6B mixture-of-experts quant of about 21.8 GB, for code completion, refactoring and agentic tool calls in llama.cpp o…

K2-Horizon-3.7B-GGUF

IFM · Kit

GGUF files of the 3.7B K2-Horizon text model in six precision levels, from BF16 to Q4_K_M, for running with llama.cpp. The model has a 512K-token context windo…

ibnsina-1.5b

ibnsina-llm · Kit

A small Persian language model for writing, summarizing, translating and chatting offline, trained from scratch mostly on Persian text. Runs through ollama, LM…

granite-4.2-8b GGUF

ibm-granite · Kit

Holds GGUF copies of IBM's Granite 4.2 8B model in several quantization levels, ready for llama.cpp. For people who want to run the model locally and read its…

ALLaM-7B-Instruct-preview

humain-ai · Kit

Answers questions and follows instructions in Arabic and English. A 7B model trained from scratch by the National Center for Artificial Intelligence at SDAIA,…

phonon-31m English for llama.cpp

gradium · Kit

Turns English text into speech from the command line or a local server, as a GGUF copy of the Phonon 31M model for llama.cpp. It suits developers who already r…

Qwen2.5-Coder-1.5B Q8_0 GGUF

ggml-org · Kit

Runs Qwen2.5-Coder 1.5B on your own machine as a Q8_0 GGUF file through llama.cpp, from the command line or as a local server. It suits developers who want a s…

embeddinggemma-2 GGUF

ggml-org · Kit

Runs a GGUF copy of Google's embeddinggemma-2, a 271M feature extraction model, with the llama command or the Llama app. For developers who want a small text e…

Clef-GGUF

ggml-org · Kit

GGUF files of Cloudflare's Clef decision model, which takes image and text input. Served with llama.cpp tools and called through a decision API path. For devel…

theOG-50M

DrunkkToys · Kit

Writes a Python function from a plain English request. A 50M parameter code model trained from scratch on one laptop, small enough to run offline in Ollama, LM…

gemma-4-31b-it-exl3-optimized

Dino0040 · Kit

Holds the measurements and recipes for building an optimized EXL3 quant of Gemma 4 31B-it at any bitrate. For people who quantize models with exllamav3 and wan…

Kiyo-65M

DedeProGames · Kit

Continues English text with a tiny 65M-parameter decoder-only model trained on 80B tokens. For learners and researchers who want a small transformers model to…

Qwen 3.6 35B-A3B REAP-48 v2 GGUF

crucible-labs · Kit

Runs a pruned and quantized Qwen3.6-35B-A3B in an 8.78 GiB GGUF file for llama.cpp, closer to the unquantized weights than the first release. For people with l…

t5-v1_1-xxl-encoder-gguf

city96 · Kit

A GGUF conversion of Google's T5 v1.1 XXL text encoder, for turning text into embeddings in llama.cpp or feeding image generation models through ComfyUI-GGUF.…

Paraphase-2B

chadstthomas · Kit

A small offline model that rewrites stiff, AI-sounding English email drafts into plain prose while keeping the facts. Runs on a 4 to 6 GB device through llama.…

Qwen3.5-9B GGUF (ShapeLearn)

byteshape · Kit

GGUF copies of Qwen3.5-9B, a model that reads text and images, compressed with ByteShape's ShapeLearn for llama.cpp. Variants for GPUs and CPUs, from about 3 G…

North-Mini-Code-1.0 GGUF

byteshape · Kit

GGUF copies of the North-Mini-Code-1.0 coding model, compressed with ByteShape's ShapeLearn for GPU use in llama.cpp. Four sizes from 12.1 GB to 21.5 GB, for d…

SalamandraTA-7b-instruct

BSC-LT · Kit

A 7.8B translation model for 40 languages and 3 varieties, covering sentence, paragraph and document translation plus HTML or XML text. For developers and rese…

MiniCPM5-2B-GGUF by bartowski

bartowski · Kit

Quantized GGUF copies of the MiniCPM5-2B text model in many sizes, made with llama.cpp and an importance matrix. For people running a small English and Chinese…

Llama-3.2-1B-Instruct GGUF

bartowski · Kit

GGUF quantizations of Llama-3.2-1B-Instruct, a 1.2B text model in eight languages, with the 4-bit file near 0.8 GB. For people who run a tiny local model.

google_gemma-4-31B-it-GGUF

bartowski · Kit

Quantized GGUF files of Google's Gemma 4 31B instruction model, from 17.15 GB to 61.41 GB, made by bartowski with an importance matrix. Reads images and text.…