AI skills and agents for developers

Qwen3-30B-A3B-GGUF by unsloth

unsloth · Kit

GGUF files of Qwen3-30B-A3B, quantized by unsloth, for running a local chat and reasoning model in llama.cpp, Ollama or Open WebUI, with a thinking mode you ca…

Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark

Sherlock-Comms · Kit

Serves the 122B Ornith 1.5 model through vLLM with DFlash2 speculative decoding, using an unquantized output head. Made for RTX Spark laptops with 128 GB of me…

MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF

sainez · Kit

Gives you a 9B coding and tool-calling model as one 3.64 GiB GGUF file, quantized at about 3.5 bits per weight for running on your own machine.

Qwen3.8-27B-NVFP4 by RadixArk

RadixArk · Kit

A pre-quantized NVFP4 copy of Qwen3.8-27B that reads text, images and video and answers in text, served with SGLang on NVIDIA Blackwell GPUs. Suits developers…

Raptor-0.6-4B-JANG_6M

OsaurusAI · Kit

Decides when to call a tool, when to answer directly and when to ask first, tuned from a 4B model and quantized for Apple Silicon Macs running Osaurus. For peo…

Luciole-8B-Instruct-1.1-GGUF

OpenLLM-France · Kit

Quantized GGUF copy of Luciole-8B-Instruct-1.1, an 8B text generation model for French and English, for running on your own computer. From LINAGORA's work in t…

Nanbeige4.3-33B-INT4

Nanbeige · Kit

INT4 quantized version of Nanbeige4.3-33B, a 33B model for coding and professional agent tasks, made with quantization-aware training. For developers who want…

Qwen3.8-27B Fable-5 Coding Distilled GGUF

mradermacher · Kit

GGUF quantized files of a 27.3B coding-distilled Qwen3.8 model, from 2-bit to 8-bit with an optional multi-modal supplement. They suit developers who run a cod…

OneJev1.6-4B-GGUF

mradermacher · Kit

Quantized GGUF copies of OneJev1.6-4B, from about 2 GB to 8.5 GB, for running the model on your own computer. The card does not say what the base model is for.

Qwen3.6-35B-A3B-2Bit-GSQ

ISTA-DASLab · Kit

A 2-bit quantized copy of Qwen3.6-35B-A3B made with GSQ, a Gumbel-Softmax method, that keeps most reasoning and coding ability at a much smaller memory footpri…

Qwen3-ASR-1.7B GGUF

cstr · Kit

Turns speech, singing and songs into text, or into English, with quantized GGUF files of Qwen3-ASR-1.7B for CrispASR. For people who want local transcription a…

Qwen 3.6 35B-A3B REAP-48 v2 GGUF

crucible-labs · Kit

Runs a pruned and quantized Qwen3.6-35B-A3B in an 8.78 GiB GGUF file for llama.cpp, closer to the unquantized weights than the first release. For people with l…

Underdog-Woof-4B-1.1

ConwayResearch · Kit

Generates text as a compact 4.2B model, post-trained from Qwen 3 4B to work better with Underdog's own setup and quantized to 4-bit for MLX. For people running…

MiniCPM5-2B-GGUF by bartowski

bartowski · Kit

Quantized GGUF copies of the MiniCPM5-2B text model in many sizes, made with llama.cpp and an importance matrix. For people running a small English and Chinese…

granite-4.2-3b-GGUF

bartowski · Kit

Quantized GGUF copies of IBM's granite-4.2-3b text model, in many sizes from full precision down to small 3-bit files. For people who run a small chat model on…

google_gemma-4-31B-it-GGUF

bartowski · Kit

Quantized GGUF files of Google's Gemma 4 31B instruction model, from 17.15 GB to 61.41 GB, made by bartowski with an importance matrix. Reads images and text.…

Qwen3.8-27B-Quark-AWQ-MXFP4

amd · Kit

Runs Qwen3.8-27B as a smaller MXFP4-quantized copy made with AMD Quark and AWQ, served through vLLM. Made for developers who want a compressed 27B vision-langu…

Granite Speech 5.0 TurboCTC 470M GGUF

vladers · Kit

Converts speech to text in seven languages, as quantized GGUF versions of IBM's Granite Speech 5.0 TurboCTC 470M, built for transcribe.cpp. For developers addi…

YuE2-GGUF

Serveurperso · Kit

Pre-quantized GGUF files for yue2.cpp, a C++ song generator that turns style tags and lyrics into stereo audio plus the score it composed. For musicians and ti…

GLM-5.3-Flash EXL3 3.5bpw

Mia-AiLab · Kit

Quantized copy of GLM-5.3-Flash in EXL3 format, 156 GB, with narrower routed experts, about 11 percent smaller than the same lab's 4-bit version. For people ru…

Slonik-27B-oQ8-mtp

kakoka · Kit

Helps write and change PostgreSQL source code with a 27B coding model, 8-bit quantized to run locally on Apple Silicon Macs through oMLX. For PostgreSQL develo…

K2-Horizon-7B-GGUF

IFM · Kit

Quantized GGUF copies of IFM's K2-Horizon-7B text model for running locally in llama.cpp, from BF16 down to 4-bit, with a 512K-token context window.

Meta-Llama-3.1-8B-Instruct-GGUF

bartowski · Kit

Quantized GGUF files of Meta's Llama 3.1 8B Instruct for local chat, such as 4.92 GB for Q4_K_M and 8.54 GB for Q8_0. Made by bartowski with an importance matr…

Ornith-1.5-122B-NVFP4

ornith-ai · Kit

Gives you an NVFP4 quantized copy of Ornith-1.5-122B, a 122B mixture-of-experts text model meant for single-GPU use and strong in coding tests. For developers…

AREX-2 GGUF

mradermacher · Kit

GGUF quantized files of BAAI's AREX-2, a 26.9B model, from 2-bit to 8-bit with an optional multi-modal supplement. They suit people who run a large model local…

Kromoss GGUF

molbal · Kit

GGUF quantized copies of Kromoss, a 1024 px text-to-image model fine-tuned from MOSS-Image-Base, for ComfyUI. For artists who want to generate images on a GPU…

RelayVSR X4 Quants

MATLOWAI · Kit

Pre-quantized Wan checkpoints for RelayVSR's 4x video upscaler, so it runs in ComfyUI on 16 GB and 24 GB graphics cards at 2560x1408 output, for video editors…

Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF

IsValorum · Kit

Runs the 35B Axion-1.0 model locally in llama.cpp from one compressed GGUF file with mixed precision. A quantized release by IsValorum for people who want a la…

Qwen3.8-27B-Randean-MXFP4

flechin · Kit

A 27B reasoning model tuned to answer plainly instead of hedging, quantized to MXFP4 so vLLM can serve it on a single 32 GB GPU, for developers who run local r…

Qwen3.6-35B-A3B GGUF (ShapeLearn Quantized)

byteshape · Kit

Quantized GGUF versions of Qwen3.6-35B-A3B that read images and text, with CPU and GPU variants from about 9 GB, made with ByteShape's ShapeLearn. For develope…

Hermes-3-Llama-3.1-8B-GGUF

NousResearch · Kit

Runs Hermes 3, an 8B general-purpose chat model from NousResearch, as a quantized GGUF file for llama.cpp. Suits people who want a local assistant with functio…

Qwen3.8-27B-exl3

turboderp · Kit

EXL3 quantized copies of Qwen3.8-27B in many bits-per-weight sizes, from 1.40 to 6.00. For people running the 27B model locally with ExLlamaV3 on limited GPU m…

DeepSeek-V4.1-Flash-1017-NVFP4

nvidia · Kit

NVIDIA's NVFP4 quantized copy of DeepSeek AI's DeepSeek-V4.1-Flash-1017, a huge multimodal mixture-of-experts model for NVIDIA Blackwell GPUs.

Qwen3.5 for ComfyUI

Comfy-Org · Kit

Repackaged Qwen3.5 2B, 4B and 9B model files for ComfyUI's text encoder folder, in bf16 and two quantized styles for the 9B size. For ComfyUI users who need a…

gemma-4-12b-it-GGUF

unsloth · Kit

Unsloth's GGUF quantized copy of Google's instruction-tuned Gemma 4 12B, a model that takes text, images, audio and video and writes text, for running locally.

Qwen3.6-35B-A3B-NVFP4

nvidia · Kit

A pre-quantized NVFP4 copy of Alibaba's Qwen3.6-35B-A3B, made by NVIDIA with Model Optimizer. It takes text, images and video, and runs on vLLM with Hopper or…

GLM-5.3-Flash-NVFP4

nvidia · Kit

Gives you NVIDIA's NVFP4 quantized copy of ZAI's GLM-5.3-Flash, a multimodal model for reasoning, coding and agent tasks. Meant for developers who deploy on Bl…

Qwen3-8B-GGUF

Qwen · Kit

Runs Qwen3-8B as quantized GGUF files in llama.cpp or Ollama, with a thinking mode for math, code and logic and a plain mode for chat. For people who want a lo…

whisper.cpp

ggerganov · Kit

OpenAI's Whisper speech recognition models converted to ggml files for whisper.cpp, in sizes from tiny to large-v3-turbo, with smaller quantized copies.

Qwen3.5-4B-GGUF

unsloth · Kit

Unsloth's GGUF quantized copy of Qwen3.5-4B, a small model that reads images and text and answers in text, for people running models locally.

Qwen3.5-9B-GGUF

unsloth · Kit

Unsloth's GGUF quantized copy of Qwen3.5-9B, a mid-size model that reads images and text and answers in text, for people running models locally.

Qwen3.8-27B-FP8

Qwen · Kit

Handles text, images and video with FP8-quantized weights of Qwen3.8-27B, a 27B dense model with adjustable thinking. For developers serving it with transforme…

Ripple-65B-Preview-GGUF

AtomicChat · Kit

GGUF quantized files of the 65B Ripple Preview agentic-coding model, for running locally with llama.cpp. For people who run large open models on their own hard…

Qwen3.8-27B GGUF (ShapeLearn Quantized)

byteshape · Kit

Runs Qwen3.8-27B as GGUF files quantized by ByteShape's ShapeLearn, in five GPU-sized options with vision input and MTP speculative decoding in llama.cpp. For…

DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

ISTA-DASLab · Kit

GGUF versions of DeepSeek-V4.1-Flash-1017 at three small sizes, with each tensor quantized at its own precision. For people running a huge mixture-of-experts m…

Qwen3.8-Flash-Next-GSQ-RCO-GGUF

ISTA-DASLab · Kit

GGUF versions of Qwen3.8-Flash-Next at four sizes, with each tensor quantized at its own precision and a vision projector for image input. For people running a…

DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF

ISTA-DASLab · Kit

An expert pruned and quantized GGUF of DeepSeek-V4.1-Flash-1017 from ISTA-DASLab, aimed at code and agent work, with transformer weights of 72.1 GB. Suits deve…

Qwen3.8-27B-GSQ-RCO-GGUF

ISTA-DASLab · Kit

ISTA-DASLab's GGUF versions of Qwen3.8-27B in four sizes, each tensor quantized at its own precision, with a vision projector for images. Suits people who run…

Local Model Explorer

LocalLLaMA · Kit

Shows which GGUF quantized models fit your graphics card, Mac or CPU, for people who run language models on their own machine and want to pick a file that work…

GGUF Model VRAM Calculator

DavidAU · Kit

Estimates how much video memory a GGUF model needs, in a static page that runs in your browser. Suits people choosing a quantized model that fits their graphic…