AI projects from Hugging Face

DeepSeek-V4.1-Flash-1017-EXL3-franken-v100

philbert440 · Kit

Runs DeepSeek V4.1 Flash at its full 1M-token context on eight Tesla V100 32 GB cards, using an EXL3 expert pack and an SM70 vLLM build. For owners of such ser…

gemma-4-31b-it-exl3-optimized

Dino0040 · Kit

Holds the measurements and recipes for building an optimized EXL3 quant of Gemma 4 31B-it at any bitrate. For people who quantize models with exllamav3 and wan…

GLM-5.3-Flash EXL3 3.05bpw

albucino · Kit

Runs GLM-5.3-Flash locally on two RTX 3090 cards with 128 GB of RAM and a 262,144-token context, using a verified 3.05 bpw EXL3 quantization. For people hostin…

DeepSeek-V4.1-Flash-1017-exl3

turboderp · Kit

Runs a compressed DeepSeek-V4.1-Flash-1017 in ExLlamaV3 at 1.68, 2.25 or 2.68 bits per weight. For people who run large models on their own hardware and want l…

Qwen3.8-27B-exl3

turboderp · Kit

EXL3 quantized copies of Qwen3.8-27B in many bits-per-weight sizes, from 1.40 to 6.00. For people running the 27B model locally with ExLlamaV3 on limited GPU m…

DeepSeek-V4.1-Flash-1017 EXL3 EAQ v1 2.9 bpw

Mia-AiLab · Kit

A 2.9 bits-per-weight EXL3 quantization of DeepSeek-V4.1-Flash-1017 that reads text and images. It cuts the model to about 210 GB for exllamav3 runtimes, and t…

DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000

diffbot · Kit

Serves a 3-bit compressed copy of DeepSeek-V4.1-Flash-1017 on two 96 GB RTX PRO 6000 Blackwell cards with vLLM, for developers who host a large coding and agen…