Mindket

GLM-4.7-Flash-GGUF by axteam

Newversion 1 · updated

Overview

Four GGUF quantizations of GLM-4.7-Flash for running a 31B-class chat model locally in llama.cpp, from 18.9 GB to 33.2 GB. For people who host models on their own hardware.

GLM-4.7-Flash-GGUF by axteam is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on . Download it on this page, a zip of its files. Nothing in it runs by itself.

At a glance

You need
  • llama.cpp build b10251 or newer
  • Memory for one GGUF file plus the KV cache
  • Python and the Hugging Face hub tool, only if you download files by hand
Runs in
  • llama.cpp: the llama-server and llama-cli programs
  • Ollama: the card gives a run command but says it was not tested there
  • LM Studio: search for the repository name in its model browser, also untested by the author
Start with
pip install -U huggingface_hub
hf download axteam/GLM-4.7-Flash-GGUF GLM-4.7-Flash-Q4_K_M.gguf --local-dir .

Platform

ServerRuns on it

About

What it is

This repository holds GGUF files of GLM-4.7-Flash, a text generation model from zai-org, made by axteam. The files come from post-training quantization with llama.cpp, with no retraining, and each one has the original chat template built in. You pick a size that fits your hardware.

Who it suits

People who run chat models on their own computers and want a ready GGUF file instead of converting weights themselves.

IndustryWho uses itHow they use itExample
SoftwareDevelopersServe an OpenAI-compatible API from a local machineA local endpoint for an app under development
ResearchResearchersTest a large chat model without sending text to a hosted serviceAsk a set of questions on a private server
IT teamsSelf-hostersChoose a quant that fits the memory they haveQ4_K_M on a machine with enough room

Skip it if you have little disk or memory, because the smallest file listed is 18.9 GB.

Where to use it

  • llama.cpp: the llama-server and llama-cli programs
  • Ollama: the card gives a run command but says it was not tested there
  • LM Studio: search for the repository name in its model browser, also untested by the author
Use caseWhoWhat they get back
Serve a local APIDevelopersAn OpenAI-compatible endpoint on port 8080
Chat in the terminalHobbyistsReplies in an interactive session
Save memorySelf-hostersA smaller KV cache through run-time options

How to set it up

You need a llama.cpp build numbered b10251 or newer, since the model architecture needs it, and enough memory for the chosen file plus the context cache.

  • llama.cpp build b10251 or newer
  • Memory for one GGUF file plus the KV cache
  • Python and the Hugging Face hub tool, only if you download files by hand
  1. Pick a quantization. The card recommends Q4_K_M as the balance of size and quality.
  2. Either let llama.cpp pull it from the Hub, or download one file yourself.
pip install -U huggingface_hub
hf download axteam/GLM-4.7-Flash-GGUF GLM-4.7-Flash-Q4_K_M.gguf --local-dir .

How to use it

Serve the model as an OpenAI-compatible API, or chat in the terminal with a downloaded file. The jinja option makes llama.cpp use the chat template stored in the file.

llama-server -hf axteam/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 --jinja
llama-cli -m GLM-4.7-Flash-Q4_K_M.gguf -c 8192 --jinja

To shrink the cache that holds the conversation, set the cache types at run time.

llama-server -hf axteam/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 --jinja --cache-type-k q8_0 --cache-type-v q8_0 -fa on
ToolWhat it does
llama-serverServes the model as an OpenAI-compatible API
llama-cliChats with a downloaded file in the terminal
-cSets the context length
--cache-type-k and --cache-type-vChoose the KV cache type, and q8_0 with -fa on halves its size

Example

With the server running, you send a chat message asking what the capital of South Korea is. The card's own check asked this kind of short question, and every file it lists answered correctly.

For newcomers and experts

If you are new to this, start with Q4_K_M, the file the card recommends. Run the llama-cli command, ask one short question and see whether your machine copes.

If you know the field, these are block-wise k-quants with per-block scales and mixed bit allocation, made from the BF16 weights without retraining. The architecture combines deepseek2 with an MTP layer. Cache type is a run-time choice, not stored in the file. The card suggests temperature 1.0.

  1. Pick a quant that fits your memory
  2. Pull or download the GGUF file
  3. Start llama-server or llama-cli
  4. Send chat messages and read replies

Good to know

Two files were removed later. One gave a perplexity result the author could not explain, and the other failed a chat check by answering a Korean question with English text. The author publishes no perplexity numbers.

File size by quantization in GB
Q4_K_M18.9 GB
Q5_K_M22.2 GB
Q6_K25.7 GB
Q8_033.2 GB
  • License: MIT, carried over from the original GLM-4.7-Flash model
  • Cost to run: none named
  • Data it reaches: the files you download from Hugging Face, plus the text you type into your own local server
  • Not for: input other than text, since the card lists text only

README and files

README

GLM-4.7-Flash-GGUF by axteam

Four GGUF quantizations of GLM-4.7-Flash for running a 31B-class chat model locally in llama.cpp, from 18.9 GB to 33.2 GB. For people who host models on their own hardware.

Made by axteam on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF files of GLM-4.7-Flash, a text generation model from zai-org, made by axteam. The files come from post-training quantization with llama.cpp, with no retraining, and each one has the original chat template built in. You pick a size that fits your hardware.

Who it suits: People who run chat models on their own computers and want a ready GGUF file instead of converting weights themselves.

Industry | Who uses it | How they use it | Example
Software | Developers | Serve an OpenAI-compatible API from a local machine | A local endpoint for an app under development
Research | Researchers | Test a large chat model without sending text to a hosted service | Ask a set of questions on a private server
IT teams | Self-hosters | Choose a quant that fits the memory they have | Q4_K_M on a machine with enough room

Skip it if you have little disk or memory, because the smallest file listed is 18.9 GB.

Where to use it:

  • llama.cpp: the llama-server and llama-cli programs
  • Ollama: the card gives a run command but says it was not tested there
  • LM Studio: search for the repository name in its model browser, also untested by the author
Use case | Who | What they get back
Serve a local API | Developers | An OpenAI-compatible endpoint on port 8080
Chat in the terminal | Hobbyists | Replies in an interactive session
Save memory | Self-hosters | A smaller KV cache through run-time options

How to set it up: You need a llama.cpp build numbered b10251 or newer, since the model architecture needs it, and enough memory for the chosen file plus the context cache.

  • llama.cpp build b10251 or newer
  • Memory for one GGUF file plus the KV cache
  • Python and the Hugging Face hub tool, only if you download files by hand
  1. Pick a quantization. The card recommends Q4_K_M as the balance of size and quality.
  2. Either let llama.cpp pull it from the Hub, or download one file yourself.
pip install -U huggingface_hub
hf download axteam/GLM-4.7-Flash-GGUF GLM-4.7-Flash-Q4_K_M.gguf --local-dir .

How to use it: Serve the model as an OpenAI-compatible API, or chat in the terminal with a downloaded file. The jinja option makes llama.cpp use the chat template stored in the file.

llama-server -hf axteam/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 --jinja
llama-cli -m GLM-4.7-Flash-Q4_K_M.gguf -c 8192 --jinja

To shrink the cache that holds the conversation, set the cache types at run time.

llama-server -hf axteam/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 --jinja --cache-type-k q8_0 --cache-type-v q8_0 -fa on
Tool | What it does
llama-server | Serves the model as an OpenAI-compatible API
llama-cli | Chats with a downloaded file in the terminal
-c | Sets the context length
--cache-type-k and --cache-type-v | Choose the KV cache type, and q8_0 with -fa on halves its size

Example: With the server running, you send a chat message asking what the capital of South Korea is. The card's own check asked this kind of short question, and every file it lists answered correctly.

For newcomers and experts: If you are new to this, start with Q4_K_M, the file the card recommends. Run the llama-cli command, ask one short question and see whether your machine copes.

If you know the field, these are block-wise k-quants with per-block scales and mixed bit allocation, made from the BF16 weights without retraining. The architecture combines deepseek2 with an MTP layer. Cache type is a run-time choice, not stored in the file. The card suggests temperature 1.0.

Pick a quant that fits your memory
Pull or download the GGUF file
Start llama-server or llama-cli
Send chat messages and read replies

Good to know: Two files were removed later. One gave a perplexity result the author could not explain, and the other failed a chat check by answering a Korean question with English text. The author publishes no perplexity numbers.

Title: File size by quantization in GB
Q4_K_M: 18.9 GB
Q5_K_M: 22.2 GB
Q6_K: 25.7 GB
Q8_0: 33.2 GB
  • License: MIT, carried over from the original GLM-4.7-Flash model
  • Cost to run: none named
  • Data it reaches: the files you download from Hugging Face, plus the text you type into your own local server
  • Not for: input other than text, since the card lists text only

On Hugging Face, 16 November 2026

  • Task: text generation
  • Library: gguf
  • Parameters: 31.2B
  • Weights: about 15.6 GB at 4-bit (estimate)
  • Formats: GGUF
  • Downloads in the last 30 days: 14,654
  • Likes: 5
  • License: mit
  • Based on: zai-org/GLM-4.7-Flash
  • Tags: gguf, llama.cpp, quantized, text-generation, conversational

Links

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download axteam/GLM-4.7-Flash-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.

Files
README.md  5 KB

1 file, 5 KB, downloaded as a zip.

What’s new

  • Version 1 · 2026-11-16

Compare with similar

Listed Hugging Face models in the same category that share the most tags with this one.

ProductDownloads, last 30 daysLikesParametersLicenseLast change
GLM-4.7-Flash-GGUF by axteam This page14,654531.2Bmit
Qwen3.8-35B-A3B-Distill-GGUFShares llama.cpp · quantized · gguf1,337,09727735.5Bapache-2.0
Qwen3.8-4B-Distill-GGUFShares llama.cpp · quantized · gguf570,9742084.3Bapache-2.0
Qwen3.8-9B-Distill-GGUFShares llama.cpp · quantized · gguf537,6953569.2Bapache-2.0
Qwen3.8-2B-Distill-GGUFShares llama.cpp · quantized · gguf509,8201841.9Bapache-2.0
Mellum2.1-12B-A2.5B-Thinking-GGUFShares llama.cpp · quantized · gguf422,63517212.1Bapache-2.0
Numbers as Mindket last read them, from 2026-11-12 to 2026-11-16.

Reviews

No ratings yet

Write a review

No reviews yet.

More from Mindket

See all from Mindket

all-mpnet-base-v2

sentence-transformers · Kit

Turns sentences and paragraphs into 768-number vectors so you can search, cluster and compare text by meaning. It suits developers and data teams working in En…

New

multilingual-e5-base

intfloat · Kit

Turns text in about 100 languages into vectors so you can compare meaning across them, for search and matching. Made for developers who build multilingual retr…

New

sprite-gen

aldegad · Skill

Turns one base drawing of a character into game-ready sprites, either an atlas with a manifest or transparent motion loops, for game developers working in Code…

New4097

Similar in Developer

See all in Developer

VibeSkills

foryourhealth111-pixel · Skill

Picks the right skills from a large installed pool and breaks each task into checkable steps for your coding agent. In its own benchmark, mean reward rose 21.1…

New3821

claude-bughunter

elementalsouls · Skill

Gives Claude Code 83 skills and 15 slash commands for bug bounty hunting and authorized red-team work, from web vulnerability classes to enterprise identity at…

New6853

Runs Qwen3.8-27B with a vocabulary cut down to ASCII, so a single 16 GB graphics card can hold a much longer prompt. A GGUF build for English text and code wit…

New