all-mpnet-base-v2
sentence-transformers · Kit
Turns sentences and paragraphs into 768-number vectors so you can search, cluster and compare text by meaning. It suits developers and data teams working in En…
Four GGUF quantizations of GLM-4.7-Flash for running a 31B-class chat model locally in llama.cpp, from 18.9 GB to 33.2 GB. For people who host models on their own hardware.
GLM-4.7-Flash-GGUF by axteam is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on . Download it on this page, a zip of its files. Nothing in it runs by itself.
pip install -U huggingface_hub hf download axteam/GLM-4.7-Flash-GGUF GLM-4.7-Flash-Q4_K_M.gguf --local-dir .
This repository holds GGUF files of GLM-4.7-Flash, a text generation model from zai-org, made by axteam. The files come from post-training quantization with llama.cpp, with no retraining, and each one has the original chat template built in. You pick a size that fits your hardware.
People who run chat models on their own computers and want a ready GGUF file instead of converting weights themselves.
| Industry | Who uses it | How they use it | Example |
|---|---|---|---|
| Software | Developers | Serve an OpenAI-compatible API from a local machine | A local endpoint for an app under development |
| Research | Researchers | Test a large chat model without sending text to a hosted service | Ask a set of questions on a private server |
| IT teams | Self-hosters | Choose a quant that fits the memory they have | Q4_K_M on a machine with enough room |
Skip it if you have little disk or memory, because the smallest file listed is 18.9 GB.
| Use case | Who | What they get back |
|---|---|---|
| Serve a local API | Developers | An OpenAI-compatible endpoint on port 8080 |
| Chat in the terminal | Hobbyists | Replies in an interactive session |
| Save memory | Self-hosters | A smaller KV cache through run-time options |
You need a llama.cpp build numbered b10251 or newer, since the model architecture needs it, and enough memory for the chosen file plus the context cache.
pip install -U huggingface_hub hf download axteam/GLM-4.7-Flash-GGUF GLM-4.7-Flash-Q4_K_M.gguf --local-dir .
Serve the model as an OpenAI-compatible API, or chat in the terminal with a downloaded file. The jinja option makes llama.cpp use the chat template stored in the file.
llama-server -hf axteam/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 --jinja llama-cli -m GLM-4.7-Flash-Q4_K_M.gguf -c 8192 --jinja
To shrink the cache that holds the conversation, set the cache types at run time.
llama-server -hf axteam/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 --jinja --cache-type-k q8_0 --cache-type-v q8_0 -fa on
| Tool | What it does |
|---|---|
| llama-server | Serves the model as an OpenAI-compatible API |
| llama-cli | Chats with a downloaded file in the terminal |
| -c | Sets the context length |
| --cache-type-k and --cache-type-v | Choose the KV cache type, and q8_0 with -fa on halves its size |
With the server running, you send a chat message asking what the capital of South Korea is. The card's own check asked this kind of short question, and every file it lists answered correctly.
If you are new to this, start with Q4_K_M, the file the card recommends. Run the llama-cli command, ask one short question and see whether your machine copes.
If you know the field, these are block-wise k-quants with per-block scales and mixed bit allocation, made from the BF16 weights without retraining. The architecture combines deepseek2 with an MTP layer. Cache type is a run-time choice, not stored in the file. The card suggests temperature 1.0.
Two files were removed later. One gave a perplexity result the author could not explain, and the other failed a chat check by answering a Korean question with English text. The author publishes no perplexity numbers.
Four GGUF quantizations of GLM-4.7-Flash for running a 31B-class chat model locally in llama.cpp, from 18.9 GB to 33.2 GB. For people who host models on their own hardware.
Made by axteam on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.
What it is: This repository holds GGUF files of GLM-4.7-Flash, a text generation model from zai-org, made by axteam. The files come from post-training quantization with llama.cpp, with no retraining, and each one has the original chat template built in. You pick a size that fits your hardware.
Who it suits: People who run chat models on their own computers and want a ready GGUF file instead of converting weights themselves.
Industry | Who uses it | How they use it | Example
Software | Developers | Serve an OpenAI-compatible API from a local machine | A local endpoint for an app under development
Research | Researchers | Test a large chat model without sending text to a hosted service | Ask a set of questions on a private server
IT teams | Self-hosters | Choose a quant that fits the memory they have | Q4_K_M on a machine with enough room
Skip it if you have little disk or memory, because the smallest file listed is 18.9 GB.
Where to use it:
Use case | Who | What they get back
Serve a local API | Developers | An OpenAI-compatible endpoint on port 8080
Chat in the terminal | Hobbyists | Replies in an interactive session
Save memory | Self-hosters | A smaller KV cache through run-time options
How to set it up: You need a llama.cpp build numbered b10251 or newer, since the model architecture needs it, and enough memory for the chosen file plus the context cache.
pip install -U huggingface_hub
hf download axteam/GLM-4.7-Flash-GGUF GLM-4.7-Flash-Q4_K_M.gguf --local-dir .
How to use it: Serve the model as an OpenAI-compatible API, or chat in the terminal with a downloaded file. The jinja option makes llama.cpp use the chat template stored in the file.
llama-server -hf axteam/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 --jinja
llama-cli -m GLM-4.7-Flash-Q4_K_M.gguf -c 8192 --jinja
To shrink the cache that holds the conversation, set the cache types at run time.
llama-server -hf axteam/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 --jinja --cache-type-k q8_0 --cache-type-v q8_0 -fa on
Tool | What it does
llama-server | Serves the model as an OpenAI-compatible API
llama-cli | Chats with a downloaded file in the terminal
-c | Sets the context length
--cache-type-k and --cache-type-v | Choose the KV cache type, and q8_0 with -fa on halves its size
Example: With the server running, you send a chat message asking what the capital of South Korea is. The card's own check asked this kind of short question, and every file it lists answered correctly.
For newcomers and experts: If you are new to this, start with Q4_K_M, the file the card recommends. Run the llama-cli command, ask one short question and see whether your machine copes.
If you know the field, these are block-wise k-quants with per-block scales and mixed bit allocation, made from the BF16 weights without retraining. The architecture combines deepseek2 with an MTP layer. Cache type is a run-time choice, not stored in the file. The card suggests temperature 1.0.
Pick a quant that fits your memory
Pull or download the GGUF file
Start llama-server or llama-cli
Send chat messages and read replies
Good to know: Two files were removed later. One gave a perplexity result the author could not explain, and the other failed a chat check by answering a Korean question with English text. The author publishes no perplexity numbers.
Title: File size by quantization in GB
Q4_K_M: 18.9 GB
Q5_K_M: 22.2 GB
Q6_K: 25.7 GB
Q8_0: 33.2 GB
Download its files with Hugging Face's command line:
pip install -U huggingface_hub
hf download axteam/GLM-4.7-Flash-GGUF
Then follow the usage in its model card. Nothing in this kit runs by itself.
README.md 5 KB
Listed Hugging Face models in the same category that share the most tags with this one.
| Product | Downloads, last 30 days | Likes | Parameters | License | Last change |
|---|---|---|---|---|---|
| GLM-4.7-Flash-GGUF by axteam This page | 14,654 | 5 | 31.2B | mit | |
| Qwen3.8-35B-A3B-Distill-GGUFShares llama.cpp · quantized · gguf | 1,337,097 | 277 | 35.5B | apache-2.0 | |
| Qwen3.8-4B-Distill-GGUFShares llama.cpp · quantized · gguf | 570,974 | 208 | 4.3B | apache-2.0 | |
| Qwen3.8-9B-Distill-GGUFShares llama.cpp · quantized · gguf | 537,695 | 356 | 9.2B | apache-2.0 | |
| Qwen3.8-2B-Distill-GGUFShares llama.cpp · quantized · gguf | 509,820 | 184 | 1.9B | apache-2.0 | |
| Mellum2.1-12B-A2.5B-Thinking-GGUFShares llama.cpp · quantized · gguf | 422,635 | 172 | 12.1B | apache-2.0 |
sentence-transformers · Kit
Turns sentences and paragraphs into 768-number vectors so you can search, cluster and compare text by meaning. It suits developers and data teams working in En…
intfloat · Kit
Turns text in about 100 languages into vectors so you can compare meaning across them, for search and matching. Made for developers who build multilingual retr…
aldegad · Skill
Turns one base drawing of a character into game-ready sprites, either an atlas with a manifest or transparent motion loops, for game developers working in Code…
foryourhealth111-pixel · Skill
Picks the right skills from a large installed pool and breaks each task into checkable steps for your coding agent. In its own benchmark, mean reward rose 21.1…
elementalsouls · Skill
Gives Claude Code 83 skills and 15 slash commands for bug bounty hunting and authorized red-team work, from web vulnerability classes to enterprise identity at…
bsaleh03 · Kit
Runs Qwen3.8-27B with a vocabulary cut down to ASCII, so a single 16 GB graphics card can hold a much longer prompt. A GGUF build for English text and code wit…