GigaChat3.1-10B-A1.8B-GGUF

GigaChat3.1-10B-A1.8B-GGUF

Chats in Russian and English, writes code and calls functions, as a compact mixture of experts model in GGUF files. For people who run assistants on their own hardware.

Made by ai-sage on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF files of GigaChat 3.1 Lightning, the compact instruct model in the GigaChat 3.1 family from ai-sage. It is a mixture of experts model with 10 billion total parameters, of which 1.8 billion are active at inference time. The card lists multilingual assistant work, reasoning, code and function calling, and the record lists Russian and English.

Who it suits: It suits developers and teams who want a compact Russian and English assistant that can reason, write code and call functions, and who prefer to run GGUF files themselves.

Industry | Who uses it | How they use it | Example
Software | Developers | Build a Russian and English assistant that calls functions | A support bot that calls an order lookup tool
Product teams | Application builders | Deploy a compact instruct model for question answering with sources | Answer a user question and cite documents
Hobby | Local model users | Run a 10B mixture of experts model with few active parameters | Chat in Russian on modest hardware

People who want the fp8 or bf16 builds can skip this repository, since the card points to separate ones for them.

Where to use it:

• Any runtime that loads GGUF files, on your own computer
• Agent and assistant setups that use function calling
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Assistant chat | Developers | Replies in Russian or English
Code help | Programmers | Code and explanations
Function calling | Agent builders | Structured calls to the tools you define

How to set it up: You need a few things first.

• Python and pip, for the Hugging Face hub tool
• A runtime that can load GGUF files
• About 5.3 GB of disk space at 4-bit, by estimate

1. Install the hub tool with pip.
2. Download the repository files.
3. Load a GGUF file in your runtime of choice.

pip install -U huggingface_hub
hf download ai-sage/GigaChat3.1-10B-A1.8B-GGUF

How to use it: The part of the card shown here gives no run command, so you follow your GGUF runtime's own steps. You then chat with the model, ask for code, or wire in tools for function calling.

Title: Model summary
Part | Detail
Architecture | Mixture of experts, 10B total and 1.8B active parameters
Attention | Multi-head latent attention, which compresses the KV cache
Decoding | Trained with multi-token prediction for faster output
Languages | Russian and English
Format | GGUF

Example: You ask it in Russian or English to write a Python function that parses a date and explain it. You can expect code with a short explanation, since code and reasoning are among the uses the card names.

For newcomers and experts: If you are new, think of this as a chat assistant that is small enough to try at home. Download one GGUF file, load it in a GGUF-capable app and start with a simple question.

Only 1.8B of the 10B parameters work on each token, so compute stays low while capacity stays high. Multi-head latent attention shrinks the KV cache to save memory. Multi-token prediction lets a serving stack decode several tokens per pass. The release adds a full DPO stage trained in native FP8, with MTP heads trained during DPO. The card reports its BF16 reference at 2,866 output tokens per second on one H100 under vLLM.

Prompt reaches the router
Only 1.8B parameters activate per token
Latent attention keeps the KV cache small
Multi-token prediction speeds decoding
Reply comes back in Russian or English

Good to know:

• License: MIT.
• Cost to run: no API key or paid service named. You supply the hardware, and about 5.3 GB is needed at 4-bit by estimate.
• Data it reaches: only your prompts. Tools you connect for function calling can reach whatever you let them.
• Not for: high-performance fp8 serving, which the card gives to a separate repository. The shown part of the card has no GGUF run command.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 10.7B
• Weights: about 5.3 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 19,408
• Likes: 95
• License: mit
• Languages: ru, en
• Based on: ai-sage/GigaChat3-10B-A1.8B-base
• Tags: transformers, gguf, instruct, multilingual, tool-use, long-context, text-generation, conversational

Links

• Model: https://huggingface.co/ai-sage/GigaChat3.1-10B-A1.8B-GGUF
• Files: https://huggingface.co/ai-sage/GigaChat3.1-10B-A1.8B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download ai-sage/GigaChat3.1-10B-A1.8B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.