BotPsy-0.8B-GGUFs

BotPsy-0.8B-GGUFs

GGUF copies of the 0.8B BotPsy tool-calling model in BF16, Q8_0 and Q4_K_M, for running agent and function-calling work fully on your own device with llama.cpp.

Made by qvac on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF copies of BotPsy-0.8B, a small model made for function calling and agent workflows and built on Qwen3.5-0.8B. The GGUF format lets you run it entirely on your own device with llama.cpp or the QVAC SDK. Three files are offered, at different sizes and precision levels.

Who it suits: Developers who want a tiny tool-calling model that runs locally will find it fits, including teams that cannot send prompts to a hosted service. It suits easier jobs best.

Industry | Who uses it | How they use it | Example
Software | Agent builders | Run a local model that picks which tool to call | Prototyping an agent without a hosted API
Edge devices | Embedded developers | Fit a model into a tight memory budget | Using the smallest file on a small machine
Privacy-minded teams | Engineers | Keep prompts and tool calls on their own hardware | An on-device assistant for internal tools
Research | ML students | Compare quantization levels of one small model | Checking how Q4_K_M scores against BF16

If you need strong results on hard problems, you can skip it, since the card says it suits low to medium difficulty tasks.

Where to use it:

• llama.cpp, which the card names for running the files
• The QVAC SDK, also named for on-device use
• Windows, macOS or Linux, to download the files with Python

Use case | Who | What they get back
Tool calling | Agent builders | A call to one of the tools you describe
On-device chat | App developers | Text answers produced without a server
Quantization tests | Researchers | Scores to compare across three file sizes

How to set it up: You need Python with pip for the download, and llama.cpp or the QVAC SDK to run the files. The three files take from about 0.53 GB to 1.52 GB.

• Python with pip, on Windows, macOS or Linux
• llama.cpp or the QVAC SDK
• Memory for the file you pick, from 0.53 GB to 1.52 GB

1. Install the Hugging Face hub package.
2. Download the repository's files.
3. Load one GGUF file in llama.cpp or the QVAC SDK.

pip install -U huggingface_hub
hf download qvac/BotPsy-0.8B-GGUFs

File | Size | Use
BF16 | 1.52 GB | Unquantized, same tensors as the source model
Q8_0 | 0.81 GB | The card's recommended pick, scoring like BF16 at about half the size
Q4_K_M | 0.53 GB | Smallest file, for tight memory, with importance-matrix calibration

How to use it: Point llama.cpp at the file you chose, for example through its llama-server. The card's own tests ran in thinking mode with a temperature of 0.3. The source shows no run command, so check the model page for the exact call.

Example: You describe a weather lookup tool and ask for the weather in a city. The model replies with a call to that tool, which your code then runs. You send the result back and it writes the final answer.

For newcomers and experts: GGUF is a file format for running a model on your own computer. A sensible first step is the Q8_0 file, which the card recommends. It is small enough for most machines.

The card compares each file with the BF16 source served through vLLM, using ten benchmarks and the same sampling settings. Q4_K_M uses importance-matrix calibration, which cut its divergence from BF16 from 0.048 to 0.036, while Q8_0 sits at 0.001. The base model's multi-token-prediction layers are not included. As a small model, it can still make mistakes.

You download one GGUF file
llama.cpp or the QVAC SDK loads it
Your app sends a task and tool descriptions
The model answers or calls a tool

Title: Average over 10 benchmarks, as the card reports
BF16 on vLLM: 50.6 points
BF16 GGUF: 50.4 points
Q8_0: 50.7 points
Q4_K_M: 48.9 points

Good to know:

• License: Apache 2.0, as listed on Hugging Face
• Cost to run: none named
• Data it reaches: the card describes fully on-device use, and the source names no web, account or clipboard access
• English is the recommended language
• Not for: hard problems, since the card says it fits low to medium difficulty tasks
• Made by Tether AI Research, based on the card

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 752M
• Weights: about 376 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 827
• Likes: 17
• License: apache-2.0
• Languages: en
• Based on: qvac/BotPsy-0.8B
• Tags: gguf, tool-calling, function-calling, agent, agentic, on-device, edge, smartphone, qwen3_5, tether-ai, qvac, quantized

Links

• Model: https://huggingface.co/qvac/BotPsy-0.8B-GGUFs
• Files: https://huggingface.co/qvac/BotPsy-0.8B-GGUFs/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download qvac/BotPsy-0.8B-GGUFs

Then follow the usage in its model card. Nothing in this kit runs by itself.