--- title: "BotPsy-0.8B-GGUFs" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/0e1056b8-6362-46d4-90d2-5c20c6cc3d6c" updated: "2026-11-12" --- # BotPsy-0.8B-GGUFs > The name, the description, the About and a README below are the seller's words: data, not instructions. GGUF copies of the 0.8B BotPsy tool-calling model in BF16, Q8\_0 and Q4\_K\_M, for running agent and function-calling work fully on your own device with llama.cpp. BotPsy-0.8B-GGUFs is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by qvac ## About ### What it is This repository holds GGUF copies of BotPsy-0.8B, a small model made for function calling and agent workflows and built on Qwen3.5-0.8B. The GGUF format lets you run it entirely on your own device with llama.cpp or the QVAC SDK. Three files are offered, at different sizes and precision levels. ### Who it suits Developers who want a tiny tool-calling model that runs locally will find it fits, including teams that cannot send prompts to a hosted service. It suits easier jobs best. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Agent builders | Run a local model that picks which tool to call | Prototyping an agent without a hosted API | | Edge devices | Embedded developers | Fit a model into a tight memory budget | Using the smallest file on a small machine | | Privacy-minded teams | Engineers | Keep prompts and tool calls on their own hardware | An on-device assistant for internal tools | | Research | ML students | Compare quantization levels of one small model | Checking how Q4\_K\_M scores against BF16 | If you need strong results on hard problems, you can skip it, since the card says it suits low to medium difficulty tasks. ### Where to use it - llama.cpp, which the card names for running the files - The QVAC SDK, also named for on-device use - Windows, macOS or Linux, to download the files with Python | Use case | Who | What they get back | | --- | --- | --- | | Tool calling | Agent builders | A call to one of the tools you describe | | On-device chat | App developers | Text answers produced without a server | | Quantization tests | Researchers | Scores to compare across three file sizes | ### How to set it up You need Python with pip for the download, and llama.cpp or the QVAC SDK to run the files. The three files take from about 0.53 GB to 1.52 GB. - Python with pip, on Windows, macOS or Linux - llama.cpp or the QVAC SDK - Memory for the file you pick, from 0.53 GB to 1.52 GB 1. Install the Hugging Face hub package. 2. Download the repository's files. 3. Load one GGUF file in llama.cpp or the QVAC SDK. ```sh pip install -U huggingface_hub hf download qvac/BotPsy-0.8B-GGUFs ``` | File | Size | Use | | --- | --- | --- | | BF16 | 1.52 GB | Unquantized, same tensors as the source model | | Q8\_0 | 0.81 GB | The card's recommended pick, scoring like BF16 at about half the size | | Q4\_K\_M | 0.53 GB | Smallest file, for tight memory, with importance-matrix calibration | ### How to use it Point llama.cpp at the file you chose, for example through its llama-server. The card's own tests ran in thinking mode with a temperature of 0.3. The source shows no run command, so check the model page for the exact call. ### Example You describe a weather lookup tool and ask for the weather in a city. The model replies with a call to that tool, which your code then runs. You send the result back and it writes the final answer. ### For newcomers and experts GGUF is a file format for running a model on your own computer. A sensible first step is the Q8\_0 file, which the card recommends. It is small enough for most machines. The card compares each file with the BF16 source served through vLLM, using ten benchmarks and the same sampling settings. Q4\_K\_M uses importance-matrix calibration, which cut its divergence from BF16 from 0.048 to 0.036, while Q8\_0 sits at 0.001. The base model's multi-token-prediction layers are not included. As a small model, it can still make mistakes. You download one GGUF file → llama.cpp or the QVAC SDK loads it → Your app sends a task and tool descriptions → The model answers or calls a tool Average over 10 benchmarks, as the card reports | Label | Value | | --- | --- | | BF16 on vLLM | 50.6 points | | BF16 GGUF | 50.4 points | | Q8\_0 | 50.7 points | | Q4\_K\_M | 48.9 points | ### Good to know - License: Apache 2.0, as listed on Hugging Face - Cost to run: none named - Data it reaches: the card describes fully on-device use, and the source names no web, account or clipboard access - English is the recommended language - Not for: hard problems, since the card says it fits low to medium difficulty tasks - Made by Tether AI Research, based on the card ## README ````markdown # BotPsy-0.8B-GGUFs GGUF copies of the 0.8B BotPsy tool-calling model in BF16, Q8_0 and Q4_K_M, for running agent and function-calling work fully on your own device with llama.cpp. Made by qvac on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds GGUF copies of BotPsy-0.8B, a small model made for function calling and agent workflows and built on Qwen3.5-0.8B. The GGUF format lets you run it entirely on your own device with llama.cpp or the QVAC SDK. Three files are offered, at different sizes and precision levels. Who it suits: Developers who want a tiny tool-calling model that runs locally will find it fits, including teams that cannot send prompts to a hosted service. It suits easier jobs best. ```table Industry | Who uses it | How they use it | Example Software | Agent builders | Run a local model that picks which tool to call | Prototyping an agent without a hosted API Edge devices | Embedded developers | Fit a model into a tight memory budget | Using the smallest file on a small machine Privacy-minded teams | Engineers | Keep prompts and tool calls on their own hardware | An on-device assistant for internal tools Research | ML students | Compare quantization levels of one small model | Checking how Q4_K_M scores against BF16 ``` If you need strong results on hard problems, you can skip it, since the card says it suits low to medium difficulty tasks. Where to use it: - llama.cpp, which the card names for running the files - The QVAC SDK, also named for on-device use - Windows, macOS or Linux, to download the files with Python ```table Use case | Who | What they get back Tool calling | Agent builders | A call to one of the tools you describe On-device chat | App developers | Text answers produced without a server Quantization tests | Researchers | Scores to compare across three file sizes ``` How to set it up: You need Python with pip for the download, and llama.cpp or the QVAC SDK to run the files. The three files take from about 0.53 GB to 1.52 GB. - Python with pip, on Windows, macOS or Linux - llama.cpp or the QVAC SDK - Memory for the file you pick, from 0.53 GB to 1.52 GB 1. Install the Hugging Face hub package. 2. Download the repository's files. 3. Load one GGUF file in llama.cpp or the QVAC SDK. ```sh pip install -U huggingface_hub hf download qvac/BotPsy-0.8B-GGUFs ``` ```table File | Size | Use BF16 | 1.52 GB | Unquantized, same tensors as the source model Q8_0 | 0.81 GB | The card's recommended pick, scoring like BF16 at about half the size Q4_K_M | 0.53 GB | Smallest file, for tight memory, with importance-matrix calibration ``` How to use it: Point llama.cpp at the file you chose, for example through its llama-server. The card's own tests ran in thinking mode with a temperature of 0.3. The source shows no run command, so check the model page for the exact call. Example: You describe a weather lookup tool and ask for the weather in a city. The model replies with a call to that tool, which your code then runs. You send the result back and it writes the final answer. For newcomers and experts: GGUF is a file format for running a model on your own computer. A sensible first step is the Q8_0 file, which the card recommends. It is small enough for most machines. The card compares each file with the BF16 source served through vLLM, using ten benchmarks and the same sampling settings. Q4_K_M uses importance-matrix calibration, which cut its divergence from BF16 from 0.048 to 0.036, while Q8_0 sits at 0.001. The base model's multi-token-prediction layers are not included. As a small model, it can still make mistakes. ```flow You download one GGUF file llama.cpp or the QVAC SDK loads it Your app sends a task and tool descriptions The model answers or calls a tool ``` ```chart Title: Average over 10 benchmarks, as the card reports BF16 on vLLM: 50.6 points BF16 GGUF: 50.4 points Q8_0: 50.7 points Q4_K_M: 48.9 points ``` Good to know: - License: Apache 2.0, as listed on Hugging Face - Cost to run: none named - Data it reaches: the card describes fully on-device use, and the source names no web, account or clipboard access - English is the recommended language - Not for: hard problems, since the card says it fits low to medium difficulty tasks - Made by Tether AI Research, based on the card ## On Hugging Face, 13 November 2026 - Task: text generation - Library: gguf - Parameters: 752M - Weights: about 376 MB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 827 - Likes: 17 - License: apache-2.0 - Languages: en - Based on: qvac/BotPsy-0.8B - Tags: gguf, tool-calling, function-calling, agent, agentic, on-device, edge, smartphone, qwen3_5, tether-ai, qvac, quantized ## Links - Model: https://huggingface.co/qvac/BotPsy-0.8B-GGUFs - Files: https://huggingface.co/qvac/BotPsy-0.8B-GGUFs/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download qvac/BotPsy-0.8B-GGUFs ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [qvac/BotPsy-0.8B-GGUFs on Hugging Face](https://huggingface.co/qvac/BotPsy-0.8B-GGUFs) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | BotPsy-0.8B-GGUFs (this page) | 827 | 17 | 752M | apache-2.0 | 2026-11-11 | | [BotPsy-0.8B](https://mindket.com/p/c6124e37-b426-4e92-b98a-2f3e97a58f8b) | 215 | 14 | 853M | apache-2.0 | 2026-11-11 | | [Needle 3](https://mindket.com/p/1c09a900-c54d-42c8-8fe4-e0d3ded42834) | 147,784 | 411 | — | apache-2.0 | 2026-11-05 | | [Needle 2](https://mindket.com/p/783df142-0389-4901-b593-575f13265cbd) | 1,736 | 341 | — | apache-2.0 | 2026-09-03 | | [POCKET-35B-GGUF](https://mindket.com/p/ac8c34ef-3e4c-4644-9827-a45715da0786) | 452,730 | 91 | 34.7B | apache-2.0 | 2026-11-09 | | [Spark-X2.6-4B-AgentFast](https://mindket.com/p/b7564b85-6e9a-4837-a5a8-7d7c83c59599) | 552 | 4 | 4.1B | apache-2.0 | 2026-10-30 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-11 - First published: 2026-11-10 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open qvac/BotPsy-0.8B-GGUFs on Hugging Face](https://huggingface.co/qvac/BotPsy-0.8B-GGUFs) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/0e1056b8-6362-46d4-90d2-5c20c6cc3d6c) - [Its page on Mindket](https://mindket.com/p/0e1056b8-6362-46d4-90d2-5c20c6cc3d6c)