Clef-LoRA-for-Qwen3.8-27B-GGUF

Clef-LoRA-for-Qwen3.8-27B-GGUF

Adds Cloudflare Clef's decision head to a Qwen3.8-27B chat model, so one llama.cpp server on a 24 GB GPU both answers typed questions and chats. It is a LoRA adapter, not a standalone model.

Made by trevest on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: The repo packs a LoRA adapter for the Qwen3.8-27B model together with the decision head of Cloudflare's Clef. It cannot run alone. On top of one copy of Qwen3.8-27B in llama.cpp it gives you two things, typed decisions from Clef and ordinary chat.

Who it suits: Developers who run local model servers and want decisions and chat from the same weights. The design aims at a 24 GB GPU. Running Clef as its own model would add roughly 15 GB more, and that would not fit next to a chat model.

Industry | Who uses it | How they use it | Example
Software | Developers with a local server | Serve decisions and chat from one resident model | Score a message against typed questions, then draft a reply
Self-hosting | Home lab and small team admins | Fit two jobs on one 24 GB GPU | Skip loading a second 15 GB model
Automation | Pipeline builders | Chain decide, write and verify steps against one server | Run the card's demo script
Research | Model researchers | Study how a LoRA on late layers carries a decision head | Inspect the adapter and the patch

People without a suitable GPU, or who want a ready standalone model, can skip it. Mac users are pointed to a separate MLX version.

Where to use it:

• A patched llama.cpp server on a computer with a 24 GB GPU
• A CPU sidecar that serves the decision endpoint
• A client that calls one path for decisions and another for chat
• Docker, which the repo uses to build the patched server and the sidecar

Use case | Who | What they get back
Typed decisions | Developers | One probability for each answer option of each question, computed in a single pass
Chat | Anyone using the same server | Normal Qwen3.8-27B replies, with MTP speculative decoding
Decide, write, verify | Pipeline builders | A Node demo that exercises both paths together

How to set it up:

• A GPU with 24 GB of memory
• A Qwen3.8-27B GGUF backbone, about 15.1 GB, such as the Tetris 4.4bpw file by the same author
• This repository's files, about 1.6 GB
• The llama.cpp source, built with the repo's patch
• Node 20 or newer, for the demo script

1. Install the Hugging Face hub tools and download this repository's files.
2. Get a Qwen3.8-27B GGUF backbone.
3. Fetch the llama.cpp source, check out the commit the patch targets, and apply the patch.
4. Build the llama-server target with CUDA, then follow the card's remaining steps to download the weights and start the server and the sidecar.

pip install -U huggingface_hub
hf download trevest/Clef-LoRA-for-Qwen3.8-27B-GGUF

git checkout ad1de39e0708e3ced9c71bb3c82d93a2c046a73f
git apply /path/to/llama.cpp-patch/0001-embeddings-lora-f16.patch
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 && cmake --build build --target llama-server -j

How to use it: Send decision requests to the sidecar's systemone endpoint, with a state and typed questions. Send chat requests to the server's chat completions endpoint. The LoRA stays at scale 0 for chat and switches to scale 1 only for the decision request.

Tool | What it does
clef-lora-f16.gguf | The llama.cpp LoRA adapter, f16, with 668M parameters
joint_head.safetensors | Clef's joint schema head, unchanged from Cloudflare
tokenizer.json | Clef's tokenizer, unchanged
llama.cpp-patch | The engine patch and a Dockerfile that builds the patched server
clefhead | The CPU sidecar that serves the decision endpoint
examples/one_model_demo.mjs | A Node pipeline that decides, writes and verifies

Example: A client sends a message and a typed question about whether it is urgent to the decision endpoint. The sidecar gets hidden states from the server with the LoRA on, and returns a probability for each option. A chat request to the other path returns an ordinary reply.

For newcomers and experts: This is advanced setup work for people who already run llama.cpp. A gentle first step is reading the card's file table, then running the demo script once both servers are up.

Clef builds on Qwen3.8-27B with a LoRA merged into layers 40 through 63, and nowhere else. The layers before that, the embeddings, the output head and the vision tower match the base byte for byte. The sidecar asks llama-server for the final hidden state of each prompt token and applies Clef's head. A small llama.cpp patch adds a per-request LoRA, compact hidden state output and MTP support next to embeddings.

Client sends a decision request to the sidecar
Sidecar asks the server for hidden states, LoRA on
Server returns the final state of each prompt token
Clef head gives a probability per option

Good to know:

• Backbone: You need a separate Qwen3.8-27B GGUF. Any file converted from the matching revision with a current converter should work.
• Patch: Stock llama.cpp needs the patch, so the build step is required.
• Hardware: The prebuilt image mentioned in the card targets CUDA 13.0 on Blackwell cards only.
• License: Apache 2.0.
• Cost to run: none named. You need a 24 GB GPU, and no API key or paid service appears in the source.
• Data it reaches: Your prompts go to your own sidecar and llama-server. Building needs the llama.cpp source and the model files from their hosts.
• Not for: Standalone use, since the adapter does nothing without its backbone. Mac users should take the MLX version instead.

On Hugging Face, 13 November 2026

• Task: text classification
• Library: gguf
• Parameters: 668M
• Weights: about 334 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 311
• Likes: 6
• License: apache-2.0
• Based on: Cloudflare/clef
• Tags: gguf, clef, lora, llama.cpp, decision-model, systemone, speculative-decoding, adapter, qwen3.8, 27b, text-classification

Links

• Model: https://huggingface.co/trevest/Clef-LoRA-for-Qwen3.8-27B-GGUF
• Files: https://huggingface.co/trevest/Clef-LoRA-for-Qwen3.8-27B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download trevest/Clef-LoRA-for-Qwen3.8-27B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.