--- title: "AsteroRank-Lepton-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/ab0befc6-e06f-4977-800b-acbcba1b2a94" updated: "2026-11-14" --- # AsteroRank-Lepton-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Ranks search results by relevance with a reranker for STEM and general search, shipped as GGUF files for llama.cpp. For developers and researchers who want local ranking, non-commercial use only. AsteroRank-Lepton-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-14. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by Asteromorph ## About ### What it is These are GGUF versions of AsteroRank-Lepton, a reranker for STEM and general search. A reranker takes a query and a list of passages and sorts the passages by how well they answer it. The files run in llama.cpp, so the ranking happens on your own machine. ### Who it suits Developers who build search or retrieval pipelines, and researchers who need technical passages sorted for a question. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Search engineers | Re-sort retrieved passages before showing results | Candidate documents sorted for a user query | | Science | Researchers | Rank abstracts or paper excerpts for a technical question | Chemistry passages sorted for a borylation query | | AI agents | Agent builders | Pass a short context line along with the query | An agent that ranks sources for the article it is writing | Skip it if you need commercial use, because the license is non-commercial. ### Where to use it - llama.cpp: the llama-server program, on Windows, macOS or Linux - Python scripts: the rerank.py client that comes with the files - GPU or CPU: the card shows a flag that offloads every layer to a GPU | Use case | Who | What they get back | | --- | --- | --- | | Rank passages for a query | Developers | A list sorted by relevance score | | Rank with extra context | Agent builders | Scores that take a short context text into account | | Limit the results | Search engineers | Only the top few passages, using the top\_n setting | ### How to set it up You need a current llama.cpp build that supports Qwen3.5 and includes llama-server. A stock build works, with no fork or patch. You also need Python 3.8 or newer with jinja2. - llama.cpp with llama-server - Python 3.8 or newer and the jinja2 package - Disk space for one quant, about 3.1 GB for Q4\_K\_M 1. Install the Hugging Face hub tool. 2. Download one quantization together with the client script. 3. Start llama-server on that file. ```bash pip install -U huggingface_hub ``` Put your own value in its key setting first. ### How to use it Import the client class from rerank.py and point it at your running llama-server. Call its rerank method with a query and a list of documents, and add a context text if you have one. ```python from rerank import GGUFReranker ``` Each result holds the document, a relevance score between 0 and 1, and its original index, sorted from most to least relevant. | Tool | What it does | | --- | --- | | GGUFReranker | Client class in rerank.py that scores passages through the server | | rerank | Takes a query, documents and optional context, and returns sorted results | | top\_n | Limits how many results come back | | -ngl 99 | Offloads all layers of the model to the GPU | | -np | Sets how many requests run in parallel | ### Example You ask it to rank three short chemistry passages for a query about para-selective borylation of aryl sulfonate ion pairs, adding one line of context about the article. You get the three passages back as a list ordered by score, each with its index. ### For newcomers and experts If you are new to this, think of it as a sorting step for search. Download the Q4\_K\_M file and the client, start the server, and rank three short passages for one question. If you know the field, the score is the gap between the model's yes and no logits at the end of a prompt that is embedded in the file as a chat template. The server's built-in rerank endpoint takes no context, so rerank.py scores through the completion endpoint instead and reads next-token log-probabilities. Download one quant and rerank.py → Start llama-server on the file → Send a query, documents and context → Read scores sorted high to low ### Good to know Quants below BF16 were made with an importance matrix file that ships in the repository. Smaller quants trade quality for size. File size by quantization in GB | Label | Value | | --- | --- | | BF16 | 9.7 GB | | Q8\_0 | 5.2 GB | | Q6\_K | 4.0 GB | | Q5\_K\_M | 3.5 GB | | Q4\_K\_M | 3.1 GB | | Q3\_K\_M | 2.5 GB | | Q2\_K | 2.1 GB | | IQ2\_XXS | 1.7 GB | - License: CC BY-NC 4.0, shared with the original reranker. Its base model, Qwen3.5-4B, is Apache 2.0 - Cost to run: none named - Data it reaches: the query, documents and context you send to a llama-server you run yourself - Not for: commercial use, which the license does not allow ## README ````markdown # AsteroRank-Lepton-GGUF Ranks search results by relevance with a reranker for STEM and general search, shipped as GGUF files for llama.cpp. For developers and researchers who want local ranking, non-commercial use only. Made by Asteromorph on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: These are GGUF versions of AsteroRank-Lepton, a reranker for STEM and general search. A reranker takes a query and a list of passages and sorts the passages by how well they answer it. The files run in llama.cpp, so the ranking happens on your own machine. Who it suits: Developers who build search or retrieval pipelines, and researchers who need technical passages sorted for a question. ```table Industry | Who uses it | How they use it | Example Software | Search engineers | Re-sort retrieved passages before showing results | Candidate documents sorted for a user query Science | Researchers | Rank abstracts or paper excerpts for a technical question | Chemistry passages sorted for a borylation query AI agents | Agent builders | Pass a short context line along with the query | An agent that ranks sources for the article it is writing ``` Skip it if you need commercial use, because the license is non-commercial. Where to use it: - llama.cpp: the llama-server program, on Windows, macOS or Linux - Python scripts: the rerank.py client that comes with the files - GPU or CPU: the card shows a flag that offloads every layer to a GPU ```table Use case | Who | What they get back Rank passages for a query | Developers | A list sorted by relevance score Rank with extra context | Agent builders | Scores that take a short context text into account Limit the results | Search engineers | Only the top few passages, using the top_n setting ``` How to set it up: You need a current llama.cpp build that supports Qwen3.5 and includes llama-server. A stock build works, with no fork or patch. You also need Python 3.8 or newer with jinja2. - llama.cpp with llama-server - Python 3.8 or newer and the jinja2 package - Disk space for one quant, about 3.1 GB for Q4_K_M 1. Install the Hugging Face hub tool. 2. Download one quantization together with the client script. 3. Start llama-server on that file. ```bash pip install -U huggingface_hub ``` Put your own value in its key setting first. How to use it: Import the client class from rerank.py and point it at your running llama-server. Call its rerank method with a query and a list of documents, and add a context text if you have one. ```python from rerank import GGUFReranker ``` Each result holds the document, a relevance score between 0 and 1, and its original index, sorted from most to least relevant. ```table Tool | What it does GGUFReranker | Client class in rerank.py that scores passages through the server rerank | Takes a query, documents and optional context, and returns sorted results top_n | Limits how many results come back -ngl 99 | Offloads all layers of the model to the GPU -np | Sets how many requests run in parallel ``` Example: You ask it to rank three short chemistry passages for a query about para-selective borylation of aryl sulfonate ion pairs, adding one line of context about the article. You get the three passages back as a list ordered by score, each with its index. For newcomers and experts: If you are new to this, think of it as a sorting step for search. Download the Q4_K_M file and the client, start the server, and rank three short passages for one question. If you know the field, the score is the gap between the model's yes and no logits at the end of a prompt that is embedded in the file as a chat template. The server's built-in rerank endpoint takes no context, so rerank.py scores through the completion endpoint instead and reads next-token log-probabilities. ```flow Download one quant and rerank.py Start llama-server on the file Send a query, documents and context Read scores sorted high to low ``` Good to know: Quants below BF16 were made with an importance matrix file that ships in the repository. Smaller quants trade quality for size. ```chart Title: File size by quantization in GB BF16: 9.7 GB Q8_0: 5.2 GB Q6_K: 4.0 GB Q5_K_M: 3.5 GB Q4_K_M: 3.1 GB Q3_K_M: 2.5 GB Q2_K: 2.1 GB IQ2_XXS: 1.7 GB ``` - License: CC BY-NC 4.0, shared with the original reranker. Its base model, Qwen3.5-4B, is Apache 2.0 - Cost to run: none named - Data it reaches: the query, documents and context you send to a llama-server you run yourself - Not for: commercial use, which the license does not allow ## On Hugging Face, 14 November 2026 - Task: text ranking - Library: llama.cpp - Parameters: 4.8B - Weights: about 2.4 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 626 - Likes: 3 - License: cc-by-nc-4.0 - Languages: en - Based on: Asteromorph/AsteroRank-Lepton - Tags: llama.cpp, gguf, reranker, qwen3.5, llama-cpp, text-ranking, conversational ## Links - Model: https://huggingface.co/Asteromorph/AsteroRank-Lepton-GGUF - Files: https://huggingface.co/Asteromorph/AsteroRank-Lepton-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download Asteromorph/AsteroRank-Lepton-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [Asteromorph/AsteroRank-Lepton-GGUF on Hugging Face](https://huggingface.co/Asteromorph/AsteroRank-Lepton-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | AsteroRank-Lepton-GGUF (this page) | 626 | 3 | 4.8B | cc-by-nc-4.0 | 2026-11-05 | | [Qwen3-Reranker-0.6B-Q8\_0-GGUF](https://mindket.com/p/ef8865aa-9c06-4cab-8674-5746900d8414) | 116,029 | 32 | 596M | apache-2.0 | 2025-10-03 | | [hs-rerank-1](https://mindket.com/p/c3b813e1-9df1-49fd-80a2-1a39304f96f1) | 1,148,007 | 6 | 114M | mit | 2026-11-03 | | [CLM-v0.1-8B](https://mindket.com/p/946fad17-f7f0-4cda-8ff3-5155aa69ae3f) | 1,092 | 794 | — | apache-2.0 | 2026-09-24 | | [hs-decide-1-flash](https://mindket.com/p/6af342ec-c349-4bbd-95ad-e2f3ef1740b4) | 858 | 20 | 752M | apache-2.0 | 2026-11-09 | | [AsteroRank-Lepton](https://mindket.com/p/82fba0ab-2599-4f3a-9a2f-94ed37271fe0) | 58 | 7 | 4.8B | cc-by-nc-4.0 | 2026-11-05 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-14. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-05 - First published: 2026-11-02 - Checked on Hugging Face: 2026-11-14 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open Asteromorph/AsteroRank-Lepton-GGUF on Hugging Face](https://huggingface.co/Asteromorph/AsteroRank-Lepton-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/ab0befc6-e06f-4977-800b-acbcba1b2a94) - [Its page on Mindket](https://mindket.com/p/ab0befc6-e06f-4977-800b-acbcba1b2a94)