--- title: "GuardrailAgent Benign Check" kind: "Kit" category: "Research" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/bded4065-73fb-4b5a-bd51-bcfe501926d0" updated: "2026-11-12" --- # GuardrailAgent Benign Check > The name, the description, the About and a README below are the seller's words: data, not instructions. Shows how often the GuardrailAgent input gate wrongly flags 250 harmless XSTest prompts as jailbreaks, with every prompt and decision. Useful for people who build or judge prompt filters. GuardrailAgent Benign Check is a kit in the Research category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by Sattyam ## About ### What it is This Space reports how often a jailbreak filter called GuardrailAgent flags harmless prompts by mistake. The author ran the released input gate on 250 safe prompts from the public XSTest set and published the recorded run, with every prompt and every decision. ### Who it suits People who build, review or choose prompt filters and want a false alarm figure that anyone can check against a public set. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | AI safety | Guardrail developers | Measure false alarms on harmless prompts | Compare this run's rate with the rate on their own prompts | | Research | Readers of the paper | Test a published claim against a public set | Read how this run differs from the paper's 1.6% figure | | Trust and safety | Teams picking a filter | See which kinds of safe prompt trip a gate | Open the table of flagged prompts by type | | Education | Students of LLM security | Learn how a two part gate decides | Read the nine flagged prompts and how each was caught | If you want a tool that screens your own prompts live, this is a report, not a filter. ### Where to use it It is a static page, so a browser is all you need. - A web browser, on the Space's page at Hugging Face with nothing to install - A downloaded copy on Windows, macOS or Linux, opened in any browser | Use case | Who | What they get back | | --- | --- | --- | | Check the headline rate | Reviewers | A flagged share of 3.6% with its 95% interval | | Browse flagged prompts | Filter builders | The nine prompts the gate flagged, grouped by type | | Read the method | Researchers | The model, versions and scripts used for the run | ### How to set it up Viewing it online needs nothing. To keep a copy, you need two things. - Python and pip - Any web browser 1. Install the Hugging Face tools. 2. Download the Space into a folder. 3. Move into that folder. 4. Open index.html in your browser. ```sh pip install -U huggingface_hub hf download Sattyam/guardrailagent-benign-check --repo-type space --local-dir guardrailagent-benign-check cd guardrailagent-benign-check ``` ### How to use it Open the page and read the headline rate first. Then scroll through the table of flagged prompts by XSTest type and the list of the nine flagged prompts. The repository also holds the recorded results and the scripts the author used. | Tool | What it does | | --- | --- | | Prompt knowledge base | A similarity search over embeddings of known prompts, with no model in the loop | | Material check | A model decides whether a prompt resembles a known way of attacking | | reproduce/measure.py | Runs the gate as released and records what each of its two tools returned | | reproduce/summarize.py | Builds summary.json from the recorded run | ### Example You wonder why a question about a drug was flagged. Open the page and find it among the nine prompts. It came from the knowledge base similarity check alone, so it does not depend on which model was used. ### For newcomers and experts A jailbreak filter should block tricks without blocking ordinary questions. XSTest is a set of safe prompts that look risky on purpose, such as asking how to kill a Python process. Start with the headline rate, then look at which prompt types were flagged most. The gate has two tools. A similarity search over known prompts needs no model, and a material check asks an LLM to match a prompt to a known attack method. Seven of the nine flags came from the material check, so they depend on the local qwen2.5:7b-instruct model used through Ollama. The paper used GPT-4.1-mini on its own 1,000 prompt set, which is not public yet, so the two rates measure different things. Take the 250 safe XSTest prompts → Run each through the released input gate → Log what the two tools returned → Summarize flags by prompt type ### Good to know The run shows 9 flags out of 250 prompts, and most of the flags fall in one type. Flagged prompts out of 25 in each XSTest type | Label | Value | | --- | --- | | Safe contexts | 5 flagged | | Homonyms | 1 flagged | | Figurative language | 1 flagged | | Safe targets | 1 flagged | | Definitions | 1 flagged | | Historical events | 0 flagged | | Privacy of public figures | 0 flagged | | Privacy of fictional characters | 0 flagged | - The page shows a recorded run and does not rerun the gate, because that needs an LLM. - It tests the released static gate only, not the full system the paper describes. - The small local model needed 921 retries over the 250 prompts, and every prompt still got a readable verdict. - License: CC BY 4.0 - Cost to run: viewing is free, and the author's rerun used a local model through Ollama with no API key - Data it reaches: the page's own files, and the public XSTest prompts it reports on - Not for: screening live prompts or re-running the gate in the browser ## README ````markdown # GuardrailAgent Benign Check Shows how often the GuardrailAgent input gate wrongly flags 250 harmless XSTest prompts as jailbreaks, with every prompt and decision. Useful for people who build or judge prompt filters. Made by Sattyam on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face. ## About What it is: This Space reports how often a jailbreak filter called GuardrailAgent flags harmless prompts by mistake. The author ran the released input gate on 250 safe prompts from the public XSTest set and published the recorded run, with every prompt and every decision. Who it suits: People who build, review or choose prompt filters and want a false alarm figure that anyone can check against a public set. ```table Industry | Who uses it | How they use it | Example AI safety | Guardrail developers | Measure false alarms on harmless prompts | Compare this run's rate with the rate on their own prompts Research | Readers of the paper | Test a published claim against a public set | Read how this run differs from the paper's 1.6% figure Trust and safety | Teams picking a filter | See which kinds of safe prompt trip a gate | Open the table of flagged prompts by type Education | Students of LLM security | Learn how a two part gate decides | Read the nine flagged prompts and how each was caught ``` If you want a tool that screens your own prompts live, this is a report, not a filter. Where to use it: It is a static page, so a browser is all you need. - A web browser, on the Space's page at Hugging Face with nothing to install - A downloaded copy on Windows, macOS or Linux, opened in any browser ```table Use case | Who | What they get back Check the headline rate | Reviewers | A flagged share of 3.6% with its 95% interval Browse flagged prompts | Filter builders | The nine prompts the gate flagged, grouped by type Read the method | Researchers | The model, versions and scripts used for the run ``` How to set it up: Viewing it online needs nothing. To keep a copy, you need two things. - Python and pip - Any web browser 1. Install the Hugging Face tools. 2. Download the Space into a folder. 3. Move into that folder. 4. Open index.html in your browser. ```sh pip install -U huggingface_hub hf download Sattyam/guardrailagent-benign-check --repo-type space --local-dir guardrailagent-benign-check cd guardrailagent-benign-check ``` How to use it: Open the page and read the headline rate first. Then scroll through the table of flagged prompts by XSTest type and the list of the nine flagged prompts. The repository also holds the recorded results and the scripts the author used. ```table Tool | What it does Prompt knowledge base | A similarity search over embeddings of known prompts, with no model in the loop Material check | A model decides whether a prompt resembles a known way of attacking reproduce/measure.py | Runs the gate as released and records what each of its two tools returned reproduce/summarize.py | Builds summary.json from the recorded run ``` Example: You wonder why a question about a drug was flagged. Open the page and find it among the nine prompts. It came from the knowledge base similarity check alone, so it does not depend on which model was used. For newcomers and experts: A jailbreak filter should block tricks without blocking ordinary questions. XSTest is a set of safe prompts that look risky on purpose, such as asking how to kill a Python process. Start with the headline rate, then look at which prompt types were flagged most. The gate has two tools. A similarity search over known prompts needs no model, and a material check asks an LLM to match a prompt to a known attack method. Seven of the nine flags came from the material check, so they depend on the local qwen2.5:7b-instruct model used through Ollama. The paper used GPT-4.1-mini on its own 1,000 prompt set, which is not public yet, so the two rates measure different things. ```flow Take the 250 safe XSTest prompts Run each through the released input gate Log what the two tools returned Summarize flags by prompt type ``` Good to know: The run shows 9 flags out of 250 prompts, and most of the flags fall in one type. ```chart Title: Flagged prompts out of 25 in each XSTest type Safe contexts: 5 flagged Homonyms: 1 flagged Figurative language: 1 flagged Safe targets: 1 flagged Definitions: 1 flagged Historical events: 0 flagged Privacy of public figures: 0 flagged Privacy of fictional characters: 0 flagged ``` - The page shows a recorded run and does not rerun the gate, because that needs an LLM. - It tests the released static gate only, not the full system the paper describes. - The small local model needed 921 retries over the 250 prompts, and every prompt still got a readable verdict. - License: CC BY 4.0 - Cost to run: viewing is free, and the author's rerun used a local model through Ollama with no API key - Data it reaches: the page's own files, and the public XSTest prompts it reports on - Not for: screening live prompts or re-running the gate in the browser ## On Hugging Face, 13 November 2026 - Likes: 1 - Built with: static - License: cc-by-4.0 ## Links - Space: https://huggingface.co/spaces/Sattyam/guardrailagent-benign-check - Files: https://huggingface.co/spaces/Sattyam/guardrailagent-benign-check/tree/main ## Try it Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [Sattyam/guardrailagent-benign-check on Hugging Face](https://huggingface.co/spaces/Sattyam/guardrailagent-benign-check) ## Compare with similar Listed Hugging Face Spaces in the same category that share the most tags with this one. | Product | Likes | Built with | License | Last change | | --- | --- | --- | --- | --- | | GuardrailAgent Benign Check (this page) | 1 | static | cc-by-4.0 | 2026-11-10 | | [Arena Leaderboard](https://mindket.com/p/613ca8f0-8146-4a48-87d6-43b8ee2ce3b6) | 5,039 | static | apache-2.0 | 2026-02-21 | | [Browser](https://mindket.com/p/d4e4cb47-8f53-4094-9a76-4fd29147a65b) | 533 | static | cc0-1.0 | 2026-10-18 | | [Model Pulse](https://mindket.com/p/017106fd-a3d1-4c20-944b-38f048d8b624) | 214 | static | apache-2.0 | 2026-11-04 | | [MTEB-BR Leaderboard](https://mindket.com/p/05f89589-d44c-4616-8efb-c50c9ba6ad94) | 126 | static | apache-2.0 | 2026-07-29 | | [Hub Stats](https://mindket.com/p/03d71953-8093-47a0-a444-c4f562d3071d) | 65 | static | apache-2.0 | 2026-11-12 | Numbers as Mindket last read them, on 2026-11-12. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-10 - First published: 2026-11-10 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open Sattyam/guardrailagent-benign-check on Hugging Face](https://huggingface.co/spaces/Sattyam/guardrailagent-benign-check) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/bded4065-73fb-4b5a-bd51-bcfe501926d0) - [Its page on Mindket](https://mindket.com/p/bded4065-73fb-4b5a-bd51-bcfe501926d0)