--- title: "roberta-base-adasplash" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/c832620a-ddeb-4ff7-98b0-5866c3e63339" updated: "2026-11-13" --- # roberta-base-adasplash > The name, the description, the About and a README below are the seller's words: data, not instructions. A RoBERTa Base sentiment classifier whose attention is swapped for a sparse AdaSplash version, which zeroes out 40% of attention weights to cut time and memory. For developers testing efficient attention. roberta-base-adasplash is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by Hooshaai ## About ### What it is This is RoBERTa Base, fine-tuned on the GLUE SST-2 sentiment task, with its attention layers replaced by AdaSplash attention from the neuralops library. AdaSplash uses a sparse projection that sets a share of attention weights to exactly zero. The card reports 40% zero attention weights in all 12 encoder blocks. It adds about 0.94% more parameters through small LoRA adapters, bringing the model to roughly 125.8 million parameters. ### Who it suits Developers and researchers who want to see how sparse attention changes speed, memory and accuracy in a familiar English text classifier. It is an experiment more than a drop-in upgrade. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Research | ML researchers | Study sparse attention in an encoder model | Comparing sparse and dense attention on one task | | Software | Inference engineers | Test memory and latency savings | Measuring peak memory for a classifier | | Education | Students | Learn how entmax attention differs from softmax | Reading the architecture notes and running the sample | | Product teams | NLP developers | Try an efficient sentiment classifier | Scoring short English reviews as positive or negative | Anyone who needs the highest sentiment accuracy can skip it. The card shows the dense baseline scoring higher on its test. ### Where to use it - Python with Transformers, PyTorch and the neuralops library. - Windows, macOS or Linux with Python, for downloading the files. - A GPU, since the card's measurements used an NVIDIA RTX 4070 with Triton kernels. | Use case | Who | What they get back | | --- | --- | --- | | Sentiment classification | Developers | A positive or negative prediction for English text | | Efficiency testing | Engineers | Lower attention time and memory than the dense model | | Attention swapping | Researchers | An existing RoBERTa converted to AdaSplash in one call | ### How to set it up You need Python and a few libraries. - Python with torch and transformers - The neuralops library, version 1.4.3 per the card - English text to classify - A GPU, which is optional for trying the sample 1. Install the Hugging Face tool. 2. Download the model files. 3. Install neuralops and run the quickstart. ```bash pip install -U huggingface_hub hf download Hooshaai/roberta-base-adasplash ``` ### How to use it Load the tokenizer and the model, tokenize a sentence, and take the highest logit as the prediction. You can also convert any RoBERTa to AdaSplash in one call. ```python import torch from transformers import AutoModelForSequenceClassification, AutoTokenizer from neuralops import replace_attention # 1. Load tokenizer and model from Hugging Face Hub tokenizer = AutoTokenizer.from_pretrained("roberta-base") model = AutoModelForSequenceClassification.from_pretrained("Hooshaai/roberta-base-adasplash") # 2. Or surgically inject AdaSplash into any RoBERTa in 1 line base_model = AutoModelForSequenceClassification.from_pretrained("roberta-base") adasplash_model = replace_attention(base_model, "adasplash_attention") # 3. Predict sentiment inputs = tokenizer("The sparse entmax projection preserves salient features with extreme precision!", return_tensors="pt") with torch.no_grad(): logits = model(**inputs).logits prediction = logits.argmax(-1).item() ``` | Tool | What it does | | --- | --- | | AutoTokenizer | Turns a sentence into model inputs | | AutoModelForSequenceClassification | Loads the classifier and returns logits | | replace\_attention | Swaps a model's attention for AdaSplash attention | ### Example Give the model a short English sentence that expresses an opinion, as the sample does. It returns logits, and the index of the largest one is the predicted sentiment class. ### For newcomers and experts If you are new, a sentiment classifier reads a sentence and says whether it sounds positive or negative. Run the quickstart on one sentence first. The rest of the card is about how its attention was made more efficient. For experts, the model uses Tsallis 1.5-entmax in place of softmax, so attention weights can be exactly zero. The query, key, value and output projections are rank-16 LoRA adapters over frozen pretrained weights. The card reports 94.38% accuracy for the dense baseline, 75.46% for zero-shot AdaSplash and 82.10% after distillation. Start from RoBERTa Base fine-tuned on SST-2 → Swap in AdaSplash attention with entmax → Add rank-16 LoRA adapters → Distill to recover accuracy → Classify text with 40% sparse attention SST-2 accuracy from the card | Label | Value | | --- | --- | | Dense baseline | 94.38% | | Zero-shot AdaSplash | 75.46% | | After distillation | 82.10% | ### Good to know The points below cover limits and costs. - Accuracy trade-off: the distilled model scores lower than the dense baseline, though the card reports about 1.52 times faster inference steps and 33% less peak memory on its test machine. - License: MIT. - Cost to run: no API key or paid service is named. - Data it reads: only the text you pass in. Downloads come from Hugging Face. - Not for: uses that need the dense model's accuracy, or languages other than English. ## README ````markdown # roberta-base-adasplash A RoBERTa Base sentiment classifier whose attention is swapped for a sparse AdaSplash version, which zeroes out 40% of attention weights to cut time and memory. For developers testing efficient attention. Made by Hooshaai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is RoBERTa Base, fine-tuned on the GLUE SST-2 sentiment task, with its attention layers replaced by AdaSplash attention from the neuralops library. AdaSplash uses a sparse projection that sets a share of attention weights to exactly zero. The card reports 40% zero attention weights in all 12 encoder blocks. It adds about 0.94% more parameters through small LoRA adapters, bringing the model to roughly 125.8 million parameters. Who it suits: Developers and researchers who want to see how sparse attention changes speed, memory and accuracy in a familiar English text classifier. It is an experiment more than a drop-in upgrade. ```table Industry | Who uses it | How they use it | Example Research | ML researchers | Study sparse attention in an encoder model | Comparing sparse and dense attention on one task Software | Inference engineers | Test memory and latency savings | Measuring peak memory for a classifier Education | Students | Learn how entmax attention differs from softmax | Reading the architecture notes and running the sample Product teams | NLP developers | Try an efficient sentiment classifier | Scoring short English reviews as positive or negative ``` Anyone who needs the highest sentiment accuracy can skip it. The card shows the dense baseline scoring higher on its test. Where to use it: - Python with Transformers, PyTorch and the neuralops library. - Windows, macOS or Linux with Python, for downloading the files. - A GPU, since the card's measurements used an NVIDIA RTX 4070 with Triton kernels. ```table Use case | Who | What they get back Sentiment classification | Developers | A positive or negative prediction for English text Efficiency testing | Engineers | Lower attention time and memory than the dense model Attention swapping | Researchers | An existing RoBERTa converted to AdaSplash in one call ``` How to set it up: You need Python and a few libraries. - Python with torch and transformers - The neuralops library, version 1.4.3 per the card - English text to classify - A GPU, which is optional for trying the sample 1. Install the Hugging Face tool. 2. Download the model files. 3. Install neuralops and run the quickstart. ```bash pip install -U huggingface_hub hf download Hooshaai/roberta-base-adasplash ``` How to use it: Load the tokenizer and the model, tokenize a sentence, and take the highest logit as the prediction. You can also convert any RoBERTa to AdaSplash in one call. ```python import torch from transformers import AutoModelForSequenceClassification, AutoTokenizer from neuralops import replace_attention # 1. Load tokenizer and model from Hugging Face Hub tokenizer = AutoTokenizer.from_pretrained("roberta-base") model = AutoModelForSequenceClassification.from_pretrained("Hooshaai/roberta-base-adasplash") # 2. Or surgically inject AdaSplash into any RoBERTa in 1 line base_model = AutoModelForSequenceClassification.from_pretrained("roberta-base") adasplash_model = replace_attention(base_model, "adasplash_attention") # 3. Predict sentiment inputs = tokenizer("The sparse entmax projection preserves salient features with extreme precision!", return_tensors="pt") with torch.no_grad(): logits = model(**inputs).logits prediction = logits.argmax(-1).item() ``` ```table Tool | What it does AutoTokenizer | Turns a sentence into model inputs AutoModelForSequenceClassification | Loads the classifier and returns logits replace_attention | Swaps a model's attention for AdaSplash attention ``` Example: Give the model a short English sentence that expresses an opinion, as the sample does. It returns logits, and the index of the largest one is the predicted sentiment class. For newcomers and experts: If you are new, a sentiment classifier reads a sentence and says whether it sounds positive or negative. Run the quickstart on one sentence first. The rest of the card is about how its attention was made more efficient. For experts, the model uses Tsallis 1.5-entmax in place of softmax, so attention weights can be exactly zero. The query, key, value and output projections are rank-16 LoRA adapters over frozen pretrained weights. The card reports 94.38% accuracy for the dense baseline, 75.46% for zero-shot AdaSplash and 82.10% after distillation. ```flow Start from RoBERTa Base fine-tuned on SST-2 Swap in AdaSplash attention with entmax Add rank-16 LoRA adapters Distill to recover accuracy Classify text with 40% sparse attention ``` ```chart Title: SST-2 accuracy from the card Dense baseline: 94.38% Zero-shot AdaSplash: 75.46% After distillation: 82.10% ``` Good to know: The points below cover limits and costs. - Accuracy trade-off: the distilled model scores lower than the dense baseline, though the card reports about 1.52 times faster inference steps and 33% less peak memory on its test machine. - License: MIT. - Cost to run: no API key or paid service is named. - Data it reads: only the text you pass in. Downloads come from Hugging Face. - Not for: uses that need the dense model's accuracy, or languages other than English. ## On Hugging Face, 13 November 2026 - Task: text classification - Library: transformers - Downloads in the last 30 days: 0 - Likes: 2 - License: mit - Languages: en - Tags: transformers, neuralops, adasplash, entmax, sparse-attention, subquadratic-attention, roberta, sst2, text-classification ## Links - Model: https://huggingface.co/Hooshaai/roberta-base-adasplash - Files: https://huggingface.co/Hooshaai/roberta-base-adasplash/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download Hooshaai/roberta-base-adasplash ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [Hooshaai/roberta-base-adasplash on Hugging Face](https://huggingface.co/Hooshaai/roberta-base-adasplash) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | roberta-base-adasplash (this page) | 0 | 2 | — | mit | 2026-10-19 | | [svd-linear-attention-roberta-base-adasplash](https://mindket.com/p/5d082590-6230-4495-9ae7-609a38b0f84b) | 36 | 2 | — | mit | 2026-10-19 | | [roberta-base-fwsvd-rank32](https://mindket.com/p/685e228c-6198-4c7e-8266-d9b812f3435b) | 0 | 2 | — | mit | 2026-10-19 | | [distilbert-sst2-rpc](https://mindket.com/p/65d38525-4461-4f95-b181-9d1a42d9df5f) | 0 | 2 | — | mit | 2026-10-19 | | [bge-reranker-v2-m3](https://mindket.com/p/5bc2430e-32f8-4153-8615-2e4e3bad70ea) | 14,685,230 | 1,301 | 568M | apache-2.0 | 2024-06-24 | | [deberta-v3-base-prompt-injection-v2](https://mindket.com/p/bbb131a8-e7e6-491b-b19e-25098c98a48d) | 476,298 | 129 | 184M | apache-2.0 | 2026-07-09 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-19 - First published: 2026-10-19 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open Hooshaai/roberta-base-adasplash on Hugging Face](https://huggingface.co/Hooshaai/roberta-base-adasplash) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/c832620a-ddeb-4ff7-98b0-5866c3e63339) - [Its page on Mindket](https://mindket.com/p/c832620a-ddeb-4ff7-98b0-5866c3e63339)