svd-linear-attention-roberta-base-adasplash
Hooshaai · Kit
Classifies English text, such as SST-2 sentiment, with a RoBERTa Base model whose attention is made sparse by AdaSplash, trading some accuracy for lower latenc…
New
AI projects from Hugging Face
Categories and filters
Listings
roberta-base-fwsvd-rank32
Hooshaai · Kit
Classifies English text with a RoBERTa Base model shrunk to about 82M parameters by Fisher-weighted SVD at rank 32, trading some accuracy for lower latency and…
New
roberta-base-adasplash
Hooshaai · Kit
A RoBERTa Base sentiment classifier whose attention is swapped for a sparse AdaSplash version, which zeroes out 40% of attention weights to cut time and memory…
New
distilbert-sst2-rpc
Hooshaai · Kit
A DistilBERT text classifier fine-tuned on GLUE SST-2 that swaps in a compressed random-projection attention to halve the scoring work. An experiment for resea…
New
qwen-2.5-0.5b-block-transformer-hiba-wikitext2
tahamajs · Kit
Swaps 18 of the 24 attention layers in Qwen2.5-0.5B for a hierarchical block attention operator, then distills it on WikiText-2. A research checkpoint for stud…
New
qwen-2.5-0.5b-bam-umoe-wikitext2
tahamajs · Kit
Swaps 18 of the 24 attention layers in Qwen2.5-0.5B for a mixture-of-experts attention block, then distills it on WikiText-2. A research checkpoint for studyin…
New