Text Surprisal Editor

Text Surprisal Editor

Colors each word of your text by how unexpected a tiny 4M-parameter language model finds it. The model has no Transformer blocks. A hackathon project for writers and researchers.

Made by build-small-hackathon on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: Text Surprisal Editor is a Gradio app that colors each word of a text by how unexpected a small language model finds it in context. The model has about 4 million parameters and no Transformer blocks or self-attention, and it carries context through learned state changes. It was built for Hugging Face's Build Small Hackathon in 2026.

Who it suits: Writers, students and language-model researchers who want to see word-level surprise on screen will get the most from it. The README names no customers, so the rows below are judgment from what the app does.

Industry | Who uses it | How they use it | Example
Research | Language-model researchers | Study an attention-free design and see where it expects or doubts words | Comparing an ordinary sentence with an odd one
Writing | Writers and editors | Spot words that read as unexpected in their context | Pasting a paragraph and looking at the heatmap
Education | Teachers and students | Show what surprisal means for each word in a text | Using a short passage to explain the idea in class

You can skip it if you need grammar fixes or a chat model, because the source describes a heatmap of word surprise and nothing else.

Where to use it: Use it in a browser tab, or run a copy on your own computer.

• A web browser: open the Space page on Hugging Face
• Windows, macOS or Linux: run a copy with Python 3.13

Use case | Who | What they get back
Check word surprise | Writers and editors | A heatmap showing which words the model finds unexpected
Study a small model | Researchers | A working example of a model with no attention layers
Run it locally | Developers with Python 3.13 | The same Gradio app on their own machine, using the weights file in the repository

How to set it up: Using the Space online needs nothing. A local copy needs Python 3.13 and the listed packages. The model file final_infer.pt is already in the repository.

• Python 3.13 with pip
• Gradio 6.16.0
• The packages in requirements.txt

1. Install the Hugging Face hub package.
2. Download the Space's files into a folder.
3. Move into that folder.
4. Install Gradio 6.16.0, then the listed packages.
5. Start the app with Python.

pip install -U huggingface_hub
hf download build-small-hackathon/pgsm-text-surprisal-editor --repo-type space --local-dir pgsm-text-surprisal-editor
cd pgsm-text-surprisal-editor
pip install gradio==6.16.0
pip install -r requirements.txt
python app.py

How to use it: The README gives no click-by-click guide. By its description, you enter text and the app colors each word by how unexpected it is, with colors that adjust to the text. Locally, start it with one command.

python app.py

Example: Paste a short paragraph about cooking that ends with the name of a planet. The odd word should stand out in the heatmap as more surprising than its neighbors, though the README shows no sample output.

For newcomers and experts: Surprisal is a measure of how unexpected a word is given the words before it. Start with one ordinary sentence and one odd sentence, paste each in, and compare the colors.

The model is an attention-free design that the author calls ExactState Memory, or PGSM, with sparse RoPE-based state transfer and a compact vocabulary of about 2k entries. It predicts words and spans rather than single tokens, and the author trained it on about 19 billion FineWeb-Edu tokens. Context passes through learned state transitions instead of pairwise attention. Scores therefore reflect what this tiny model expects, not what a large one would.

You enter text in the app
The model reads it through learned states
Each word gets a surprisal score
Scores are drawn as a heatmap

Good to know:

• License: Apache-2.0
• Cost to run: no API key or paid service named, and the README says inference runs locally with no external model APIs
• Data it reaches: the text you enter goes to the Space's own model, since no outside model service is used
• Not for: replacing a Transformer language model, since the README calls it an experiment in a different design

On Hugging Face, 13 November 2026

• Likes: 2
• Built with: gradio
• License: apache-2.0

Links

• Space: https://huggingface.co/spaces/build-small-hackathon/pgsm-text-surprisal-editor
• Files: https://huggingface.co/spaces/build-small-hackathon/pgsm-text-surprisal-editor/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.