Mindket

J++ Lenses

Newversion 1 · updated

Overview

Shows which words a language model is leaning toward at its middle layers. Lens files for six open models, made for interpretability researchers who want to inspect hidden states.

J++ Lenses is a kit in the Research category, listed by Mindket. It is free. This is version 1, last updated on . Download it on this page, a zip of its files. Nothing in it runs by itself.

At a glance

You need
  • Python with the huggingface_hub package
  • The code released with the paper, which provides workspace_lens
  • Disk space for the lens file, 6.6 GB in fp32 for Qwen3.6-27B, plus the base model itself
Runs in
  • Python scripts that use the paper's workspace_lens code
  • Windows, macOS or Linux with Python installed
  • Next to the base model, loaded from Hugging Face
Start with
pip install -U huggingface_hub
hf download koayon/jpp-lenses

Platform

WindowsOperating system
macOSOperating system
LinuxOperating system

About

What it is

A set of workspace lens files for studying language models, released with a research paper on Jacobian filtering. A lens takes the hidden state at a middle layer and shows which tokens the model is leaning toward saying. It uses one linear map per layer, then the model's own unembedding step.

Who it suits

Researchers who study what happens inside large language models, and engineers who want to inspect a model's intermediate guesses. You should be comfortable with Python and PyTorch.

IndustryWho uses itHow they use itExample
Academic researchInterpretability researchersRead out middle-layer states of the supported modelsTracking the layer where a concept first shows up
AI labsModel analysis teamsCompare what different models lean toward at each depthChecking Qwen and Gemma readouts side by side
EducationStudents of model internalsReproduce the paper's readouts on a small promptRunning the quickstart prompt on one layer
Model reviewAuditorsInspect hidden states behind a given answerAsking why an answer differs from earlier readouts

You can skip it if you only chat with models, or if your model is not one of the six listed.

Where to use it

  • Python scripts that use the paper's workspace_lens code
  • Windows, macOS or Linux with Python installed
  • Next to the base model, loaded from Hugging Face
Use caseWhoWhat they get back
Read a layer's hidden state as tokensResearchersLens logits next to the model's own logits
Compare layers of one promptAnalystsReadouts at several chosen layers
Check a result against the paperStudentsThe same layers the paper reads out

How to set it up

  • Python with the huggingface_hub package
  • The code released with the paper, which provides workspace_lens
  • Disk space for the lens file, 6.6 GB in fp32 for Qwen3.6-27B, plus the base model itself
  1. Install the Hugging Face client.
  2. Download the lens files.
  3. Get the paper's released code so workspace_lens is available.
  4. Load a lens file and the matching base model in Python.
pip install -U huggingface_hub
hf download koayon/jpp-lenses
from workspace_lens import get_hf_model
from workspace_lens.lenses.base_lens import BaseLens

lens = BaseLens.from_pretrained("koayon/jpp-lenses", filename="qwen3.6-27b/lens.pt")
model = get_hf_model("Qwen/Qwen3.6-27B", attn_implementation="sdpa")
lens_logits, model_logits, input_ids = lens.apply(
    model, "Fact: The number of legs on the animal that spins webs is",
    layers=[16, 32, 48], token_positions_for_residuals=[-1],
)

How to use it

Load a lens and its base model, then call apply with a prompt and the layers you want to read. You get the lens's guesses and the model's own logits back. The released code also includes a quickstart script that shows the best-ranked tokens at every layer for a prompt you give it.

ToolWhat it does
BaseLens.from_pretrainedLoads one lens file from the repository
get_hf_modelLoads the base model the lens was fitted on
lens.applyReturns lens logits, model logits and input ids for chosen layers

Example

On the paper's first figure prompt, a question about how many chambers the human heart has, the lens already shows heart, cardiac and 心脏 as its guesses at layer 24 and later layers. The model itself answers with the number 4. You see the concept appear in the middle layers before the final answer.

For newcomers and experts

A language model builds its answer through many layers, and a lens lets you peek at the middle ones. A first try is the snippet above on the Qwen3.6-27B lens, reading layers 16, 32 and 48.

If you know the field: each layer gets a Jacobian that maps its residual stream to the final block. A k-means router with 8 clusters assigns each position to an expert Jacobian, and the experts are combined with signed weights tuned for recall at 10. The paper reads seven evenly spaced depths per model, such as layers 8, 16, 24, 32, 40, 48 and 56 for Qwen3.6-27B.

  1. Hidden state at a layer
  2. One linear map per layer
  3. Final-layer space
  4. Model's unembedding
  5. Readable tokens

Good to know

Recall at 10 differs by model. The Gemma 4 31B figure leaves out the poetry task, so it is not in the chart.

Recall@10 by base model, in percent
Qwen3.6-27B55.2%
Qwen3.5-9B55.0%
Olmo-3-1125-32B50.0%
Qwen3.5-122B-A10B48.4%
DeepSeek-V4-Flash61.4%
  • File format: each lens file is a dictionary saved with torch, read back by torch in weights-only mode
  • Fitting data: 64 WikiText-103 records, 128 tokens each
  • License: Apache 2.0, and the lens files build on Qwen3.6-27B for that model
  • Cost to run: none named
  • Data it reaches: lens files downloaded from Hugging Face and the base model you load, nothing else named
  • Not for: models outside the six listed, since each lens is fitted to one model

README and files

README

J++ Lenses

Shows which words a language model is leaning toward at its middle layers. Lens files for six open models, made for interpretability researchers who want to inspect hidden states.

Made by koayon on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: A set of workspace lens files for studying language models, released with a research paper on Jacobian filtering. A lens takes the hidden state at a middle layer and shows which tokens the model is leaning toward saying. It uses one linear map per layer, then the model's own unembedding step.

Who it suits: Researchers who study what happens inside large language models, and engineers who want to inspect a model's intermediate guesses. You should be comfortable with Python and PyTorch.

Industry | Who uses it | How they use it | Example
Academic research | Interpretability researchers | Read out middle-layer states of the supported models | Tracking the layer where a concept first shows up
AI labs | Model analysis teams | Compare what different models lean toward at each depth | Checking Qwen and Gemma readouts side by side
Education | Students of model internals | Reproduce the paper's readouts on a small prompt | Running the quickstart prompt on one layer
Model review | Auditors | Inspect hidden states behind a given answer | Asking why an answer differs from earlier readouts

You can skip it if you only chat with models, or if your model is not one of the six listed.

Where to use it:

  • Python scripts that use the paper's workspace_lens code
  • Windows, macOS or Linux with Python installed
  • Next to the base model, loaded from Hugging Face
Use case | Who | What they get back
Read a layer's hidden state as tokens | Researchers | Lens logits next to the model's own logits
Compare layers of one prompt | Analysts | Readouts at several chosen layers
Check a result against the paper | Students | The same layers the paper reads out

How to set it up:

  • Python with the huggingface_hub package
  • The code released with the paper, which provides workspace_lens
  • Disk space for the lens file, 6.6 GB in fp32 for Qwen3.6-27B, plus the base model itself
  1. Install the Hugging Face client.
  2. Download the lens files.
  3. Get the paper's released code so workspace_lens is available.
  4. Load a lens file and the matching base model in Python.
pip install -U huggingface_hub
hf download koayon/jpp-lenses
from workspace_lens import get_hf_model
from workspace_lens.lenses.base_lens import BaseLens

lens = BaseLens.from_pretrained("koayon/jpp-lenses", filename="qwen3.6-27b/lens.pt")
model = get_hf_model("Qwen/Qwen3.6-27B", attn_implementation="sdpa")
lens_logits, model_logits, input_ids = lens.apply(
    model, "Fact: The number of legs on the animal that spins webs is",
    layers=[16, 32, 48], token_positions_for_residuals=[-1],
)

How to use it: Load a lens and its base model, then call apply with a prompt and the layers you want to read. You get the lens's guesses and the model's own logits back. The released code also includes a quickstart script that shows the best-ranked tokens at every layer for a prompt you give it.

Tool | What it does
BaseLens.from_pretrained | Loads one lens file from the repository
get_hf_model | Loads the base model the lens was fitted on
lens.apply | Returns lens logits, model logits and input ids for chosen layers

Example: On the paper's first figure prompt, a question about how many chambers the human heart has, the lens already shows heart, cardiac and 心脏 as its guesses at layer 24 and later layers. The model itself answers with the number 4. You see the concept appear in the middle layers before the final answer.

For newcomers and experts: A language model builds its answer through many layers, and a lens lets you peek at the middle ones. A first try is the snippet above on the Qwen3.6-27B lens, reading layers 16, 32 and 48.

If you know the field: each layer gets a Jacobian that maps its residual stream to the final block. A k-means router with 8 clusters assigns each position to an expert Jacobian, and the experts are combined with signed weights tuned for recall at 10. The paper reads seven evenly spaced depths per model, such as layers 8, 16, 24, 32, 40, 48 and 56 for Qwen3.6-27B.

Hidden state at a layer
One linear map per layer
Final-layer space
Model's unembedding
Readable tokens

Good to know: Recall at 10 differs by model. The Gemma 4 31B figure leaves out the poetry task, so it is not in the chart.

Title: Recall@10 by base model, in percent
Qwen3.6-27B: 55.2%
Qwen3.5-9B: 55.0%
Olmo-3-1125-32B: 50.0%
Qwen3.5-122B-A10B: 48.4%
DeepSeek-V4-Flash: 61.4%
  • File format: each lens file is a dictionary saved with torch, read back by torch in weights-only mode
  • Fitting data: 64 WikiText-103 records, 128 tokens each
  • License: Apache 2.0, and the lens files build on Qwen3.6-27B for that model
  • Cost to run: none named
  • Data it reaches: lens files downloaded from Hugging Face and the base model you load, nothing else named
  • Not for: models outside the six listed, since each lens is fitted to one model

On Hugging Face, 16 November 2026

  • Downloads in the last 30 days: 0
  • Likes: 11
  • License: apache-2.0
  • Based on: Qwen/Qwen3.6-27B
  • Tags: interpretability, workspace-lens, jacobian-lens

Links

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download koayon/jpp-lenses

Then follow the usage in its model card. Nothing in this kit runs by itself.

Files
README.md  6 KB

1 file, 6 KB, downloaded as a zip.

What’s new

  • Version 1 · 2026-11-15

Reviews

No ratings yet

Write a review

No reviews yet.

More from Mindket

See all from Mindket

all-mpnet-base-v2

sentence-transformers · Kit

Turns sentences and paragraphs into 768-number vectors so you can search, cluster and compare text by meaning. It suits developers and data teams working in En…

New

multilingual-e5-base

intfloat · Kit

Turns text in about 100 languages into vectors so you can compare meaning across them, for search and matching. Made for developers who build multilingual retr…

New

sprite-gen

aldegad · Skill

Turns one base drawing of a character into game-ready sprites, either an atlas with a manifest or transparent motion loops, for game developers working in Code…

New4097

Similar in Research

See all in Research

Claude Code skills that guide academic research from planning to publication, with your confirmation at every stage, integrity checks and an optional citation…

New57.1k

GAIA Leaderboard

gaia-benchmark · Kit

Shows a leaderboard built around the GAIA dataset, so people can compare AI systems on one shared benchmark. It is a Gradio app from the gaia-benchmark group.

New