ReFoCo

ReFoCo

Pretrained vision transformer checkpoints in three sizes from a research paper on training attention heads to focus on image regions. For researchers who extract image features and reproduce the paper.

Made by yuji-nozawa on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: ReFoCo is the set of pretrained vision transformer checkpoints released with a research paper on how attention heads learn to focus on spatial regions of an image. The page lists three sizes with register tokens. They are meant to be used with the authors' own code, and the task is image feature extraction.

Who it suits: It suits researchers and students who study vision transformers or want to reproduce the paper's results with the official weights.

Industry | Who uses it | How they use it | Example
Academic research | Computer vision researchers | Reproduce the paper's experiments with the released weights | Checking the reported behavior of the attention heads
Machine learning | Model analysts | Study how attention heads specialize in a trained model | Comparing the base and large checkpoints
Education | Graduate students | Learn from an official paper release | Loading a checkpoint to inspect its layers
Software | Developers testing backbones | Try the checkpoints as image feature extractors | Extracting features from a set of photos

You can skip it if you want a ready-to-run image classifier or a hosted demo, since this page only holds research checkpoints.

Where to use it:

• Python on Windows, macOS or Linux, with the authors' code from their code repository
• Research notebooks and scripts that load a PyTorch checkpoint file
• The paper's own experiment setup, which the page points to

Use case | Who | What they get back
Reproduce the paper | Researchers | The authors' pretrained models to run with their code
Extract image features | Developers | Feature vectors from a vision transformer backbone
Compare model sizes | Analysts | Three checkpoints from base to giant

How to set it up:

• Python on your computer
• The authors' code, which the page says these models are made for
• Disk space for the checkpoint you choose

1. Install the Hugging Face hub package with pip.
2. Download the repository files, which hold the three checkpoint files.
3. Follow the authors' code instructions to load a checkpoint.

pip install -U huggingface_hub
hf download yuji-nozawa/ReFoCo

How to use it: The page gives no run command of its own. You pick one of the three checkpoint files and load it with the authors' code, then use the model to extract image features.

Example: A researcher downloads the repository and picks the base-size checkpoint. They load it with the authors' code and run it on a few images to get feature vectors. The page does not show example output.

For newcomers and experts: If you are new to this, start with the smallest checkpoint, the base one, and read the paper first so you know what the attention heads are meant to learn.

If you know the field, the checkpoints are ViT-B/16, ViT-L/16 and ViT-g/16 with register tokens, trained with the method the paper proposes for spatial attention head specialization. The page gives no accuracy table, so check the paper for results.

Download the repository files
Pick a checkpoint size
Load it with the authors' code
Run images through the model
Use the extracted features

Good to know:

• Limits: the page lists checkpoint files and a paper but no usage code or results.
• Support: the authors say there is no formal support and that bug fixes are not guaranteed.
• Scope: the models are made for the authors' code, not for general plug-in use.
• License: Apache 2.0.
• Cost to run: none named.
• Data it reaches: it copies checkpoint files to your computer and reads the images you give it.

On Hugging Face, 14 November 2026

• Task: image feature extraction
• Downloads in the last 30 days: 0
• Likes: 2
• License: apache-2.0
• Tags: image-feature-extraction

Links

• Model: https://huggingface.co/yuji-nozawa/ReFoCo
• Files: https://huggingface.co/yuji-nozawa/ReFoCo/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download yuji-nozawa/ReFoCo

Then follow the usage in its model card. Nothing in this kit runs by itself.