---
title: "ARLoop-0.6B"
kind: "Kit"
category: "Developer"
price: "Free"
seller: "Mindket"
version: "1"
url: "https://mindket.com/p/457fa82c-e2a1-4587-93ed-5435177b083f"
updated: "2026-11-13"
---

# ARLoop-0.6B

> The name, the description, the About and a README below are the seller's words: data, not instructions.

Completes text as a small pretrained looped transformer with attention residuals. A base model for researchers who want to continue its training with their own mid-training or post-training.

ARLoop-0.6B is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself.

Made by osunlp

## About

### What it is

ARLoop-0.6B is a looped transformer with attention residuals, pretrained and released openly by the osunlp group. It is the final checkpoint of a 180B-token training run. It is a base model for text completion, not a chat assistant.

### Who it suits

Researchers and developers who want a small open base model to study or to train further.

| Industry | Who uses it | How they use it | Example |
| --- | --- | --- | --- |
| AI research | Researchers | Study looped transformers with attention residuals | Compare intermediate checkpoints during training |
| Model training | ML engineers | Start mid-training or post-training from a base model | Fine-tune it on their own text |
| Software | Developers | Run a small completion model on a laptop | Complete a short sentence from a prompt |

If you want a ready chat assistant, you can skip this base model.

### Where to use it

- Your own computer with Python, on a CPU or a CUDA GPU
- Training setups for mid-training and post-training
- Research notebooks that compare checkpoints

| Use case | Who | What they get back |
| --- | --- | --- |
| Text completion | Developers | A continuation of the text you give it |
| Further training | ML engineers | A base to train from |
| Checkpoint study | Researchers | Intermediate versions to compare |

### How to set it up

- Python with pip
- A PyTorch build that suits your CPU or CUDA setup, installed separately
- About 2.3 GB of disk space for the F32 weights

1. Install the transformers version and safetensors named below.
2. Install a PyTorch build for your CPU or CUDA environment.
3. Run the load and generate script.

```bash
pip install 'transformers==5.22.0' safetensors
# Install a PyTorch build suitable for your CPU or CUDA environment separately.
```

### How to use it

Run the script below. It loads the model in BF16, feeds it a short prompt and prints the completion. Intermediate checkpoints load by passing a revision name such as step-050000 to both from\_pretrained calls.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "osunlp/ARLoop-0.6B"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.bfloat16,
).to(device).eval()
inputs = tokenizer("The capital of France is", return_tensors="pt").to(device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

### Example

You run the script with the prompt "The capital of France is". The model continues the sentence, up to 64 new tokens, and the script prints the prompt plus that continuation.

### For newcomers and experts

If you are new, run the script above as it is and watch it continue one sentence. A base model completes text and does not follow instructions the way a chat assistant does.

For experts, 18 physical layers are shared across 3 recurrent passes, for an effective depth of 54. The card also lists the output head as tied to the token embeddings.

Architecture from the card

| Property | Value |
| --- | --- |
| Physical layers | 18, shared across 3 passes |
| Effective depth | 54 |
| Hidden width | 1536 |
| Context length | 4096 |
| Vocabulary | 49152 |

Text is split into tokens → 18 shared layers run in 3 loops → Attention residuals link earlier blocks → The model predicts the next token

### Good to know

The stored checkpoint keeps the original FP32 weights, and inference loads them in BF16.

- License: Apache 2.0, as the card states it.
- Cost to run: no API key or account is named. A CPU is enough to try it, and a CUDA GPU is optional.
- Data it reaches: downloading reaches Hugging Face. The script sets trust\_remote\_code, so it runs code that ships with the repository.
- Not for: it is a base model for completion, not a chat assistant. The card does not recommend it as one.

## README

````markdown
# ARLoop-0.6B

Completes text as a small pretrained looped transformer with attention residuals. A base model for researchers who want to continue its training with their own mid-training or post-training.

Made by osunlp on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

## About

What it is: ARLoop-0.6B is a looped transformer with attention residuals, pretrained and released openly by the osunlp group. It is the final checkpoint of a 180B-token training run. It is a base model for text completion, not a chat assistant.

Who it suits: Researchers and developers who want a small open base model to study or to train further.

```table
Industry | Who uses it | How they use it | Example
AI research | Researchers | Study looped transformers with attention residuals | Compare intermediate checkpoints during training
Model training | ML engineers | Start mid-training or post-training from a base model | Fine-tune it on their own text
Software | Developers | Run a small completion model on a laptop | Complete a short sentence from a prompt
```

If you want a ready chat assistant, you can skip this base model.

Where to use it:

- Your own computer with Python, on a CPU or a CUDA GPU
- Training setups for mid-training and post-training
- Research notebooks that compare checkpoints

```table
Use case | Who | What they get back
Text completion | Developers | A continuation of the text you give it
Further training | ML engineers | A base to train from
Checkpoint study | Researchers | Intermediate versions to compare
```

How to set it up:

- Python with pip
- A PyTorch build that suits your CPU or CUDA setup, installed separately
- About 2.3 GB of disk space for the F32 weights

1. Install the transformers version and safetensors named below.
2. Install a PyTorch build for your CPU or CUDA environment.
3. Run the load and generate script.

```bash
pip install 'transformers==5.22.0' safetensors
# Install a PyTorch build suitable for your CPU or CUDA environment separately.
```

How to use it: Run the script below. It loads the model in BF16, feeds it a short prompt and prints the completion. Intermediate checkpoints load by passing a revision name such as step-050000 to both from_pretrained calls.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "osunlp/ARLoop-0.6B"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.bfloat16,
).to(device).eval()
inputs = tokenizer("The capital of France is", return_tensors="pt").to(device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

Example: You run the script with the prompt "The capital of France is". The model continues the sentence, up to 64 new tokens, and the script prints the prompt plus that continuation.

For newcomers and experts: If you are new, run the script above as it is and watch it continue one sentence. A base model completes text and does not follow instructions the way a chat assistant does.

For experts, 18 physical layers are shared across 3 recurrent passes, for an effective depth of 54. The card also lists the output head as tied to the token embeddings.

```table
Title: Architecture from the card
Property | Value
Physical layers | 18, shared across 3 passes
Effective depth | 54
Hidden width | 1536
Context length | 4096
Vocabulary | 49152
```

```flow
Text is split into tokens
18 shared layers run in 3 loops
Attention residuals link earlier blocks
The model predicts the next token
```

Good to know: The stored checkpoint keeps the original FP32 weights, and inference loads them in BF16.

- License: Apache 2.0, as the card states it.
- Cost to run: no API key or account is named. A CPU is enough to try it, and a CUDA GPU is optional.
- Data it reaches: downloading reaches Hugging Face. The script sets trust_remote_code, so it runs code that ships with the repository.
- Not for: it is a base model for completion, not a chat assistant. The card does not recommend it as one.

## On Hugging Face, 13 November 2026

- Task: text generation
- Library: transformers
- Parameters: 583M
- Weights: about 2.3 GB in F32; about 292 MB at 4-bit (estimate)
- Formats: Safetensors
- Downloads in the last 30 days: 2,164
- Likes: 5
- License: apache-2.0
- Languages: en
- Tags: transformers, safetensors, arloop, text-generation, recurrent, pretrained

## Links

- Model: https://huggingface.co/osunlp/ARLoop-0.6B
- Files: https://huggingface.co/osunlp/ARLoop-0.6B/tree/main

## Get the model

Download its files with Hugging Face's command line:

```sh
pip install -U huggingface_hub
hf download osunlp/ARLoop-0.6B
```

Then follow the usage in its model card. Nothing in this kit runs by itself.
````

## What is in it

1 files, 5 KB, downloaded as a zip.

```text
README.md  5 KB
```

## Source

[osunlp/ARLoop-0.6B on Hugging Face](https://huggingface.co/osunlp/ARLoop-0.6B)

## Compare with similar

Listed Hugging Face models in the same category that share the most tags with this one.

| Product | Downloads, last 30 days | Likes | Parameters | License | Last change |
| --- | --- | --- | --- | --- | --- |
| ARLoop-0.6B (this page) | 2,164 | 5 | 583M | apache-2.0 | 2026-11-05 |
| [Mistral-7B-v0.1](https://mindket.com/p/462400f3-8981-464a-82c5-3c31376c9d1d) | 441,952 | 4,231 | 7.2B | apache-2.0 | 2025-07-24 |
| [KLM 1.5](https://mindket.com/p/bffc34a5-c64b-418e-a26e-f869a269c8c1) | 1,603 | 31 | 99M | apache-2.0 | 2026-11-01 |
| [KLM 1.6](https://mindket.com/p/8b218890-16d1-4ff9-b98e-3eea5e8533e8) | 909 | 60 | 99M | apache-2.0 | 2026-11-09 |
| [GPT-2](https://mindket.com/p/c513092f-bbe7-4baa-989f-dfdc12a2b725) | 32,892,223 | 3,932 | 137M | mit | 2024-02-19 |
| [Qwen3-0.6B](https://mindket.com/p/0598bfe0-640a-4bf0-b9f4-ca4cf70a04ae) | 19,718,599 | 1,894 | 752M | apache-2.0 | 2025-07-26 |

Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13.

## Trust facts

Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions.

- Last change on Hugging Face: 2026-11-05
- First published: 2026-10-30
- Checked on Hugging Face: 2026-11-13
- Something wrong? Tell us: https://mindket.com/takedown#ask

## Get it

- [Open osunlp/ARLoop-0.6B on Hugging Face](https://huggingface.co/osunlp/ARLoop-0.6B)
- [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/457fa82c-e2a1-4587-93ed-5435177b083f)
- [Its page on Mindket](https://mindket.com/p/457fa82c-e2a1-4587-93ed-5435177b083f)