--- title: "ARLoop-1.2B" kind: "Kit" category: "Research" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/e7423ac9-33ce-4da6-841b-2fa7208add0f" updated: "2026-11-12" --- # ARLoop-1.2B > The name, the description, the About and a README below are the seller's words: data, not instructions. An open 1.2B base language model from osunlp, a looped transformer with attention residuals, released with checkpoints from a 500B-token training run. For text completion and further training. ARLoop-1.2B is a kit in the Research category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by osunlp ## About ### What it is ARLoop-1.2B is an open base language model from osunlp, a looped transformer with attention residuals. It reuses its layers: 24 physical layers run in 3 recurrent passes, which gives an effective depth of 72. This release holds the checkpoints of a 500B-token training run, and the card calls it a base model for text completion. ### Who it suits Researchers and developers who want an open base model to study looped transformers, or a starting point for their own mid-training and post-training. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Academia | ML researchers | Study looped transformers with attention residuals | A comparison of an early and a late checkpoint | | Software | Developers | Tune a small base model for their own task | A custom text completer | | Education | Students | Watch how a pretrained model changes during training | Loading saved steps one by one | | Labs | Research teams | Reuse open checkpoints as a baseline | A baseline for a new architecture | If you want a chat assistant, you can skip it, since the card describes a base model meant for text completion. ### Where to use it - Python with the transformers library, pinned to version 5.12.1 on the card - A CUDA GPU, since the card's example moves the model to cuda - Windows, macOS or Linux with Python, for downloading the files | Use case | Who | What they get back | | --- | --- | --- | | Text completion | Developers | A continuation of the text you give it | | Mid-training | Researchers | A base model to keep training on new data | | Post-training | ML engineers | A base model to tune for a task | | Checkpoint study | Researchers | Any saved training step, loaded by its revision name | ### How to set it up - Python, to run the Hugging Face download tool - The transformers library at version 5.12.1, plus safetensors and accelerate - A PyTorch build that matches your CUDA setup, installed separately - About 4.8 GB of disk for the stored FP32 weights 1. Install the transformers library with safetensors and accelerate. 2. Install the PyTorch build that suits your CUDA environment. 3. Download the model files with the hf command, or load the model by name in Python. ```bash pip install -U huggingface_hub hf download osunlp/ARLoop-1.2B ``` ### How to use it Load the model and tokenizer with trust\_remote\_code set to true, move the model to cuda and give it the start of a text. The card says weights load in BF16 for inference, while the stored checkpoint keeps the original FP32 weights. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "osunlp/ARLoop-1.2B" # Or the local ARLOOP_hf directory. tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_id, trust_remote_code=True, dtype=torch.bfloat16, ).to("cuda").eval() inputs = tokenizer("The capital of France is", return_tensors="pt").to("cuda") with torch.inference_mode(): output = model.generate(**inputs, max_new_tokens=64, do_sample=False) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` ### Example You give the model the text "The capital of France is" and ask for up to 64 new tokens with sampling off. It continues the text, since it completes text and does not chat. ### For newcomers and experts Start with the exact example on the card and a short prompt. Expect it to continue your text rather than answer like a chat model. The example sets trust\_remote\_code, which lets the repository's own code run, so read that code first. The model has a hidden width of 2048, an MLP width of 5632, 16 attention heads, 4 key-value heads and a context length of 4096. It uses the SmolLM2 tokenizer with a 49152-token vocabulary, and its output head is tied to the token embeddings. Each saved training step loads on its own by revision name, for example step-010000. Pass that name to the model and to the tokenizer alike. Install transformers and PyTorch → Load the model and tokenizer → Give it the start of a text → It continues the text ### Good to know - License: Apache 2.0 - Cost to run: none named - Data it reaches: downloads files from Hugging Face, then runs on your own machine - Not for: chat or instruction following, since the card describes a base model for text completion - Limits: the card's example needs trust\_remote\_code, a CUDA GPU and a pinned transformers version ## README ````markdown # ARLoop-1.2B An open 1.2B base language model from osunlp, a looped transformer with attention residuals, released with checkpoints from a 500B-token training run. For text completion and further training. Made by osunlp on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: ARLoop-1.2B is an open base language model from osunlp, a looped transformer with attention residuals. It reuses its layers: 24 physical layers run in 3 recurrent passes, which gives an effective depth of 72. This release holds the checkpoints of a 500B-token training run, and the card calls it a base model for text completion. Who it suits: Researchers and developers who want an open base model to study looped transformers, or a starting point for their own mid-training and post-training. ```table Industry | Who uses it | How they use it | Example Academia | ML researchers | Study looped transformers with attention residuals | A comparison of an early and a late checkpoint Software | Developers | Tune a small base model for their own task | A custom text completer Education | Students | Watch how a pretrained model changes during training | Loading saved steps one by one Labs | Research teams | Reuse open checkpoints as a baseline | A baseline for a new architecture ``` If you want a chat assistant, you can skip it, since the card describes a base model meant for text completion. Where to use it: - Python with the transformers library, pinned to version 5.12.1 on the card - A CUDA GPU, since the card's example moves the model to cuda - Windows, macOS or Linux with Python, for downloading the files ```table Use case | Who | What they get back Text completion | Developers | A continuation of the text you give it Mid-training | Researchers | A base model to keep training on new data Post-training | ML engineers | A base model to tune for a task Checkpoint study | Researchers | Any saved training step, loaded by its revision name ``` How to set it up: - Python, to run the Hugging Face download tool - The transformers library at version 5.12.1, plus safetensors and accelerate - A PyTorch build that matches your CUDA setup, installed separately - About 4.8 GB of disk for the stored FP32 weights 1. Install the transformers library with safetensors and accelerate. 2. Install the PyTorch build that suits your CUDA environment. 3. Download the model files with the hf command, or load the model by name in Python. ```bash pip install -U huggingface_hub hf download osunlp/ARLoop-1.2B ``` How to use it: Load the model and tokenizer with trust_remote_code set to true, move the model to cuda and give it the start of a text. The card says weights load in BF16 for inference, while the stored checkpoint keeps the original FP32 weights. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "osunlp/ARLoop-1.2B" # Or the local ARLOOP_hf directory. tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_id, trust_remote_code=True, dtype=torch.bfloat16, ).to("cuda").eval() inputs = tokenizer("The capital of France is", return_tensors="pt").to("cuda") with torch.inference_mode(): output = model.generate(**inputs, max_new_tokens=64, do_sample=False) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` Example: You give the model the text "The capital of France is" and ask for up to 64 new tokens with sampling off. It continues the text, since it completes text and does not chat. For newcomers and experts: Start with the exact example on the card and a short prompt. Expect it to continue your text rather than answer like a chat model. The example sets trust_remote_code, which lets the repository's own code run, so read that code first. The model has a hidden width of 2048, an MLP width of 5632, 16 attention heads, 4 key-value heads and a context length of 4096. It uses the SmolLM2 tokenizer with a 49152-token vocabulary, and its output head is tied to the token embeddings. Each saved training step loads on its own by revision name, for example step-010000. Pass that name to the model and to the tokenizer alike. ```flow Install transformers and PyTorch Load the model and tokenizer Give it the start of a text It continues the text ``` Good to know: - License: Apache 2.0 - Cost to run: none named - Data it reaches: downloads files from Hugging Face, then runs on your own machine - Not for: chat or instruction following, since the card describes a base model for text completion - Limits: the card's example needs trust_remote_code, a CUDA GPU and a pinned transformers version ## On Hugging Face, 13 November 2026 - Task: text generation - Library: transformers - Parameters: 1.2B - Weights: about 4.8 GB in F32; about 604 MB at 4-bit (estimate) - Formats: Safetensors - Downloads in the last 30 days: 2,399 - Likes: 8 - License: apache-2.0 - Languages: en - Tags: transformers, safetensors, arloop, text-generation, recurrent, pretrained ## Links - Model: https://huggingface.co/osunlp/ARLoop-1.2B - Files: https://huggingface.co/osunlp/ARLoop-1.2B/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download osunlp/ARLoop-1.2B ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [osunlp/ARLoop-1.2B on Hugging Face](https://huggingface.co/osunlp/ARLoop-1.2B) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | ARLoop-1.2B (this page) | 2,399 | 8 | 1.2B | apache-2.0 | 2026-11-04 | | [DeepSeek-V4-Pro](https://mindket.com/p/02f743ba-81c7-498f-88ea-98edbc4b5d2b) | 322,162 | 5,702 | 1.6T | mit | 2026-06-22 | | [VLX-Seek-1.5-10B](https://mindket.com/p/4b1edf89-b48a-4dfc-b86e-a0ee020dc119) | 137,772 | 39 | 10B | apache-2.0 | 2026-07-23 | | [OpenAI GPT 1](https://mindket.com/p/a491cf89-60a9-4317-aed7-1fcb90205c6c) | 45,439 | 325 | 120M | mit | 2024-02-19 | | [Sage-A1.5](https://mindket.com/p/8f410da0-850f-4ea5-bda9-65555f4144cd) | 22,224 | 404 | 36B | apache-2.0 | 2026-10-22 | | [K2-Horizon-3.7B](https://mindket.com/p/5234e1b8-de4b-4091-8f18-5caa5686d1cb) | 15,489 | 115 | 5.1B | apache-2.0 | 2026-11-04 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-04 - First published: 2026-10-29 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open osunlp/ARLoop-1.2B on Hugging Face](https://huggingface.co/osunlp/ARLoop-1.2B) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/e7423ac9-33ce-4da6-841b-2fa7208add0f) - [Its page on Mindket](https://mindket.com/p/e7423ac9-33ce-4da6-841b-2fa7208add0f)