--- title: "OLMoE-1B-7B-0924" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/e29f1b5e-c361-4e68-bf44-e60ac0a18edb" updated: "2026-11-13" --- # OLMoE-1B-7B-0924 > The name, the description, the About and a README below are the seller's words: data, not instructions. Continues English text with a small-cost Mixture-of-Experts model that has 1B active and 7B total parameters. Fully open code, data and checkpoints make it a fit for researchers and developers using transformers. OLMoE-1B-7B-0924 is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by allenai ## About ### What it is OLMoE-1B-7B-0924 is an open language model from allenai, released in September 2024. It is a Mixture-of-Experts design with 1B active parameters out of 7B in total, so each token costs about as much as a 1B model. The card calls it fully open and says it holds its own against larger models such as Llama2-13B. ### Who it suits Researchers who want to inspect every training stage fit best, and so do developers who want to try a Mixture-of-Experts model through transformers. Students of how language models are trained can follow the released stages. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Research | Language-model researchers | Study a model whose code, data and logs are published for each training stage | Loading an early checkpoint by passing its name as a revision | | Software | Developers | Run a Mixture-of-Experts model with 1B active parameters in their own Python code | Generating a continuation from a short text prompt | | Education | Students of language-model training | Follow the pretraining, supervised fine-tuning and preference stages with released checkpoints | Comparing the main checkpoint with its tuned Instruct version | You can skip it if you need replies in a language other than English, since the record lists English only. ### Where to use it It runs in Python through the transformers library and torch. - Windows, macOS or Linux, where the files download through Python - A CUDA GPU when one is present, otherwise the CPU, as the card's sample chooses - Your own scripts and notebooks that load models with transformers | Use case | Who | What they get back | | --- | --- | --- | | Text continuation | Developers | A passage that carries on from a prompt | | Checkpoint study | Researchers | Models from different points in training | | Model comparison | Students | A small open model to set beside other 1B-class ones | ### How to set it up You need Python with a few libraries and room for the weights. - Python on Windows, macOS or Linux - torch - transformers installed from source, which the card asks for until a later release carries the needed change - About 13.8 GB of disk for the BF16 weights, or about 3.5 GB at 4-bit, which is an estimate 1. Install the Hugging Face hub library. 2. Download the model files with the hf command. 3. Install torch and transformers from source, as the card says. ```sh pip install -U huggingface_hub hf download allenai/OLMoE-1B-7B-0924 ``` ### How to use it Load the model and tokenizer, tokenize a prompt, and call generate. The card's sample moves everything to the GPU when one exists and prints a reply of up to 64 tokens. ```python from transformers import OlmoeForCausalLM, AutoTokenizer import torch DEVICE = "cuda" if torch.cuda.is_available() else "cpu" model = OlmoeForCausalLM.from_pretrained("allenai/OLMoE-1B-7B-0924").to(DEVICE) tokenizer = AutoTokenizer.from_pretrained("allenai/OLMoE-1B-7B-0924") inputs = tokenizer("Bitcoin is", return_tensors="pt") inputs = {k: v.to(DEVICE) for k, v in inputs.items()} out = model.generate(**inputs, max_length=64) print(tokenizer.decode(out[0])) ``` To load a different checkpoint, pass a revision name such as step10000-tokens41B to from\_pretrained. This snippet lists every revision. ```python from huggingface_hub import list_repo_refs out = list_repo_refs("allenai/OLMoE-1B-7B-0924") branches = [b.name for b in out.branches] ``` | Tool | What it does | | --- | --- | | OlmoeForCausalLM.from\_pretrained | Loads the model weights | | AutoTokenizer.from\_pretrained | Loads the matching tokenizer | | generate | Writes the continuation, here up to 64 tokens | | list\_repo\_refs | Lists the saved revisions and branches | ### Example You give the model the start of a sentence, "Bitcoin is", and it writes the rest. In the card's sample, the continuation explains that bitcoin is a digital currency created and held electronically. ### For newcomers and experts The main model continues whatever text you give it, so the card's sample starts from a sentence opening. The first small thing to try is that sample, with a prompt of your own. Tuned versions for instructions are separate checkpoints. Only 1B of the 7B parameters work on each token, so serving cost tracks a 1B model while the weights stay larger. The main branch was annealed from the step1200000-tokens5033B checkpoint for 100B more tokens, and that is the one the tuned versions build on. Weights were kept in FP32 during training, and the card says casting to BF16 showed no performance drop. A separate fp32 branch holds the original, which gives slightly different results. Download the weights with hf → Load model and tokenizer in transformers → Your prompt becomes tokens → A few experts handle each token → The continuation is decoded to text ### Good to know The card names three branches that matter, and tuned versions come as separate checkpoints. - step1200000-tokens5033B: the pretraining checkpoint used for annealing. - main: the annealed checkpoint, used for the SFT and Instruct adaptations. - fp32: the FP32 version of main. - Other versions: the record counts 9 quantized and 7 fine-tuned versions made by other people. MMLU score among models with about 1B active parameters | Label | Value | | --- | --- | | OLMoE-1B-7B | 54.1 points | | DCLM-1B | 48.5 points | | TinyLlama-1B | 33.6 points | | OLMo-1B (0724) | 32.1 points | | Pythia-1B | 31.1 points | - License: Apache 2.0. - Cost to run: no API key, account or paid service is named, but the BF16 weights take about 13.8 GB. - Data it reaches: it downloads files from Hugging Face, and the sample generates text on your own machine. - Not for: the source names no forbidden uses. ## README ````markdown # OLMoE-1B-7B-0924 Continues English text with a small-cost Mixture-of-Experts model that has 1B active and 7B total parameters. Fully open code, data and checkpoints make it a fit for researchers and developers using transformers. Made by allenai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: OLMoE-1B-7B-0924 is an open language model from allenai, released in September 2024. It is a Mixture-of-Experts design with 1B active parameters out of 7B in total, so each token costs about as much as a 1B model. The card calls it fully open and says it holds its own against larger models such as Llama2-13B. Who it suits: Researchers who want to inspect every training stage fit best, and so do developers who want to try a Mixture-of-Experts model through transformers. Students of how language models are trained can follow the released stages. ```table Industry | Who uses it | How they use it | Example Research | Language-model researchers | Study a model whose code, data and logs are published for each training stage | Loading an early checkpoint by passing its name as a revision Software | Developers | Run a Mixture-of-Experts model with 1B active parameters in their own Python code | Generating a continuation from a short text prompt Education | Students of language-model training | Follow the pretraining, supervised fine-tuning and preference stages with released checkpoints | Comparing the main checkpoint with its tuned Instruct version ``` You can skip it if you need replies in a language other than English, since the record lists English only. Where to use it: It runs in Python through the transformers library and torch. - Windows, macOS or Linux, where the files download through Python - A CUDA GPU when one is present, otherwise the CPU, as the card's sample chooses - Your own scripts and notebooks that load models with transformers ```table Use case | Who | What they get back Text continuation | Developers | A passage that carries on from a prompt Checkpoint study | Researchers | Models from different points in training Model comparison | Students | A small open model to set beside other 1B-class ones ``` How to set it up: You need Python with a few libraries and room for the weights. - Python on Windows, macOS or Linux - torch - transformers installed from source, which the card asks for until a later release carries the needed change - About 13.8 GB of disk for the BF16 weights, or about 3.5 GB at 4-bit, which is an estimate 1. Install the Hugging Face hub library. 2. Download the model files with the hf command. 3. Install torch and transformers from source, as the card says. ```sh pip install -U huggingface_hub hf download allenai/OLMoE-1B-7B-0924 ``` How to use it: Load the model and tokenizer, tokenize a prompt, and call generate. The card's sample moves everything to the GPU when one exists and prints a reply of up to 64 tokens. ```python from transformers import OlmoeForCausalLM, AutoTokenizer import torch DEVICE = "cuda" if torch.cuda.is_available() else "cpu" model = OlmoeForCausalLM.from_pretrained("allenai/OLMoE-1B-7B-0924").to(DEVICE) tokenizer = AutoTokenizer.from_pretrained("allenai/OLMoE-1B-7B-0924") inputs = tokenizer("Bitcoin is", return_tensors="pt") inputs = {k: v.to(DEVICE) for k, v in inputs.items()} out = model.generate(**inputs, max_length=64) print(tokenizer.decode(out[0])) ``` To load a different checkpoint, pass a revision name such as step10000-tokens41B to from_pretrained. This snippet lists every revision. ```python from huggingface_hub import list_repo_refs out = list_repo_refs("allenai/OLMoE-1B-7B-0924") branches = [b.name for b in out.branches] ``` ```table Tool | What it does OlmoeForCausalLM.from_pretrained | Loads the model weights AutoTokenizer.from_pretrained | Loads the matching tokenizer generate | Writes the continuation, here up to 64 tokens list_repo_refs | Lists the saved revisions and branches ``` Example: You give the model the start of a sentence, "Bitcoin is", and it writes the rest. In the card's sample, the continuation explains that bitcoin is a digital currency created and held electronically. For newcomers and experts: The main model continues whatever text you give it, so the card's sample starts from a sentence opening. The first small thing to try is that sample, with a prompt of your own. Tuned versions for instructions are separate checkpoints. Only 1B of the 7B parameters work on each token, so serving cost tracks a 1B model while the weights stay larger. The main branch was annealed from the step1200000-tokens5033B checkpoint for 100B more tokens, and that is the one the tuned versions build on. Weights were kept in FP32 during training, and the card says casting to BF16 showed no performance drop. A separate fp32 branch holds the original, which gives slightly different results. ```flow Download the weights with hf Load model and tokenizer in transformers Your prompt becomes tokens A few experts handle each token The continuation is decoded to text ``` Good to know: The card names three branches that matter, and tuned versions come as separate checkpoints. - step1200000-tokens5033B: the pretraining checkpoint used for annealing. - main: the annealed checkpoint, used for the SFT and Instruct adaptations. - fp32: the FP32 version of main. - Other versions: the record counts 9 quantized and 7 fine-tuned versions made by other people. ```chart Title: MMLU score among models with about 1B active parameters OLMoE-1B-7B: 54.1 points DCLM-1B: 48.5 points TinyLlama-1B: 33.6 points OLMo-1B (0724): 32.1 points Pythia-1B: 31.1 points ``` - License: Apache 2.0. - Cost to run: no API key, account or paid service is named, but the BF16 weights take about 13.8 GB. - Data it reaches: it downloads files from Hugging Face, and the sample generates text on your own machine. - Not for: the source names no forbidden uses. ## On Hugging Face, 13 November 2026 - Task: text generation - Library: transformers - Parameters: 6.9B - Weights: about 13.8 GB in BF16; about 3.5 GB at 4-bit (estimate) - Formats: Safetensors - Downloads in the last 30 days: 45,654 - Likes: 158 - License: apache-2.0 - Languages: en - Quantized versions: 9 - Fine-tunes: 7 - Tags: transformers, safetensors, olmoe, text-generation, olmo, co2_eq_emissions ## Links - Model: https://huggingface.co/allenai/OLMoE-1B-7B-0924 - Files: https://huggingface.co/allenai/OLMoE-1B-7B-0924/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download allenai/OLMoE-1B-7B-0924 ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 7 KB, downloaded as a zip. ```text README.md 7 KB ``` ## Source [allenai/OLMoE-1B-7B-0924 on Hugging Face](https://huggingface.co/allenai/OLMoE-1B-7B-0924) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | OLMoE-1B-7B-0924 (this page) | 45,654 | 158 | 6.9B | apache-2.0 | 2024-10-19 | | [Molmo2-8B](https://mindket.com/p/c37def6b-10ae-4cfc-ab12-0bd2dad290b9) | 45,213 | 201 | 8.7B | apache-2.0 | 2026-01-23 | | [Molmo2-4B](https://mindket.com/p/9044f3b5-24c2-4d90-8ce5-d8a330fdaf12) | 26,915 | 57 | 4.9B | apache-2.0 | 2026-01-23 | | [GPT-2](https://mindket.com/p/c513092f-bbe7-4baa-989f-dfdc12a2b725) | 32,892,223 | 3,932 | 137M | mit | 2024-02-19 | | [Qwen3-0.6B](https://mindket.com/p/0598bfe0-640a-4bf0-b9f4-ca4cf70a04ae) | 19,718,599 | 1,894 | 752M | apache-2.0 | 2025-07-26 | | [Qwen3-14B](https://mindket.com/p/56074bfc-f2ce-4109-8314-1af039530db4) | 12,965,376 | 540 | 14.8B | apache-2.0 | 2025-07-26 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2024-10-19 - First published: 2024-07-20 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open allenai/OLMoE-1B-7B-0924 on Hugging Face](https://huggingface.co/allenai/OLMoE-1B-7B-0924) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/e29f1b5e-c361-4e68-bf44-e60ac0a18edb) - [Its page on Mindket](https://mindket.com/p/e29f1b5e-c361-4e68-bf44-e60ac0a18edb)