OpenEuroLLM Prelude-base 9B

OpenEuroLLM Prelude-base 9B

Intermediate checkpoints of an experimental 9B base language model trained on 10T tokens, mostly English with a multilingual share. For researchers who follow open European model training.

Made by openeurollm on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Prelude-base is an experimental base language model with about 9 billion parameters, made by the OpenEuroLLM project. It is still training on the Leonardo system at CINECA, so the repository holds intermediate checkpoints, one per branch. It is marked as a research work in progress, and the final checkpoint is meant for the main branch later.

Who it suits: It suits researchers and builders who want to inspect how an open European base model changes during training. Each checkpoint is a complete Hugging Face model, so loading one takes a few lines of Python.

Industry | Who uses it | How they use it | Example
Research | Language model researchers | Load a checkpoint branch and study its generations | Loading the branch iter_0124800 and sampling from short prompts
Education | Students of language models | Read the architecture table and training mix to learn how a 9B model is built | The layer, head and sequence length figures in the card
Multilingual work | Teams working on European languages | Check how a base model continues prompts in other languages | The card's greetings in Spanish, French, German, Italian and Finnish
Model building | Engineers who fine-tune | Start from a checkpoint and train it further | Hugging Face lists six fine-tuned versions of it

Skip it if you want a finished assistant. Only intermediate checkpoints exist, and the main branch is empty on purpose.

Where to use it:

• Python with the transformers library, on Windows, macOS or Linux
• A machine with room for about 18.19 GB of bf16 weights
• A research notebook or script that loads a checkpoint by branch name

Use case | Who | What they get back
Sample from a checkpoint | Researchers | Generated text from short prompts
Compare training stages | Model builders | A way to load different branches with the same code
Check the tokenizer setup | Developers | Tokenized input without a start token

How to set it up:

• Python
• The transformers and torch Python packages, which the sample code imports
• About 18.19 GB of disk space for the weights, plus memory to run them

1. Install the Hugging Face hub package.
2. Download the repository files.
3. In Python, pick a branch name such as iter_0124800 and pass it as the revision.

pip install -U huggingface_hub
hf download openeurollm/prelude-base

The main branch is empty on purpose, so the checkpoints live on the per-iteration branches that the Python below selects.

How to use it: Pass the branch name as the revision, load the model and tokenizer, then generate from your prompts. The sample below runs five greetings in different languages.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "openeurollm/prelude-base"
revision = "iter_0124800"

model = AutoModelForCausalLM.from_pretrained(
    repo, revision=revision, dtype=torch.bfloat16, device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(repo, revision=revision)

prompts = [
    "Hola, ¿qué tal?",
    "Bonjour, comment ça va ?",
    "Guten Tag, wie geht es dir?",
    "Ciao, come stai?",
    "Hei, mitä kuuluu?",
]
generate_kwargs = dict(max_new_tokens=50, do_sample=True, temperature=0.7, top_p=0.9, top_k=50, repetition_penalty=1.3, no_repeat_ngram_size=3)

for prompt in prompts:
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    outputs = model.generate(**inputs, **generate_kwargs)
    print(tokenizer.decode(outputs[0], skip_special_tokens=True))

The model was trained without a start token, so the tokenizers on the branches do not add one. If a copy you downloaded earlier adds a start token anyway, set the add_special_tokens option to False when you tokenize.

Tool | What it does
AutoModelForCausalLM.from_pretrained | Loads the checkpoint branch named by revision
AutoTokenizer.from_pretrained | Loads the tokenizer from the same branch
model.generate | Writes text that continues your prompt
validation.json | Holds example greedy completions saved with each checkpoint

Example: Load the branch iter_0124800 and give it the prompt "Bonjour, comment ça va ?" The script prints the prompt followed by whatever the model writes next, up to 50 new tokens.

For newcomers and experts: This is raw material for a language model, a base model that continues text. For a first try, download the files, run the Python above on one prompt, and read what it writes. Expect rough output, since training is not finished.

Underneath, it is a dense, decoder-only Transformer that follows Qwen3 closely. It has 36 layers, hidden size 4096, 32 attention heads with 8 key-value groups, RoPE, RMSNorm with QK-norm, SwiGLU and untied embeddings. The vocabulary has 262,145 entries, padded for tensor parallel efficiency. The published checkpoints come from the constant learning rate phase of a WSD schedule, before any cooldown.

Pick a checkpoint branch
Load model and tokenizer with revision
Tokenize the prompt without a start token
Generate with sampling settings
Read the continuation

Good to know: The training mix covers 36 target languages, held as 41 language and script codes. The sequence length is 4096.

Title: Rough share of the 10T training tokens
English, code and math: 80 percent
Multilingual text: 20 percent

• License: Apache 2.0
• Cost to run: no API key or paid service is named, you supply the compute
• Data it reaches: downloads checkpoint files from Hugging Face, and the sample runs the model on your own machine
• Not for: a finished or cooled down model, since the main branch is empty and every published checkpoint comes before the cooldown

On Hugging Face, 13 November 2026

• Downloads in the last 30 days: 6,203
• Likes: 20
• License: apache-2.0
• Languages: en, es, ka, fr, de, pt, it, pl
• Fine-tunes: 6

Links

• Model: https://huggingface.co/openeurollm/prelude-base
• Files: https://huggingface.co/openeurollm/prelude-base/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download openeurollm/prelude-base

Then follow the usage in its model card. Nothing in this kit runs by itself.