Memorilla-Qwen3-8B

Memorilla-Qwen3-8B

Holds memory-module checkpoints for Memorilla, which add a memory to a frozen Qwen3-8B model through a Qwen3-Embedding-4B encoder. For researchers testing memory for language models.

Made by memorilla on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Memorilla-Qwen3-8B is a set of memory module checkpoints for the Memorilla project. Each one was trained while the Qwen3-8B decoder and the Qwen3-Embedding-4B encoder stayed frozen.

The modules only work together with those same two models. The repository has five folders, three training stages and two fine-tuned variants.

Who it suits: Researchers and developers who study memory for language models, personalization or long-term knowledge, and who can run the Memorilla code.

Industry | Who uses it | How they use it | Example
Research | Language model researchers | Evaluate or extend trained memory modules | Continue training from the stage 2 checkpoint
Software | Developers testing personalization | Load a fine-tuned memory checkpoint | Try the PersonaMem-v2 variant
Education | Students of model memory methods | Reproduce the staged training recipe | Run the evaluation script on one checkpoint

Skip it if you want a chat model you can use on its own, since this repository holds memory modules and not a standalone model.

Where to use it:

• The Memorilla code from GitHub, with its Python API and scripts
• Windows, macOS or Linux with Python, to download the checkpoints
• A setup that already runs Qwen3-8B and Qwen3-Embedding-4B

Use case | Who | What they get back
Load a trained memory | Researchers | A memory module object from a checkpoint folder
Evaluate a checkpoint | Experiment runners | Results from the evaluation script
Continue training | Method developers | A new run that starts from an earlier checkpoint

How to set it up:

• Python and pip
• The Memorilla code installed from its GitHub project, following its installation notes
• The Qwen3-8B and Qwen3-Embedding-4B models, which the modules are used with

1. Install the Memorilla code.
2. Download the checkpoints into a local folder.
3. Load the checkpoint you want from Python.

hf download memorilla/Memorilla-Qwen3-8B --local-dir checkpoints

Folder | Checkpoint | What it is
stage1 | Stage 1 | The first training stage
stage2 | Stage 2 | Continued from Stage 1
stage3 | Stage 3 | Continued from Stage 2, saved at epoch 00
pv4 | PersonalizationV4 | Fine-tuned from Stage 1
pmv2 | PersonaMem-v2 | Fine-tuned from Stage 1

How to use it: Load a checkpoint folder with MemoryModule. Two scripts cover the rest: one scores a checkpoint, and the other resumes training from an earlier one through the init_memory flag.

from memorilla import MemoryModule

memory = MemoryModule.from_pretrained("checkpoints/stage3")

scripts/evaluate.sh checkpoints/pv4 pv4
scripts/train.sh configs/stage3_multitask.yaml --init_memory checkpoints/stage2

Tool | What it does
MemoryModule.from_pretrained | Loads a memory module from a checkpoint folder
scripts/evaluate.sh | Evaluates a checkpoint
scripts/train.sh | Trains from a config file, optionally starting from a checkpoint

Example: You download the checkpoints and load the stage 3 folder with MemoryModule. You get a memory module that you then use with the frozen Qwen3-8B and Qwen3-Embedding-4B models.

For newcomers and experts: If you are new to this, think of the module as an add-on memory for a language model. Download the files, install the Memorilla code, and load the stage 3 checkpoint first.

If you know the field, each folder holds memory.pt, which has 194M parameters in bfloat16, and config.json, which sets 16 memory tokens and no document self-attention. The staged recipe moves from an English Wikipedia stage to a mixture and then to multitask training.

Download the checkpoint folders
Install the Memorilla code
Load a memory module
Use it with the two frozen models

Good to know:

• The modules are trained against frozen models and must be used with the same two
• The two fine-tuned variants both start from Stage 1
• The training guide in the Memorilla project explains the stages and recipes
• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the checkpoint files you download and whatever data you evaluate on
• Not for: standalone use, or use with models other than Qwen3-8B and Qwen3-Embedding-4B

On Hugging Face, 13 November 2026

• Downloads in the last 30 days: 139
• Likes: 19
• License: apache-2.0
• Based on: Qwen/Qwen3-8B
• Tags: memory, long-context

Links

• Model: https://huggingface.co/memorilla/Memorilla-Qwen3-8B
• Files: https://huggingface.co/memorilla/Memorilla-Qwen3-8B/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download memorilla/Memorilla-Qwen3-8B

Then follow the usage in its model card. Nothing in this kit runs by itself.