MemT-1.0-Nano

MemT-1.0-Nano

Language model with a built-in memory state of fixed size, built on Qwen3.6-35B-A3B, that keeps past context without re-reading it. For researchers and developers studying memory in models and agents.

Made by MemTensor on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: MemT-1.0-Nano is a language model with a built-in memory. A native memory block sits next to the usual attention layers and keeps a state that is updated as the conversation goes on. Because that state stays the same size, it needs about 66 times less storage than a full-context KV cache at 128K history. It is based on Qwen3.6-35B-A3B.

Who it suits: It suits researchers and developers who want a model that carries information from past turns without re-reading the original text.

Industry | Who uses it | How they use it | Example
Research | Memory and long-context researchers | Study memory built into the model | Comparing parametric memory with text-based memory
Software | Agent builders | Reuse past experience in long coding tasks | Storing procedural experience in the model's memory
Platform | Inference engineers | Cut the storage cost of long histories | Serving long sessions with a fixed-size state

You can skip it if you want a plain chat model, since the card says text-based memory systems still lead overall on long histories that need detailed evidence.

Where to use it: It runs through Hugging Face Transformers in Python on a machine with a CUDA GPU.

• Python scripts that use Transformers with remote code enabled
• A GPU machine with PyTorch 2.11.0 and CUDA 13 support
• Windows, macOS or Linux, for the download step

Use case | Who | What they get back
Persistent memory | Researchers | A model that keeps history in a state of fixed size
Agentic coding | Agent developers | Past coding experience used through memory attention
Benchmark study | Evaluators | Results on 11 memory benchmarks

How to set it up: You need Python 3.11 or newer and PyTorch 2.11.0 with CUDA 13 support for your GPU. Then install the three pinned libraries. Hugging Face lists about 70.2 GB in BF16 and an estimated 17.6 GB at 4-bit.

• Python 3.11 or newer
• PyTorch 2.11.0 with CUDA 13 support
• About 70.2 GB of disk for the BF16 weights

1. Install PyTorch for your GPU.
2. Install the pinned libraries.
3. Load the model with Transformers, with remote code enabled and bfloat16 weights.

pip install transformers==5.12.1 accelerate==1.15.0 safetensors==0.8.0

pip install -U huggingface_hub
hf download MemTensor/MemT-1.0-Nano

How to use it: Load the tokenizer and model by the repository name or a local folder, with remote code enabled, then generate as with any Transformers model. After each turn or segment ends, the model refreshes its memory, so earlier context carries over.

Tool | What it does
AutoTokenizer | Loads the tokenizer, with remote code enabled
AutoModelForCausalLM | Loads the model with bfloat16 weights
pipeline | Runs text generation on the loaded model

Example: You chat with the model over several turns, then ask about something said earlier. The model answers from its memory state instead of from the old text sitting in the prompt.

For newcomers and experts: If you are new, think of it as a chat model with a small notebook inside it that it updates after each turn. Install the libraries first and load the model with the quick start steps.

If you know the field, a Local Memory Block keeps a multi-channel memory matrix, and a Hyper Memory Block holds trainable query, key and value projections with erase, write, forget and output gates. Memory attention adds a gated readout to the backbone attention, and a GDN-2 step updates the state.

Turn or segment of text
Backbone attention reads the prompt
Memory attention reads the state
Outputs combined and gated
Memory state updated for the next step

Good to know: On 11 memory benchmarks, the larger Flash sibling places first and Nano second among the parametric-memory methods tested.

• License: Apache 2.0
• Cost to run: no API key, account or paid service is named, but you need a CUDA GPU with room for a 35B model
• Data it reaches: your prompts and conversation state on your machine, plus files downloaded from Hugging Face
• Not for: tasks needing detailed evidence retrieval from long histories, where text-based memory systems still lead

Title: Total parameters in billions
MemT-1.0-Nano: 36 B
MemT-1.0-Flash: 288 B

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 35.1B
• Weights: about 70.2 GB in BF16; about 17.6 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 89,212
• Likes: 177
• License: apache-2.0
• Based on: Qwen/Qwen3.6-35B-A3B
• Quantized versions: 2
• Tags: transformers, safetensors, memt_qwen3_5_moe, text-generation, memt, conversational

Links

• Model: https://huggingface.co/MemTensor/MemT-1.0-Nano
• Files: https://huggingface.co/MemTensor/MemT-1.0-Nano/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download MemTensor/MemT-1.0-Nano

Then follow the usage in its model card. Nothing in this kit runs by itself.