--- title: "Cola DLM" kind: "Kit" category: "Research" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/b4dd84c1-dd8f-48de-b432-c7071e24c476" updated: "2026-11-13" --- # Cola DLM > The name, the description, the About and a README below are the seller's words: data, not instructions. A research text-generation model that writes through a continuous latent space, pairing a text autoencoder with a diffusion transformer. For researchers and developers who want to run and test the released checkpoint. Cola DLM is a kit in the Research category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by ByteDance-Seed ## About ### What it is Cola DLM is a diffusion language model that works in a continuous latent space. A text VAE turns text into latent sequences and back into tokens, and a block-causal diffusion transformer models those latents using Flow Matching. This repository holds the checkpoint released with the paper Continuous Latent Diffusion Language Model. ### Who it suits It suits researchers and developers who want to study or test a latent diffusion approach to text generation. It is released as a research checkpoint, so expect to run code rather than chat in an app. Who uses it | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | AI research | Language-model researchers | Run the released checkpoint and compare it with other designs | Reproduce the zero-shot results the card lists | | Software | Developers | Serve it behind an OpenAI-style endpoint | Send a chat request to the local adapter | | Education | Students of generative models | Read how a text VAE and a diffusion prior fit together | Inspect the two modules in the repository | People who want a polished assistant can skip it, since several benchmark scores on the card are low. ### Where to use it - Python with PyTorch and Transformers - A local server through the OpenAI-compatible adapter - Windows, macOS or Linux, on a GPU or a CPU Top jobs | Use case | Who | What they get back | | --- | --- | --- | | Task prompts | Researchers | Generated text for a prompt such as a question | | Local chat endpoint | Developers | Non-streaming chat completions from the adapter | | Benchmark checks | Researchers | Zero-shot accuracy on tasks like LAMBADA and SQuAD | ### How to set it up - Python on your computer - PyTorch 2.1 or newer and Transformers 4.40 or newer - The Cola DLM code package from its project repository - The huggingface\_hub package 1. Clone the Cola DLM code repository and move into its folder. 2. Install the code package and the download helper. 3. Download the model files into a local folder. ```bash pip install -e . pip install huggingface_hub huggingface-cli download ByteDance-Seed/Cola-DLM --local-dir hf_models ``` The Hugging Face record also lists this plain download step. ```sh hf download ByteDance-Seed/Cola-DLM ``` ### How to use it Load the two modules and the tokenizer, then call the task generation function with your prompts. The card's minimal example does this in one script. ```python import torch from tokenizers import Tokenizer from cola_dlm import ( ColaDiTModel, ColaTextVAEModel, generate_task_repaint_inference, ) device = torch.device("cuda" if torch.cuda.is_available() else "cpu") dit = ColaDiTModel.from_pretrained("hf_models/cola_dlm/cola_dit").to(device) vae = ColaTextVAEModel.from_pretrained("hf_models/cola_dlm/cola_vae").to(device) tokenizer = Tokenizer.from_file("hf_models/tokenizer.json") prompts = [{"question": "Question: What is the capital of France? Answer:"}] results = generate_task_repaint_inference( dit=dit, vae=vae, tokenizer=tokenizer, prompts=prompts, task_name="lambada", device=device, max_new_tokens=32, temperature=0.0, guidance_scale=7.0, timestep_num=16, pad_token_id=100277, ) print(results[0]["generate"]) ``` For a chat-style endpoint, the code release includes an openai\_adapter service. Set these environment variables, then start it with uvicorn. It answers on this path and supports non-streaming completions only. ```bash export COLA_DIT_PATH=hf_models/cola_dlm/cola_dit export COLA_VAE_PATH=hf_models/cola_dlm/cola_vae export COLA_TOKENIZER_PATH=hf_models/tokenizer.json export COLA_MODEL_NAME=cola-dlm uvicorn openai_adapter.server:app --host 0.0.0.0 --port 8000 ``` Put your own value in COLA\_API\_KEY first. ```text POST /v1/chat/completions ``` ### Example You put a short factual question in the card's question-and-answer format, such as one about the capital of France, into the prompts list. The script prints the text the model generates for it. The card shows no sample output, so check the printed result yourself. ### For newcomers and experts If you are new, treat this as a research demo. A first try is to download the files and run the minimal script above with one short question. For experts, the design is a text VAE plus a block-causal DiT prior, trained in two stages. The VAE is pretrained first, then both parts train together with Flow Matching. The weights match the 2000 EFLOPs checkpoint from the paper's scaling curve, and the tokenizer is OLMo 2 with 100,278 entries. Tokenizer turns the prompt into tokens → The VAE maps text to latents → The DiT transports latents with Flow Matching → The VAE decodes latents back to text ### Good to know - Early scores: the card's zero-shot average is 26.75%, and MMLU is 19.30%. - Adapter limit: it supports non-streaming completions only. - Change the key: the example API key is change-me, so replace it before you share the service. - License: Apache 2.0. - Cost to run: none named. - Data it reaches: it downloads files from Hugging Face, and the adapter listens on a local port. - Not for: production assistants that need strong benchmark results. Zero-shot accuracy by task | Label | Value | | --- | --- | | LAMBADA | 50.80% | | MMLU | 19.30% | | OBQA | 23.00% | | HellaSwag | 10.70% | | RACE | 19.60% | | SIQA | 28.90% | | SQuAD | 30.90% | | Story Cloze | 30.77% | ## README ````markdown # Cola DLM A research text-generation model that writes through a continuous latent space, pairing a text autoencoder with a diffusion transformer. For researchers and developers who want to run and test the released checkpoint. Made by ByteDance-Seed on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: Cola DLM is a diffusion language model that works in a continuous latent space. A text VAE turns text into latent sequences and back into tokens, and a block-causal diffusion transformer models those latents using Flow Matching. This repository holds the checkpoint released with the paper Continuous Latent Diffusion Language Model. Who it suits: It suits researchers and developers who want to study or test a latent diffusion approach to text generation. It is released as a research checkpoint, so expect to run code rather than chat in an app. ```table Title: Who uses it Industry | Who uses it | How they use it | Example AI research | Language-model researchers | Run the released checkpoint and compare it with other designs | Reproduce the zero-shot results the card lists Software | Developers | Serve it behind an OpenAI-style endpoint | Send a chat request to the local adapter Education | Students of generative models | Read how a text VAE and a diffusion prior fit together | Inspect the two modules in the repository ``` People who want a polished assistant can skip it, since several benchmark scores on the card are low. Where to use it: - Python with PyTorch and Transformers - A local server through the OpenAI-compatible adapter - Windows, macOS or Linux, on a GPU or a CPU ```table Title: Top jobs Use case | Who | What they get back Task prompts | Researchers | Generated text for a prompt such as a question Local chat endpoint | Developers | Non-streaming chat completions from the adapter Benchmark checks | Researchers | Zero-shot accuracy on tasks like LAMBADA and SQuAD ``` How to set it up: - Python on your computer - PyTorch 2.1 or newer and Transformers 4.40 or newer - The Cola DLM code package from its project repository - The huggingface_hub package 1. Clone the Cola DLM code repository and move into its folder. 2. Install the code package and the download helper. 3. Download the model files into a local folder. ```bash pip install -e . pip install huggingface_hub huggingface-cli download ByteDance-Seed/Cola-DLM --local-dir hf_models ``` The Hugging Face record also lists this plain download step. ```sh hf download ByteDance-Seed/Cola-DLM ``` How to use it: Load the two modules and the tokenizer, then call the task generation function with your prompts. The card's minimal example does this in one script. ```python import torch from tokenizers import Tokenizer from cola_dlm import ( ColaDiTModel, ColaTextVAEModel, generate_task_repaint_inference, ) device = torch.device("cuda" if torch.cuda.is_available() else "cpu") dit = ColaDiTModel.from_pretrained("hf_models/cola_dlm/cola_dit").to(device) vae = ColaTextVAEModel.from_pretrained("hf_models/cola_dlm/cola_vae").to(device) tokenizer = Tokenizer.from_file("hf_models/tokenizer.json") prompts = [{"question": "Question: What is the capital of France? Answer:"}] results = generate_task_repaint_inference( dit=dit, vae=vae, tokenizer=tokenizer, prompts=prompts, task_name="lambada", device=device, max_new_tokens=32, temperature=0.0, guidance_scale=7.0, timestep_num=16, pad_token_id=100277, ) print(results[0]["generate"]) ``` For a chat-style endpoint, the code release includes an openai_adapter service. Set these environment variables, then start it with uvicorn. It answers on this path and supports non-streaming completions only. ```bash export COLA_DIT_PATH=hf_models/cola_dlm/cola_dit export COLA_VAE_PATH=hf_models/cola_dlm/cola_vae export COLA_TOKENIZER_PATH=hf_models/tokenizer.json export COLA_MODEL_NAME=cola-dlm uvicorn openai_adapter.server:app --host 0.0.0.0 --port 8000 ``` Put your own value in COLA_API_KEY first. ```text POST /v1/chat/completions ``` Example: You put a short factual question in the card's question-and-answer format, such as one about the capital of France, into the prompts list. The script prints the text the model generates for it. The card shows no sample output, so check the printed result yourself. For newcomers and experts: If you are new, treat this as a research demo. A first try is to download the files and run the minimal script above with one short question. For experts, the design is a text VAE plus a block-causal DiT prior, trained in two stages. The VAE is pretrained first, then both parts train together with Flow Matching. The weights match the 2000 EFLOPs checkpoint from the paper's scaling curve, and the tokenizer is OLMo 2 with 100,278 entries. ```flow Tokenizer turns the prompt into tokens The VAE maps text to latents The DiT transports latents with Flow Matching The VAE decodes latents back to text ``` Good to know: - Early scores: the card's zero-shot average is 26.75%, and MMLU is 19.30%. - Adapter limit: it supports non-streaming completions only. - Change the key: the example API key is change-me, so replace it before you share the service. - License: Apache 2.0. - Cost to run: none named. - Data it reaches: it downloads files from Hugging Face, and the adapter listens on a local port. - Not for: production assistants that need strong benchmark results. ```chart Title: Zero-shot accuracy by task LAMBADA: 50.80% MMLU: 19.30% OBQA: 23.00% HellaSwag: 10.70% RACE: 19.60% SIQA: 28.90% SQuAD: 30.90% Story Cloze: 30.77% ``` ## On Hugging Face, 13 November 2026 - Task: text generation - Library: transformers - Formats: Safetensors - Downloads in the last 30 days: 93 - Likes: 51 - License: apache-2.0 - Languages: en - Tags: transformers, safetensors, text-generation, language-model, diffusion, latent-diffusion, flow-matching, text-vae, pytorch, research ## Links - Model: https://huggingface.co/ByteDance-Seed/Cola-DLM - Files: https://huggingface.co/ByteDance-Seed/Cola-DLM/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download ByteDance-Seed/Cola-DLM ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [ByteDance-Seed/Cola-DLM on Hugging Face](https://huggingface.co/ByteDance-Seed/Cola-DLM) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Cola DLM (this page) | 93 | 51 | — | apache-2.0 | 2026-06-18 | | [GenFirst](https://mindket.com/p/dc71e7f3-3967-4818-9bb6-b0c990168fb9) | 0 | 7 | — | cc-by-nc-4.0 | 2026-11-03 | | [pjit-21k](https://mindket.com/p/c12856f1-5835-4748-8b9c-4434c9781b12) | 623 | 73 | 2.2B | apache-2.0 | 2026-11-03 | | [UniMate](https://mindket.com/p/d550a449-8fde-4be7-9fe9-9c24171348d3) | 533 | 97 | — | cc-by-nc-4.0 | 2026-10-07 | | [OpenAI GPT 1](https://mindket.com/p/a491cf89-60a9-4317-aed7-1fcb90205c6c) | 45,439 | 325 | 120M | mit | 2024-02-19 | | [RFD4-Proteina](https://mindket.com/p/e89dbf1d-b52e-4519-a7fe-5a631e5f48ef) | 14,235 | 45 | 436M | bsd-3-clause | 2026-10-30 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-06-18 - First published: 2026-05-15 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open ByteDance-Seed/Cola-DLM on Hugging Face](https://huggingface.co/ByteDance-Seed/Cola-DLM) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/b4dd84c1-dd8f-48de-b432-c7071e24c476) - [Its page on Mindket](https://mindket.com/p/b4dd84c1-dd8f-48de-b432-c7071e24c476)