--- title: "Kiyo-65M" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/41f070ee-1394-4748-845b-ecb0e0d53d8e" updated: "2026-11-13" --- # Kiyo-65M > The name, the description, the About and a README below are the seller's words: data, not instructions. Continues English text with a tiny 65M-parameter decoder-only model trained on 80B tokens. For learners and researchers who want a small transformers model to study, compare or test code against. Kiyo-65M is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by DedeProGames ## About ### What it is A tiny English language model with 65M parameters, the smallest of the Kiyo family. It is a base model, so it continues the text you give it and does not follow instructions. It was trained on 80B tokens starting from random weights, using web text, code and math, and it follows the SmolLM2 architecture. ### Who it suits People who want a small, fully described model to learn from, compare against, or use as a light test target. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Education | Students of language models | Study a small decoder-only model from its layer and head counts | Read the model table, then run the sample script | | Research | Small-model researchers | Use it as a baseline at the 50M to 65M scale | Compare its ARC-Easy score with other base models | | Software | Developers testing pipelines | Test generation code against a model of about 130 MB | Check a generate loop before switching to a larger model | You can skip it if you need a chat assistant, instruction following, long documents or reliable facts. ### Where to use it It runs in Python with the transformers library, on Windows, macOS or Linux. - Python scripts that use transformers and torch - A machine with a CUDA GPU, since the card's sample moves the model onto one - Windows, macOS or Linux with Python, to download the files | Use case | Who | What they get back | | --- | --- | --- | | Text continuation | Learners | The start of a sentence continued by up to 64 new tokens in the card's sample | | Baseline comparison | Researchers | Zero-shot scores on four benchmarks to measure against | | Pipeline testing | Developers | A small checkpoint to run code against before using a bigger model | ### How to set it up Install three Python packages, then run the card's short script. The script loads the model by its ID. - Python - The packages transformers, safetensors and torch - A CUDA GPU, because the sample script calls cuda - About 130 MB of space for the BF16 weights 1. Install the three Python packages. 2. Save the card's script to a file. 3. Run the file with Python and read the printed text. 4. Or fetch the files yourself with the Hugging Face tools. ```bash pip install -U transformers safetensors torch ``` ```python import torch from transformers import AutoTokenizer, AutoModelForCausalLM model_id = "DedeProGames/Kiyo-65M" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16 if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else torch.float16, ).cuda().eval() prompt = "The meaning of life is " input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(model.device) with torch.no_grad(): output = model.generate( input_ids=input_ids, max_new_tokens=64, do_sample=False, repetition_penalty=1.1, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id, ) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` ```bash pip install -U huggingface_hub hf download DedeProGames/Kiyo-65M ``` ### How to use it Change the prompt line to the start of a text and run the script again. The model writes a continuation of that start, so phrase your prompt as an opening and not as a question. | Tool | What it does | | --- | --- | | AutoTokenizer | Turns your text into token IDs and back into text | | AutoModelForCausalLM | Loads the model so it can continue text | | generate | Continues the prompt, here by up to 64 new tokens without sampling | ### Example With the prompt The meaning of life is followed by a space, the script prints that opening plus up to 64 more tokens. Expect a fluent continuation, not a checked answer. ### For newcomers and experts A language model predicts what text comes next, and this one is small enough to read about in full. A first small try is to run the card's script as it is, then change the prompt and compare the continuations. For experts, it is a Llama-style decoder with 15 layers, hidden size 512, 8 attention heads and 4 KV heads, so grouped query attention. It also uses RMSNorm, SwiGLU, RoPE with theta 10,000, tied embeddings and the 49,152-token SmolLM2 vocabulary. Training ran 152,587 optimizer steps of 524,288 tokens each, with a warmup-stable-decay schedule and a peak learning rate of 2.0e-3. The data mix is 45.0% FineWeb-Edu, 30.0% DCLM-Baseline, 17.5% Stack-v3-train code and 7.5% FineMath. The trade is size against reasoning, since harder items sit near chance. Tokenize your prompt → Run the 65M decoder → Pick the next token → Repeat up to the limit → Decode back to text ### Good to know The card compares it with other small base models on four zero-shot benchmarks. Kiyo-65M zero-shot accuracy | Label | Value | | --- | --- | | ARC-Easy | 46.51% | | PIQA | 62.30% | | ARC-Challenge | 23.29% | | HellaSwag | 31.98% | - Base model: it continues text and does not follow instructions. - Weak spots: number-based reasoning, chained logic and remembering earlier context, where harder items score near chance. - Context: 2,048 tokens, which limits long documents. - Facts: it can state wrong facts, so verify before any high-stakes use. - Generation: keep a finite length limit to avoid repetition or drift. - License: Apache 2.0. - Cost to run: none named, no API key, account or paid service. - Data it reaches: it downloads the model files from Hugging Face and reads the prompt you give it. - Not for: following instructions, long documents, or high-stakes decisions without checking. ## README ````markdown # Kiyo-65M Continues English text with a tiny 65M-parameter decoder-only model trained on 80B tokens. For learners and researchers who want a small transformers model to study, compare or test code against. Made by DedeProGames on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: A tiny English language model with 65M parameters, the smallest of the Kiyo family. It is a base model, so it continues the text you give it and does not follow instructions. It was trained on 80B tokens starting from random weights, using web text, code and math, and it follows the SmolLM2 architecture. Who it suits: People who want a small, fully described model to learn from, compare against, or use as a light test target. ```table Industry | Who uses it | How they use it | Example Education | Students of language models | Study a small decoder-only model from its layer and head counts | Read the model table, then run the sample script Research | Small-model researchers | Use it as a baseline at the 50M to 65M scale | Compare its ARC-Easy score with other base models Software | Developers testing pipelines | Test generation code against a model of about 130 MB | Check a generate loop before switching to a larger model ``` You can skip it if you need a chat assistant, instruction following, long documents or reliable facts. Where to use it: It runs in Python with the transformers library, on Windows, macOS or Linux. - Python scripts that use transformers and torch - A machine with a CUDA GPU, since the card's sample moves the model onto one - Windows, macOS or Linux with Python, to download the files ```table Use case | Who | What they get back Text continuation | Learners | The start of a sentence continued by up to 64 new tokens in the card's sample Baseline comparison | Researchers | Zero-shot scores on four benchmarks to measure against Pipeline testing | Developers | A small checkpoint to run code against before using a bigger model ``` How to set it up: Install three Python packages, then run the card's short script. The script loads the model by its ID. - Python - The packages transformers, safetensors and torch - A CUDA GPU, because the sample script calls cuda - About 130 MB of space for the BF16 weights 1. Install the three Python packages. 2. Save the card's script to a file. 3. Run the file with Python and read the printed text. 4. Or fetch the files yourself with the Hugging Face tools. ```bash pip install -U transformers safetensors torch ``` ```python import torch from transformers import AutoTokenizer, AutoModelForCausalLM model_id = "DedeProGames/Kiyo-65M" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16 if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else torch.float16, ).cuda().eval() prompt = "The meaning of life is " input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(model.device) with torch.no_grad(): output = model.generate( input_ids=input_ids, max_new_tokens=64, do_sample=False, repetition_penalty=1.1, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id, ) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` ```bash pip install -U huggingface_hub hf download DedeProGames/Kiyo-65M ``` How to use it: Change the prompt line to the start of a text and run the script again. The model writes a continuation of that start, so phrase your prompt as an opening and not as a question. ```table Tool | What it does AutoTokenizer | Turns your text into token IDs and back into text AutoModelForCausalLM | Loads the model so it can continue text generate | Continues the prompt, here by up to 64 new tokens without sampling ``` Example: With the prompt The meaning of life is followed by a space, the script prints that opening plus up to 64 more tokens. Expect a fluent continuation, not a checked answer. For newcomers and experts: A language model predicts what text comes next, and this one is small enough to read about in full. A first small try is to run the card's script as it is, then change the prompt and compare the continuations. For experts, it is a Llama-style decoder with 15 layers, hidden size 512, 8 attention heads and 4 KV heads, so grouped query attention. It also uses RMSNorm, SwiGLU, RoPE with theta 10,000, tied embeddings and the 49,152-token SmolLM2 vocabulary. Training ran 152,587 optimizer steps of 524,288 tokens each, with a warmup-stable-decay schedule and a peak learning rate of 2.0e-3. The data mix is 45.0% FineWeb-Edu, 30.0% DCLM-Baseline, 17.5% Stack-v3-train code and 7.5% FineMath. The trade is size against reasoning, since harder items sit near chance. ```flow Tokenize your prompt Run the 65M decoder Pick the next token Repeat up to the limit Decode back to text ``` Good to know: The card compares it with other small base models on four zero-shot benchmarks. ```chart Title: Kiyo-65M zero-shot accuracy ARC-Easy: 46.51% PIQA: 62.30% ARC-Challenge: 23.29% HellaSwag: 31.98% ``` - Base model: it continues text and does not follow instructions. - Weak spots: number-based reasoning, chained logic and remembering earlier context, where harder items score near chance. - Context: 2,048 tokens, which limits long documents. - Facts: it can state wrong facts, so verify before any high-stakes use. - Generation: keep a finite length limit to avoid repetition or drift. - License: Apache 2.0. - Cost to run: none named, no API key, account or paid service. - Data it reaches: it downloads the model files from Hugging Face and reads the prompt you give it. - Not for: following instructions, long documents, or high-stakes decisions without checking. ## On Hugging Face, 13 November 2026 - Task: text generation - Library: transformers - Parameters: 65M - Weights: about 130 MB in BF16; about 32 MB at 4-bit (estimate) - Formats: Safetensors - Downloads in the last 30 days: 1,179 - Likes: 5 - License: apache-2.0 - Languages: en - Quantized versions: 1 - Tags: transformers, safetensors, llama, text-generation, causal-lm, language-model, base-model, smollm, kiyo, pytorch, conversational ## Links - Model: https://huggingface.co/DedeProGames/Kiyo-65M - Files: https://huggingface.co/DedeProGames/Kiyo-65M/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download DedeProGames/Kiyo-65M ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 7 KB, downloaded as a zip. ```text README.md 7 KB ``` ## Source [DedeProGames/Kiyo-65M on Hugging Face](https://huggingface.co/DedeProGames/Kiyo-65M) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Kiyo-65M (this page) | 1,179 | 5 | 65M | apache-2.0 | 2026-10-23 | | [RWKV7-G1 GooseOne](https://mindket.com/p/a54a5930-5095-46b2-b2f8-a48a30bbceb0) | 9,911 | 253 | — | apache-2.0 | 2026-11-04 | | [ALLaM-7B-Instruct-preview](https://mindket.com/p/95ed1553-ca4e-4a47-9739-80a0aa93810c) | 93,626 | 207 | 7B | apache-2.0 | 2025-07-14 | | [GPT-X3-150M](https://mindket.com/p/36e26f24-6570-49bc-9611-515f3e31cfe9) | 3,000 | 104 | 149M | apache-2.0 | 2026-10-25 | | [Maple-Preview](https://mindket.com/p/3ee0840f-eaf7-4a88-a790-788a9c9f032d) | 1,867 | 433 | 20.2B | mit | 2026-08-04 | | [Meta-Llama-3.1-8B-Instruct-GGUF](https://mindket.com/p/51184fc4-b11d-4c65-93f0-2107bafe9b0e) | 322,920 | 433 | 8B | llama3.1 | 2024-12-01 | Numbers as Mindket last read them, on 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-23 - First published: 2026-09-03 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open DedeProGames/Kiyo-65M on Hugging Face](https://huggingface.co/DedeProGames/Kiyo-65M) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/41f070ee-1394-4748-845b-ecb0e0d53d8e) - [Its page on Mindket](https://mindket.com/p/41f070ee-1394-4748-845b-ecb0e0d53d8e)