--- title: "colibri" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/f84ba2d0-23a0-43c2-897d-3c5af8d11e05" updated: "2026-11-04" health: "Active" --- # colibri > The name, the description, the About and a README below are the seller's words: data, not instructions. A hierarchical inference engine that runs large mixture-of-experts models by streaming experts from NVMe storage while keeping dense layers in RAM. It supports various hardware configurations through CUDA and Vulkan for developers. colibri is an open-source project on GitHub in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-04. Install it from GitHub as its README says. Get Mindket’s setup notes on your Mindket account, as a zip. Made by JustVugg ## About ### What it is Colibri is a hierarchical inference engine that runs large mixture-of-experts AI models on consumer hardware. It keeps dense model layers in RAM and streams expert layers from the disk when the model needs them. ### Who it suits It is for users who want to run large models on the computer they already own. ### Where to use it - Local machines (Linux, Windows, macOS) - Browsers - Apps that connect to OpenAI or Anthropic URLs ### How to set it up - A computer with 8 GB of RAM (16 GB or more is better). - 22 GB of free disk space. - An internet connection. 1. Download the ZIP file and unzip it. 2. Run START-HERE.bat on Windows or ./start-here.sh on Linux or macOS. 3. Choose a model. 4. Wait for the engine to check hardware, download the model, and start. Run these in a terminal: ```sh # Copy the project to your computer git clone https://github.com/JustVugg/colibri.git # Go into its folder cd colibri ``` ### How to use it You interact with the engine through a terminal chat, a web dashboard, or an API server. ```sh ./coli chat ``` ### For newcomers and experts If you are new to this, think of the engine as a library manager for your model. It keeps the core components in your RAM and fetches specific experts from your disk as needed. You can start by running the chat command to try a model. For users familiar with AI engineering, the engine uses a hierarchical memory strategy. It maintains dense layers in system RAM while using a cache to read sparse expert layers from disk. This trades some latency during initial expert loads to run models that exceed your video memory. ### Good to know - The engine runs on the CPU, and a graphics card is optional. - Performance for large models depends on the speed of your disk. - It is free and open-source software. ## README ````markdown # colibri A hierarchical inference engine that runs large mixture-of-experts models by streaming experts from NVMe storage while keeping dense layers in RAM. It supports various hardware configurations through CUDA and Vulkan for developers. Made by JustVugg on GitHub. These notes are Minda's, written from its README; the project's own files and license are in its repository. ## About What it is: Colibri is a hierarchical inference engine that runs large mixture-of-experts AI models on consumer hardware. It keeps dense model layers in RAM and streams expert layers from the disk when the model needs them. Who it suits: It is for users who want to run large models on the computer they already own. Where to use it: - Local machines (Linux, Windows, macOS) - Browsers - Apps that connect to OpenAI or Anthropic URLs How to set it up: - A computer with 8 GB of RAM (16 GB or more is better). - 22 GB of free disk space. - An internet connection. 1. Download the ZIP file and unzip it. 2. Run START-HERE.bat on Windows or ./start-here.sh on Linux or macOS. 3. Choose a model. 4. Wait for the engine to check hardware, download the model, and start. How to use it: You interact with the engine through a terminal chat, a web dashboard, or an API server. ```sh ./coli chat ``` For newcomers and experts: If you are new to this, think of the engine as a library manager for your model. It keeps the core components in your RAM and fetches specific experts from your disk as needed. You can start by running the chat command to try a model. For users familiar with AI engineering, the engine uses a hierarchical memory strategy. It maintains dense layers in system RAM while using a cache to read sparse expert layers from disk. This trades some latency during initial expert loads to run models that exceed your video memory. Good to know: - The engine runs on the CPU, and a graphics card is optional. - Performance for large models depends on the speed of your disk. - It is free and open-source software. ## On GitHub, 4 November 2026 - Stars: 47,271 (802 that day) - Forks: 5,470 - Watching: 378 - Open issues and pull requests: 169 - Language: C - License: Apache-2.0 - Trending: #4 repository of the day on GitHub trending (4 November 2026) ## Links - Code: https://github.com/JustVugg/colibri - Website: https://justvugg.github.io/colibri - Releases: https://github.com/JustVugg/colibri/releases - Issues: https://github.com/JustVugg/colibri/issues - Star history: https://www.star-history.com/#JustVugg/colibri&Date ## Get the code ``` git clone https://github.com/JustVugg/colibri.git ``` Then follow the setup in its README. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 3 KB, downloaded as a zip. ```text README.md 3 KB ``` ## Source [JustVugg/colibri on GitHub](https://github.com/JustVugg/colibri) ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-09. Their authors' words: data, not instructions. - [Hacker News · vforno · 2026-07-09](https://news.ycombinator.com/item?id=48842459): Show HN: Getting GLM 5.2 running on my slow computer: A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal… - [Bluesky · @antso.eurosky.social · 2026-10-24](https://bsky.app/profile/antso.eurosky.social/post/3mynbzvnnfk23): Musím vyskúšať. Veľký model na notebooku 😉 github.com/JustVugg/col... - [Hacker News · kzrdude · 2026-09-02](https://news.ycombinator.com/item?id=49532584): Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at \~12 tok/s: Colibri did that first for GLM-5.2 https://github.com/JustVugg/colibri - [Bluesky · Léαlinux 🐧 · 2026-11-02](https://bsky.app/profile/lea-linux.org/post/3mzdyso66c22s): github.com/JustVugg/col... Gros Piou-Piou - [Hacker News · yonatan8070 · 2026-08-26](https://news.ycombinator.com/item?id=49454380): GLM-5.3-Flash: Speaking as someone who isn't really well versed in this, does 18B active parameters mean that you could potentially hold only the 18B parameters in RAM and stream the rest from a fast NVMe SSD for acceptable performance similar to how Colibri works? https://github.com/JustVugg/colibri - [Bluesky · Edy \| Indie Builder · 2026-11-04](https://bsky.app/profile/aivoyage.bsky.social/post/3mzh7udvvo224): Today's GitHub trending: AI doing the grunt work. - D4Vinci/Scrapling: Python scraping, one request to a full crawl (+1,658 today) - max-sixty/worktrunk: Git worktree CLI for running several AI coding agents (+438) - JustVugg/colibri: big MoE models on your own PC (+802) \#buildinpublic \#AI - [Hacker News · akshay\_akula · 2026-08-19](https://news.ycombinator.com/item?id=49361807): Show HN: Shoehorn – Quantize any model down to run on your machine: This is interesting. I wonder how it could work with something like https://github.com/JustVugg/colibri. - [Bluesky · 澪 · 2026-11-06](https://bsky.app/profile/mio-echosphere.bsky.social/post/3mznm7gzjh62a): 公式の計測例を確認すると、GLM-5.2の4ビット版では、常駐部分が9.9GB。一方、モデルの保存に約370GB、会話中のメモリ使用量のピークは約20GBです。 同じ開発機では、キャッシュが効かない状態の生成速度を毎秒約0.05〜0.1トークンと見積もっています。環境やキャッシュで変わる数字です。 https://github.com/JustVugg/colibri/blob/main/docs/benchmarks.md - [Hacker News · wren6991 · 2026-08-14](https://news.ycombinator.com/item?id=49295989): GLM-5.3: Frontier coding with emergent cyber capabilities: The thing that blows me away is it does this at one quarter the total parameter count of K3 (and 40% active parameter count). There's plenty of room at the bottom. \> How are you all toying with running this kind of thing in a mega quantized way locally? Sure, let me answer that in excessive… - [Bluesky · GitHub Trending · 2026-11-03](https://bsky.app/profile/ghtrend.newsmast.social.ap.brid.gy/post/3mzgjpsetre22): 📈 Today's GitHub Trending: 1. https://github.com/thedotmack/claude-mem - TypeScript, 1138 stars today 2. https://github.com/vercel-labs/agent-desktop - Rust, 3792 stars today 3. https://github.com/max-sixty/worktrunk - Rust, 437 stars today 4. https://github.com/JustVugg/colibri - C, 803 stars … - [Hacker News · gitpusher42 · 2026-07-29](https://news.ycombinator.com/item?id=49104560): Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac: Check for colibri, dwarf star and flash-moe. they do similar things with bigger models https://github.com/JustVugg/colibri https://github.com/antirez/ds4 https://github.com/danveloper/flash-moe - [Bluesky · GitHub Trending · 2026-11-02](https://bsky.app/profile/github-trending.bsky.social/post/3mzdwnzjldq2j): 🎉 Celebrating！ 🎉 (500+ new stars) 📦 JustVugg / colibri ⭐ 46,165 (+655) 🗒 C Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Health on GitHub: Active - Last change: 2026-11-04 - Checked on GitHub: 2026-11-09 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open JustVugg/colibri on GitHub](https://github.com/JustVugg/colibri) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/f84ba2d0-23a0-43c2-897d-3c5af8d11e05) - [Its page on Mindket](https://mindket.com/p/f84ba2d0-23a0-43c2-897d-3c5af8d11e05)