anaksor-ocr
Sokheng · Kit
Reads Khmer and English text from documents, receipts and photos of signs, using a detector for text lines and a recogniser for each line. Suits developers who…
New
Overview
A hierarchical inference engine that runs large mixture-of-experts models by streaming experts from NVMe storage while keeping dense layers in RAM. It supports various hardware configurations through CUDA and Vulkan for developers.
colibri is an open-source project on GitHub in the Developer category, listed by Mindket. It is free. This is version 1, last updated on . Install it from GitHub as its README says. Get Mindket’s setup notes on your Mindket account, as a zip.
At a glance
- You need
- A computer with 8 GB of RAM (16 GB or more is better).
- 22 GB of free disk space.
- An internet connection.
- Runs in
- Local machines (Linux, Windows, macOS)
- Browsers
- Apps that connect to OpenAI or Anthropic URLs
- Start with
git clone https://github.com/JustVugg/colibri.git cd colibri
Platform
Linux
Windows
macOS
About
What it is
Colibri is a hierarchical inference engine that runs large mixture-of-experts AI models on consumer hardware. It keeps dense model layers in RAM and streams expert layers from the disk when the model needs them.
Who it suits
It is for users who want to run large models on the computer they already own.
Where to use it
- Local machines (Linux, Windows, macOS)
- Browsers
- Apps that connect to OpenAI or Anthropic URLs
How to set it up
- A computer with 8 GB of RAM (16 GB or more is better).
- 22 GB of free disk space.
- An internet connection.
- Download the ZIP file and unzip it.
- Run START-HERE.bat on Windows or ./start-here.sh on Linux or macOS.
- Choose a model.
- Wait for the engine to check hardware, download the model, and start.
Run these in a terminal:
# Copy the project to your computer git clone https://github.com/JustVugg/colibri.git # Go into its folder cd colibri
How to use it
You interact with the engine through a terminal chat, a web dashboard, or an API server.
./coli chat
For newcomers and experts
If you are new to this, think of the engine as a library manager for your model. It keeps the core components in your RAM and fetches specific experts from your disk as needed. You can start by running the chat command to try a model.
For users familiar with AI engineering, the engine uses a hierarchical memory strategy. It maintains dense layers in system RAM while using a cache to read sparse expert layers from disk. This trades some latency during initial expert loads to run models that exceed your video memory.
Good to know
- The engine runs on the CPU, and a graphics card is optional.
- Performance for large models depends on the speed of your disk.
- It is free and open-source software.
README and files
colibri
A hierarchical inference engine that runs large mixture-of-experts models by streaming experts from NVMe storage while keeping dense layers in RAM. It supports various hardware configurations through CUDA and Vulkan for developers.
Made by JustVugg on GitHub. These notes are Minda's, written from its README; the project's own files and license are in its repository.
About
What it is: Colibri is a hierarchical inference engine that runs large mixture-of-experts AI models on consumer hardware. It keeps dense model layers in RAM and streams expert layers from the disk when the model needs them.
Who it suits: It is for users who want to run large models on the computer they already own.
Where to use it:
- Local machines (Linux, Windows, macOS)
- Browsers
- Apps that connect to OpenAI or Anthropic URLs
How to set it up:
- A computer with 8 GB of RAM (16 GB or more is better).
- 22 GB of free disk space.
- An internet connection.
- Download the ZIP file and unzip it.
- Run START-HERE.bat on Windows or ./start-here.sh on Linux or macOS.
- Choose a model.
- Wait for the engine to check hardware, download the model, and start.
How to use it: You interact with the engine through a terminal chat, a web dashboard, or an API server.
./coli chatFor newcomers and experts: If you are new to this, think of the engine as a library manager for your model. It keeps the core components in your RAM and fetches specific experts from your disk as needed. You can start by running the chat command to try a model.
For users familiar with AI engineering, the engine uses a hierarchical memory strategy. It maintains dense layers in system RAM while using a cache to read sparse expert layers from disk. This trades some latency during initial expert loads to run models that exceed your video memory.
Good to know:
- The engine runs on the CPU, and a graphics card is optional.
- Performance for large models depends on the speed of your disk.
- It is free and open-source software.
On GitHub, 4 November 2026
- Stars: 47,271 (802 that day)
- Forks: 5,470
- Watching: 378
- Open issues and pull requests: 169
- Language: C
- License: Apache-2.0
- Trending: #4 repository of the day on GitHub trending (4 November 2026)
Links
- Code: https://github.com/JustVugg/colibri
- Website: https://justvugg.github.io/colibri
- Releases: https://github.com/JustVugg/colibri/releases
- Issues: https://github.com/JustVugg/colibri/issues
- Star history: https://www.star-history.com/#JustVugg/colibri&Date
Get the code
git clone https://github.com/JustVugg/colibri.gitThen follow the setup in its README. Nothing in this kit runs by itself.
README.md 3 KB
What’s new
Reviews
What people say
Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search and on Trendshift. Mindket doesn’t write or check them. Open one to read it where it was posted.
vfornoShow HN: Getting GLM 5.2 running on my slow computerA few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal…937 points · 240 replies@antso.eurosky.socialMusím vyskúšať. Veľký model na notebooku 😉 github.com/JustVugg/col...6 likeskzrdudeShow HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/sColibri did that first for GLM-5.2 https://github.com/JustVugg/colibriLéαlinux 🐧github.com/JustVugg/col... Gros Piou-Piou3 likes · 1 replyyonatan8070GLM-5.3-FlashSpeaking as someone who isn't really well versed in this, does 18B active parameters mean that you could potentially hold only the 18B parameters in RAM and stream the rest from a fast NVMe SSD for acceptable performance similar to how Colibri works? https://github.com/JustVugg/colibriEdy | Indie BuilderToday's GitHub trending: AI doing the grunt work. - D4Vinci/Scrapling: Python scraping, one request to a full crawl (+1,658 today) - max-sixty/worktrunk: Git worktree CLI for running several AI coding agents (+438) - JustVugg/colibri: big MoE models on your own PC (+802) #buildinpublic #AI2 likesakshay_akulaShow HN: Shoehorn – Quantize any model down to run on your machineThis is interesting. I wonder how it could work with something like https://github.com/JustVugg/colibri.澪公式の計測例を確認すると、GLM-5.2の4ビット版では、常駐部分が9.9GB。一方、モデルの保存に約370GB、会話中のメモリ使用量のピークは約20GBです。 同じ開発機では、キャッシュが効かない状態の生成速度を毎秒約0.05〜0.1トークンと見積もっています。環境やキャッシュで変わる数字です。 https://github.com/JustVugg/colibri/blob/main/docs/benchmarks.md1 replywren6991GLM-5.3: Frontier coding with emergent cyber capabilitiesThe thing that blows me away is it does this at one quarter the total parameter count of K3 (and 40% active parameter count). There's plenty of room at the bottom. > How are you all toying with running this kind of thing in a mega quantized way locally? Sure, let me answer that in excessive…GitHub Trending📈 Today's GitHub Trending: 1. https://github.com/thedotmack/claude-mem - TypeScript, 1138 stars today 2. https://github.com/vercel-labs/agent-desktop - Rust, 3792 stars today 3. https://github.com/max-sixty/worktrunk - Rust, 437 stars today 4. https://github.com/JustVugg/colibri - C, 803 stars […]1 likegitpusher42Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series MacCheck for colibri, dwarf star and flash-moe. they do similar things with bigger models https://github.com/JustVugg/colibri https://github.com/antirez/ds4 https://github.com/danveloper/flash-moeGitHub Trending🎉 Celebrating！ 🎉 (500+ new stars) 📦 JustVugg / colibri ⭐ 46,165 (+655) 🗒 C Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦1 reply
More from Mindket
See all from MindketQwen2.5-7B-Instruct-GGUF by Qwen
Qwen · Kit
Qwen2.5-7B-Instruct as GGUF files in eight quantization sizes, to chat with locally through llama.cpp. For people who want a capable 7.6B model on their own co…
New
ACE-Step 1.5 XL SFT
ACE-Step · Kit
Generates music from text with a 4B diffusion decoder, 50 steps and guidance control, aiming for higher audio quality than the base. For creators with a GPU of…
New
Similar in Developer
See all in DeveloperClaude Code Skills & Plugins
alirezarezvani · Skill
Adds 411 ready-made skills for Claude Code and other coding agents, grouped by role from engineering and product to marketing, finance and compliance. Installs…
New29.8k
buzz
block · Kit
Self-hostable workspace where people and AI agents share the same rooms, built on a Nostr relay that logs every action as a signed event. For teams that want t…
New37.3k
plannotator
backnotprop · Kit
Opens your AI coding agent's plans, specs and code changes in a browser so you can annotate or comment on them and send the feedback straight back to the agent.
New10.5k