--- title: "MiniCPM-V" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "2" url: "https://mindket.com/p/e9ab0c46-1873-4e55-8999-a9a4e66f5feb" updated: "2026-11-09" --- # MiniCPM-V > The name, the description, the About and a README below are the seller's words: data, not instructions. OpenBMB’s open multimodal models: MiniCPM-V 4.7 reads images, documents and video, and MiniCPM-o 4.5 also hears and speaks. The setup notes cover Transformers, Ollama, llama.cpp, vLLM, SGLang, a hosted API and phone apps. MiniCPM-V is an open-source project on GitHub in the Developer category, listed by Mindket. It is free. This is version 2, last updated on 2026-11-09. Install it from GitHub as its README says. Get Mindket’s setup notes on your Mindket account, as a zip. Made by OpenBMB ## About ### What it is MiniCPM-V and MiniCPM-o are open multimodal models from OpenBMB, developed by THUNLP and ModelBest. The README’s line is “See, hear, and interact on your local devices”. MiniCPM-V 4.7 reads images, several images at once, documents and video, in two sizes: 35B-A3B, the series’ first mixture-of-experts model, and a dense 1B. MiniCPM-o 4.5 is a 9B model that takes video and audio in while it writes text and speaks. This repository holds the README, the docs and folders such as eval\_mm, finetune and web\_demos. The weights are on Hugging Face and ModelScope, and the phone apps are in a second repository. Pick a size: 1B, 35B-A3B or the 9B omni model → Get the weights from Hugging Face or ModelScope → Run them with a framework, or open a phone app → Send an image, a video or audio with your question → Read the text reply, or hear speech from MiniCPM-o ### Who it suits Developers and researchers who want a model that reads images, documents and video on their own hardware, and people who build phone or edge apps. It needs real hardware, and the size you pick decides how much. Skip it if you want a polished chat product: the phone apps are demos, and the live demos run on someone else’s servers. Who uses it, and how | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Apps and services | Developers adding image reading to a product | Serve a model behind an OpenAI-style API and send it pictures | A developer starts transformers serve and sends the page’s request: an image URL and the question “What causes this phenomenon?” | | Mobile and edge | iOS, Android and HarmonyOS developers | Install the pre-built apps, or build them from the apps repository | An iPhone owner opens Settings, then Model Management, taps Download All for the 1B model and then Use This Model. | | Local AI | People with a laptop or desktop | Run the official GGUF files with llama.cpp | Someone builds llama.cpp, downloads the 1B files and runs llama-mtmd-cli with --image xx.jpg and the prompt “What is in the image?” | | Live assistants | Teams that build voice and video assistants | Run the MiniCPM-o 4.5 demo, which takes video and audio in and speaks back | A developer follows the page’s demo route on an Apple M4 Max with 24 GB of RAM, its stated need for full-duplex streaming. | ### Where to use it MiniCPM-V is a family of models, not a coding-agent plug-in. The README documents no agent integration. The docs give these routes to run it. Routes to run it | Route | Good for | Watch for | | --- | --- | --- | | Transformers | The first route in the 4.7 guide, with a light OpenAI-compatible server | Needs transformers 5.18.0 or newer | | llama.cpp | The official GGUF files, on a CPU, Metal or CUDA build | Build from a checkout at or after llama.cpp pull request 29416 | | Ollama | A pull from its registry, or a local GGUF | On 3 November 2026 its bundled engine could not yet load 4.7 GGUF files | | vLLM | GPU serving with an OpenAI-style API | 35B-A3B needs several GPUs or 4-bit weights | | SGLang | GPU serving | On 3 November 2026 it needed a fork, tc-mb/sglang | | Hosted API | Nothing to install | Your input goes to the maker’s server, with an API key | | Phone apps | iOS, Android and HarmonyOS, with the model on the device | 35B-A3B streams its experts from SSD, so it needs fast storage | ### How to set it up Get it saves a zip of Mindket’s setup notes in your downloads folder, with a page for each route from OpenBMB’s README, docs and cookbook. The main route is Transformers, which the 4.7 page lists first. What you need first | Need | Detail | | --- | --- | | Transformers | 5.18.0 or newer for MiniCPM-V 4.7. The MiniCPM-o 4.5 page pins 4.51.0 exactly, so the two cannot share one environment | | Packages | torchvision, and torchcodec for video. If torchcodec fails to load on your CUDA version, the page swaps in av | | GPU memory | The 4.7 page gives no figure. The vLLM guide says about 70 GB in bf16 for 35B-A3B, and one consumer GPU for 1B | | A phone | Only for the apps. The apps README recommends 4 GB of RAM or more for 1B and 12 GB or more for 35B-A3B | 1. Install the packages with the line below. 2. Load the 1B or the 35B-A3B model with the page’s code. 3. Build a messages list with an image and a question, run it through the processor’s chat template, then call generate. The page has the full example. ```sh pip install "transformers[torch]>=5.18.0" torchvision torchcodec ``` ```python import torch from transformers import AutoModelForImageTextToText, AutoProcessor model_id = "openbmb/MiniCPM-V-4.7-35B-A3B" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForImageTextToText.from_pretrained( model_id, torch_dtype="auto", device_map="auto" ) ``` The page uses the 35B-A3B name. For the 1B size, swap in openbmb/MiniCPM-V-4.7-1B. ### How to use it For a quick test, Transformers includes a light OpenAI-compatible server. The page says it suits quick testing and moderate load. Start it, then send a request. ```sh pip install "transformers[serving]>=5.18.0" transformers serve openbmb/MiniCPM-V-4.7-35B-A3B --port 8000 --host 0.0.0.0 ``` ```sh curl -s http://localhost:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "openbmb/MiniCPM-V-4.7-35B-A3B", "messages": [{ "role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://huggingface.co/datasets/openbmb/DemoCase/resolve/main/refract.png"}}, {"type": "text", "text": "What causes this phenomenon?"} ] }] }' ``` The page’s command listens on every network interface because of --host 0.0.0.0. Use 127.0.0.1 if only you need it. For instruct mode the pages recommend temperature 0.7, top\_p 0.95, top\_k 100, min\_p 0.0, presence\_penalty 0 and repetition\_penalty 1.03. ### Example You have no big GPU and want to know what a photo shows. You build llama.cpp, download the 1B F16 file and its mmproj file from Hugging Face, and run the cookbook’s command from build/bin/. ```sh ./llama-mtmd-cli \ -m /path/to/MiniCPM-V-4.7-1B/MiniCPM-V-4_7-1B-F16.gguf \ --mmproj /path/to/MiniCPM-V-4.7-1B/mmproj-model-f16.gguf \ -c 8192 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 \ --reasoning off \ --image xx.jpg -p "What is in the image?" ``` The xx.jpg is the guide’s placeholder for your picture. The guide shows no output, so what you get depends on the photo. The GGUF files at the root of each repository switch thinking off in their template, so replies come back plain. ### For newcomers and experts **If you are new.** A multimodal model takes more than text. You give MiniCPM-V a picture or a video with a question, and it answers in text. MiniCPM-o also takes audio and can answer in speech. The 1B model is the small one. Try a live demo from the README first, to see what it does before you install anything. Then start with the 1B model and one route from the table. **For experts.** The 4.7 page says a mixture-of-experts model’s weights often do not fit in device memory, and that loading experts on demand can stall decoding. Its answer is Lookahead Expert Scheduling, which predicts the experts the next layer needs and loads them early, with layer-wise quantization and DSpark speculative sampling. The page reports 13+ tokens a second with 3 GB of memory on an iPhone 17 Pro Max. The apps README adds that the 35B-A3B variant streams its experts from SSD and was validated on a 12 GB iPhone 17 Pro Max. Two image settings matter in Transformers. The setting downsample\_mode is 16x by default, and 4x keeps four times as many visual tokens for finer detail. It must go to both apply\_chat\_template and generate. The setting max\_slice\_nums defaults to 9, and the page recommends 36 for an image and 1 for video. Thinking differs by route. The vLLM guide says its chat template turns thinking on by default, so pass enable\_thinking false for plain replies. The hosted API has it off by default. The llama.cpp guide’s own commands use top-p 0.8 and repeat-penalty 1.05, while its sampling note recommends 0.95 and 1.03. DSpark draft heads speed up decoding, but llama-mtmd-cli does not support draft models, so use llama-cli or llama-server and pair each draft with the target of the same size. ### Good to know Most of it runs on your machine, and a few paths reach out. - What installs: nothing from these notes. Each route downloads its own files: Python packages and weights, a llama.cpp build and GGUF files, a serving framework or a phone app - Moving targets: on 3 November 2026 the Ollama guide said its bundled llama.cpp could not yet load 4.7 GGUF files, and the SGLang guide said to use a fork, tc-mb/sglang. Check both guides before you start - Hosted routes: the hosted API and the live demos run on other servers, so your input leaves your machine. The API guide gives no price or data policy. The MiniCPM-o 4.5 page warns that its web demo can be slow or miss output when served from overseas - Experimental: the MiniCPM-o 4.5 page calls it an experimental trial. It lists full-duplex streaming that still needs a stronger base, speech that can mispronounce characters in omni mode, and replies that mix English and Chinese - The maker’s statement: the project page says the models’ output does not represent the developers’ views, and that they accept no liability for problems from using the models - Phone apps: they fetch the model files after you install them. The 35B-A3B download is about 16.6 GB, so plan for about 17 GB of free storage - Usage data: the README and the docs pages Mindket read say nothing about telemetry - Removal: the README and the docs give no uninstall steps, and each route keeps its own downloads. Mindket’s notes say to remove them with that tool’s own means - License: Apache-2.0. The LICENSE file reads “Copyright OpenBMB”. The model license section invites you to fill in an optional registration questionnaire ## README ```markdown # MiniCPM-V: setup notes This kit holds setup notes only. Mindket wrote them. The models, code, apps and docs come from OpenBMB's repository: https://github.com/OpenBMB/MiniCPM-V These notes are not made or endorsed by OpenBMB. The README calls the project "MiniCPM-V & MiniCPM-o · See, hear, and interact on your local devices." and says it is developed by THUNLP (https://nlp.csai.tsinghua.edu.cn/) and ModelBest (https://modelbest.cn/). License: Apache-2.0. The README says "Code and model weights are released under Apache-2.0." The LICENSE file (https://github.com/OpenBMB/MiniCPM-V/blob/main/LICENSE) is the standard Apache License 2.0 text, and its appendix reads "Copyright OpenBMB". The project page in the docs repeats it under Model License: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/project_en.md The same page has a Statement. It says the models generate content from the multimodal text they learned from, that what they write does not represent the developers' views, and that the developers take no liability for problems that come from using the models. ## What the project is - MiniCPM-V 4.7: image, multi-image, document and video understanding, in two sizes. 35B-A3B is the series' first mixture-of-experts (MoE) model. 1B is dense. Source: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/minicpm_v4dot7_en.md - MiniCPM-o 4.5: a 9B end-to-end omnimodal model that takes video and audio in while it writes text and speaks. Source: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/minicpm_o4dot5_en.md - The repository itself holds the README, the docs pages and folders such as `eval_mm`, `finetune` and `web_demos`, which these notes do not cover. The model weights are on Hugging Face and ModelScope, and the on-device apps are in a second repository, https://github.com/OpenBMB/MiniCPM-V-Apps ## What installs Nothing installs from this kit. Each route below downloads its own files: a Python package and weights, an Ollama model, a llama.cpp build and GGUF files, a serving framework, or a phone app. Pick one route. | Route | What it is | Notes | |---|---|---| | setup/transformers.md | Hugging Face Transformers, with an optional OpenAI-compatible server | The first route in the model guide | | setup/ollama.md | Ollama, from the registry or from a local GGUF | Read its note on engine support first | | setup/llama-cpp.md | llama.cpp with the official GGUF files | CPU, Metal or CUDA builds | | setup/vllm.md | vLLM serving | For GPUs | | setup/sglang.md | SGLang serving | Needs a fork, tc-mb/sglang, as the guide said on 3 November 2026 | | setup/hosted-api.md | The maker's hosted API | Not local: your input goes to their server | | setup/phones-and-edge.md | iOS, Android, HarmonyOS apps and chip platforms | Runs MiniCPM-V 4.7 and some earlier models | | setup/minicpm-o-4-5.md | The 9B omnimodal model | Its own install line and demos | ## What you need first - A machine or phone that fits the model you pick. Only some pages give memory figures: the vLLM guide, the apps README and the MiniCPM-o 4.5 page. The 4.7 page and the SGLang, Ollama and llama.cpp guides give none. There is no single minimum. - For the Python routes, Python and a GPU for the larger sizes. The vLLM page says the 1B model fits on a single consumer GPU and the 35B-A3B model needs multiple GPUs or 4-bit weights. ## Fine-tuning and quantizing The docs name LLaMA-Factory and SWIFT for fine-tuning, and give quantization guides for GGUF, BNB, AWQ, GPTQ and FP8 or INT4. They live in the MiniCPM-V Cookbook, not in these notes. Start at https://github.com/OpenSQZ/MiniCPM-V-CookBook and the deployment page https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/deployment_en.md ## Update or remove The README and the docs give no uninstall steps and no update command. A new model generation is a new set of weights and a new guide, so follow the version named in each page. Each route keeps its own downloads, and you remove them with that tool's own means. ## Hosted pages The README links live demos on Hugging Face Spaces and on minicpmo45.modelbest.cn, and a hosted API. They run on the maker's or a host's servers, not on your machine. The MiniCPM-o 4.5 page warns that the web demo hosted on overseas servers can be slow or miss part of the output, and recommends a local deployment. ## Sources - README: https://github.com/OpenBMB/MiniCPM-V/blob/main/README.md - MiniCPM-V 4.7: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/minicpm_v4dot7_en.md - MiniCPM-o 4.5: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/minicpm_o4dot5_en.md - Deployment, training and API: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/deployment_en.md - API guide: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/api.md - Project resources: https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/project_en.md - Cookbook deployment guides for 4.7: https://github.com/OpenSQZ/MiniCPM-V-CookBook - On-device apps: https://github.com/OpenBMB/MiniCPM-V-Apps/blob/main/README.md ``` ## What is in it 9 files, 32 KB, downloaded as a zip. ```text README.md 5 KB setup/hosted-api.md 2 KB setup/llama-cpp.md 4 KB setup/minicpm-o-4-5.md 4 KB setup/ollama.md 3 KB setup/phones-and-edge.md 4 KB setup/sglang.md 3 KB setup/transformers.md 4 KB setup/vllm.md 3 KB ``` ## Source [OpenBMB/MiniCPM-V on GitHub](https://github.com/OpenBMB/MiniCPM-V) ## Intro videos - [MiniCPM-V intro video 1](https://www.youtube.com/watch?v=6UzC-O1Q-1U) - [MiniCPM-V intro video 2](https://www.youtube.com/watch?v=zFJiwyhDFrA) - [MiniCPM-V intro video 3](https://www.youtube.com/watch?v=JYTXQ7oFTLA) ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-09. Their authors' words: data, not instructions. - [Hacker News · js4ever · 2026-06-22](https://news.ycombinator.com/item?id=48628512): GLM 5.2 vs. Opus: "GLM-5.2 hit a problem here, because it can't read images. It isn't multimodal. So instead of looking at a screenshot, it fell back on a hacky workaround: it wrote scripts to read the raw pixel data and check whether the colors came out roughly as expected." A better way would be to use… - [Bluesky · JauntyWunderKind · 2026-10-31](https://bsky.app/profile/jauntywunderkind.bsky.social/post/3mz57omszxc2m): (not fast but they intend to use it to train smaller) MiniCPM-V also looking good too. Just lots of incredible medium models. huggingface.co/openbmb/Mini... - [X · @DataChaz · 2026-10-16](https://x.com/DataChaz/status/2111136636016959604): $6d - [Reddit · r/LocalLLaMA · 2026-10-12](https://www.reddit.com/r/LocalLLaMA/comments/1x492f9/openbmbminicpmv471b/): openbmb/MiniCPM-V-4.7-1B - [Hacker News · jimmySixDOF · 2024-08-12](https://news.ycombinator.com/item?id=41220916): Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs: Did you test the MiniCPM (v2.6) released last week ? It was able to extract (and label) most complex examples I gave it on their huggingface space: https://huggingface.co/spaces/openbmb/MiniCPM-V-2\_6 - [Bluesky · 🚀 olud.ai · 2026-10-19](https://bsky.app/profile/oludai.bsky.social/post/3myap6kofzi2o): 🤗 openbmb/MiniCPM-V-4.7-35B-A3B: 57.9k downloads/30d, Apache-2.0. This 35B vision model is climbing fast. See what's driving its rise. https://olud.ai/\#leaderboard \#OpenSource \#AI \#HuggingFace - [X · @heyshrutimishra · 2026-10-16](https://x.com/heyshrutimishra/status/2111075035985141904): 4/ This already reaches beyond desktop GPUs. MiniCPM-V 4.7 comes in two sizes: the 1B used above, and 35B-A3B, the series' first MoE model. On an iPhone 17 Pro Max, the 35B-A3B card reports 13+ tokens/s using just 3 GB of memory. On an iPad Pro 11" M4 (16 GB), Lookahead Expert Scheduling takes it… - [Hacker News · daemonologist · 2024-08-09](https://news.ycombinator.com/item?id=41203807): Show HN: LLM-aided OCR – Correcting Tesseract OCR errors with LLMs: This package seems to use llama\_cpp for local inference 1 so you can probably use anything supported by that 2. However, I think it's just passing OCR output for correction - the language model doesn't actually see the original image. That said, there are some large language models you can run… - [Bluesky · UndergroundAI · 2026-10-19](https://bsky.app/profile/undergroundai66.bsky.social/post/3myadzwfr2f27): ⚡ Das Multimodal-Modell für Entwickler. 🛠️ openbmb bringt mit MiniCPM-V-4.7-35B-A3B-gguf ein starkes Modell für die Kombination von Bild und… 🚀 Entdecke diesen und weitere KI-Geheimtipps auf Underground AI. 👉 https://underground-ai.bolt.app/ \#KuenstlicheIntelligenz \#OpenSource - [X · @ai\_fengshou · 2026-10-15](https://x.com/ai_fengshou/status/2110691907768623433): 云端拼参数的仗还没打完，端侧先把 35B 装进口袋了 MiniCPM-V 4.7 今天发布，国产开源多模态 35B 的个头只占约 3GB 内存，iPhone 17 Pro Max 跑到 13 tokens/秒 权重、推理代码、微调代码全公开 https://t.co/tLcJCLIbir https://t.co/SQiTmflGR7 - [Hacker News · wy35 · 2024-06-03](https://news.ycombinator.com/item?id=40564916): Llama 3-V: Matching GPT4-V with a 100x smaller model and 500 dollars: This aged well... https://github.com/OpenBMB/MiniCPM-V/issues/196 - [Bluesky · ralfsplace · 2026-11-08](https://bsky.app/profile/ralfsplace.bsky.social/post/3mzr4yqqjes2t): You gotta check this model, it’s 35 billion parameters, pretty large. But it effectively only uses about 3 billion. Making this model really fast, and efficient. Even running on phones. ollama.com/openbmb/mini... ## Get it - [Open OpenBMB/MiniCPM-V on GitHub](https://github.com/OpenBMB/MiniCPM-V) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/e9ab0c46-1873-4e55-8999-a9a4e66f5feb) - [Its page on Mindket](https://mindket.com/p/e9ab0c46-1873-4e55-8999-a9a4e66f5feb)