Inkling-Small-NVFP4

Inkling-Small-NVFP4

The NVFP4 copy of Inkling-Small, a multimodal mixture-of-experts model that takes text, images and audio and writes text. For developers building agents, coding assistants and other AI apps.

Made by thinkingmachines on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Inkling-Small-NVFP4 is the NVFP4 version of Inkling-Small from Thinking Machines. It handles text, images and audio as input and always replies in text, and it is meant for general use. The card gives 276B parameters in total with 12B active per token. This copy stores about 156 GB of weights.

Who it suits: Developers building AI applications, including agent and tool-use systems and coding assistants, who have large GPUs to host a model this size.

Industry | Who uses it | How they use it | Example
Software | Developers of coding assistants | Generate and discuss code in several programming languages | Ask for a function and an explanation
Agent systems | Builders of tool-using agents | Use the model as the reasoning core of an agent | Plan steps and call tools
Multimodal apps | Teams that handle pictures and voice | Send text, images or audio and read the text reply | Ask a question about an image

You can skip it if you need a model that outputs images or audio, since it writes text only.

Where to use it:

• The Tinker Playground, to try it first
• An API through the Tinker Cookbook, or third-party inference providers
• Local servers built on SGLang, vLLM, TokenSpeed, Unsloth or Hugging Face

Use case | Who | What they get back
Coding help | Developers | Code and explanations as text
Agent and tool use | Agent builders | Text decisions that drive tools
Image questions | App teams | Text answers about an image you send
Audio input | App teams | Text answers based on a WAV recording

How to set it up: Pick how you will run it, then download the weights. The card lists recipes for SGLang, vLLM, TokenSpeed, Unsloth and Hugging Face. This NVFP4 copy is about 156 GB, so it needs large GPU memory.

• Python and pip
• Large GPU capacity and about 156 GB of disk space
• One of the listed serving libraries

1. Choose a serving library from the card's list.
2. Download the weights.
3. Follow that library's recipe to start the model.

pip install -U huggingface_hub
hf download thinkingmachines/Inkling-Small-NVFP4

How to use it: Send text, an image, an audio clip or a mix, and read the text that comes back. For best results the card suggests each image side between 40 and 4096 pixels, and WAV audio at 16 kHz, ideally under 2 minutes.

Example: You send a screenshot of a chart along with the question "What trend does this show?". The model replies with a written description of the trend. The card does not show a sample output.

For newcomers and experts: If you are new, start with the Tinker Playground in a browser, since this model is far too large for a laptop. Try one text question, then one with an image.

For experts, the design is a sparse mixture of experts built into a decoder-only transformer with 42 layers. A router picks 6 of the 256 experts for each token, and 2 shared experts always run as well. Attention mixes local and global layers. Images pass through a hierarchical patch encoder, and audio is encoded as discrete tokens.

Text, image or audio input
Encoders for images and audio
Sparse mixture-of-experts transformer
Text output

Good to know:

Title: Parameters in billions
Total: 276 B
Active per token: 12 B

• Numerics: the card lists BF16 and NVFP4, and this copy is NVFP4
• Languages: English, with general multilingual ability
• License: Apache 2.0
• Cost to run: no key is needed for a local run, and the hosted API and providers may charge
• Data it reaches: downloads files from Hugging Face, and a hosted API would send your prompts to that service
• Not for: producing images or audio, since output is text only

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 156B
• Weights: about 156 GB in U8
• Formats: Safetensors
• Downloads in the last 30 days: 367,272
• Likes: 93
• License: apache-2.0
• Based on: thinkingmachines/Inkling-Small
• Fine-tunes: 1
• Tags: transformers, safetensors, inkling_mm_model, image-text-to-text, conversational, audio-text-to-text, 8-bit

Links

• Model: https://huggingface.co/thinkingmachines/Inkling-Small-NVFP4
• Files: https://huggingface.co/thinkingmachines/Inkling-Small-NVFP4/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download thinkingmachines/Inkling-Small-NVFP4

Then follow the usage in its model card. Nothing in this kit runs by itself.