flint-mixed-3-4bit

flint-mixed-3-4bit

A community MLX build of Recursive's 40B Flint coding model that shrinks the experts to 3 bits, so it fits a 24 GB or 32 GB Apple Silicon Mac. Made for developers who want a local coding model on a Mac.

Made by mlx-community on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a community build of Recursive Flint, a 40B coding model, for MLX on Apple Silicon. Only the routed experts, about 95 percent of the weights, were cut from 4 to 3 bits. The result takes 17.8 GB on disk and peaks near 19.5 GB in use. It is unofficial and not endorsed by Recursive.

Who it suits: It suits developers whose Mac, with Apple Silicon and either 24 or 32 GB of memory, can host a large coding model locally rather than calling a cloud service.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a local coding assistant on a Mac | Writing a function from a plain request
Dev tools | Agent users | Point a coding agent at a local server | Using a local endpoint from an agent tool
Research | ML hobbyists | Study mixed-bit quantization of a mixture of experts | Comparing perplexity across builds
Teams with privacy needs | Engineering teams | Keep source code on the laptop | Reviewing code without sending it out

Skip it if you need the official build, or if your Mac has less than 24 GB, since the card has not been tested on a 24 GB machine yet.

Where to use it:

• Apple Silicon Macs with 24 GB or 32 GB of memory, through mlx-vlm
• A local server that coding agents can call, such as pi or OpenCode
• The terminal, for one-off prompts

Use case | Who | What they get back
Code generation | Developers | Code in reply, with the thinking returned separately
Local agent backend | Agent users | A local endpoint for coding tools
Fit testing | Mac owners | Reports on memory and speed for the community

How to set it up:

• An Apple Silicon Mac with 24 GB or 32 GB of memory
• Python, with mlx-vlm 0.7.10 or later
• About 17.8 GB of disk space

1. Install mlx-vlm.
2. On a 24 GB Mac only, raise the GPU memory limit until the next restart.
3. Run the model with the sampling settings the card keeps.

pip install -U mlx-vlm

sudo sysctl iogpu.wired_limit_mb=20480

pip install -U huggingface_hub
hf download mlx-community/flint-mixed-3-4bit

How to use it: You run the generate command with the card's sampling settings, which follow Recursive's advice. For coding agents, start the local server and send the same settings with each request. The model thinks before it answers, often for 10 to 30 thousand tokens, so keep total context near 32K.

mlx_vlm.generate --model mlx-community/flint-mixed-3-4bit --temperature 1.0 --top-p 0.95 --top-k 20 \
  --max-tokens 32768 --prompt "Write a Python function that merges overlapping intervals."

mlx_vlm.server --model mlx-community/flint-mixed-3-4bit --port 8080

Option | What it does
temperature 1.0 | Sampling setting the card says to keep
top-p 0.95 | Sampling setting the card says to keep
top-k 20 | Sampling setting the card says to keep
max-tokens 32768 | Leaves room for the long thinking before the answer

Example: You ask for a Python function that merges overlapping intervals. The model thinks for a while, then returns the function. On a 24 GB Mac, close heavy apps such as browsers with many tabs first.

For newcomers and experts: This is a big coding model squeezed to fit a normal Mac. A good first try is the card's merge intervals prompt after you raise the GPU limit on a 24 GB machine.

If you know the field, the card started from Recursive's INT4 weights, which were trained at 4 bits. It then lowered just the routed experts to 3 bits, using for every 64-weight group whichever scale rounds with the least error. Attention, embeddings and the output layer keep their trained values. In the card's tests, 86 percent of next tokens match the INT4.

Install mlx-vlm
Raise the GPU memory limit on 24 GB
Run generate or start the server
Model thinks, then answers

Title: Perplexity on the card's test, lower is better
This build: 4.42
Recursive INT4: 4.62
6-bit from BF16: 4.86
3-bit experts and 6-bit rest: 5.19
3-bit experts and 4-bit rest: 8.44

Good to know:

• Fit: the card measured memory on a 64 GB Mac and has not tested a real 24 GB one yet.
• Tool calls: with mlx-vlm 0.7.10, argument names are sometimes wrong, so agents retry. One OpenCode test saw 41 of 85 calls fail, and every task still finished.
• Perplexity: a lower number than the INT4 does not mean it is better, the card warns.
• Coding tasks: it solved the same 6 of 6 tasks as Recursive's INT4.
• License: Apache 2.0.
• Cost to run: none named.
• Data it reaches: it downloads model files from Hugging Face, and a local server runs on your machine.
• Not for: Macs with less than 24 GB, or anyone who needs an official Recursive release.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: mlx
• Parameters: 39.8B
• Weights: about 159 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 5,091
• Likes: 12
• License: apache-2.0
• Based on: RecursiveSI/flint-int4
• Tags: safetensors, flint, code, coding-agent, apple-silicon, 24gb, 32gb, 3-bit, text-generation, conversational, 4-bit

Links

• Model: https://huggingface.co/mlx-community/flint-mixed-3-4bit
• Files: https://huggingface.co/mlx-community/flint-mixed-3-4bit/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download mlx-community/flint-mixed-3-4bit

Then follow the usage in its model card. Nothing in this kit runs by itself.