JevK5-chat-mlx-4bit

JevK5-chat-mlx-4bit

A 4-bit MLX copy of the 4.2B JevK5 chat model for Apple Silicon, with a 262,144 token context and a small memory footprint. Made for Mac users who want a quick local chat model on a base machine.

Made by mlx-community on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a 4-bit MLX conversion of alibiserikbay's JevK5, a chat model with about 4.2B parameters and a Qwen3_5ForCausalLM architecture. It averages 4.50 bits per weight and supports a 262,144 token context. The card gives its active memory use as about 2.7 GB, with 8 GB of unified memory as the recommended minimum.

Who it suits: It suits people with an Apple Silicon Mac, even a base 8 GB one, who want a small local chat model that can run next to other apps.

Industry | Who uses it | How they use it | Example
Software | Developers | Chat with a local model beside an editor and browser | Asking a quick coding question
Education | Students | Try a local language model on a laptop | Asking for a simple explanation
Writing | Hobbyists | Draft short text offline | Writing a short poem
Research | ML hobbyists | Compare 4-bit, 8-bit and 16-bit copies | Reading the variant table

Skip it if you have a 36 GB or larger Mac and want full precision, since the card lists 8-bit and 16-bit copies for that.

Where to use it:

• Apple Silicon Macs, through the mlx-lm package
• The terminal, in chat or one-shot mode
• Python scripts, and LM Studio with custom stop tokens

Use case | Who | What they get back
Interactive chat | Mac users | Replies in a terminal conversation
One-shot generation | Developers | Text from a single prompt
Chat in Python | Developers | A reply built with the chat template

How to set it up:

• An Apple Silicon Mac, with 8 GB or more of unified memory
• Python, with the mlx-lm package
• About 2.7 GB of disk space

1. Install mlx-lm.
2. Run the chat or generate command with the model name.
3. If you use a tool such as LM Studio, add the stop tokens the card lists.

pip install mlx-lm

pip install -U huggingface_hub
hf download mlx-community/JevK5-chat-mlx-4bit

How to use it: You run the chat command for a conversation, or the generate command with a prompt. The card's examples name the converter's own repository, SirSahOl, so use the repository name you downloaded. In Python, you build the prompt with the chat template.

# Chat interactively
mlx_lm.chat --model SirSahOl/JevK5-chat-mlx-4bit

# Generate text
mlx_lm.generate --model SirSahOl/JevK5-chat-mlx-4bit --prompt "Write a short poem about artificial intelligence."

from mlx_lm import load, generate

model, tokenizer = load("SirSahOl/JevK5-chat-mlx-4bit")

messages = [
    {"role": "user", "content": "Explain quantum superposition in simple terms."}
]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

response = generate(model, tokenizer, prompt=prompt, verbose=True)
print(response)

Tool | What it does
mlx_lm.chat | Starts an interactive chat in the terminal
mlx_lm.generate | Produces text from one prompt
apply_chat_template | Builds the prompt from a list of messages

Example: You run the generate command with a prompt that asks for a short poem about artificial intelligence. The model prints the poem. In Python, you can ask for a simple explanation of quantum superposition and print the reply.

For newcomers and experts: This is a small chat model you can run on almost any recent Mac. A good first try is the interactive chat command.

If you know the field, the card lists stop tokens for local runtimes: im_start, im_end and endoftext, written with the pipe markers, to prevent runaway loops. It also gives the ChatML style prefixes and suffixes for system, user and assistant turns. The card's M1 test shows 17.97 tokens per second at 4-bit.

Install mlx-lm
Load the 4-bit model
Build the prompt with the chat template
Generate and print the reply

Title: Disk size by variant, in GB
4-bit MLX: 2.7 GB
8-bit MLX: 4.9 GB
16-bit MLX: 9.2 GB

Good to know:

• Speed: on an Apple M1 with 8 GB, the card measured 17.97 tokens per second at 4-bit and 10.35 at 8-bit.
• Variants: the card also lists 8-bit and 16-bit copies for Macs with more memory.
• Stop tokens: add im_start, im_end and endoftext in your runtime to keep turns in order.
• License: Apache 2.0.
• Cost to run: none named.
• Data it reaches: it downloads model files from Hugging Face and works on the text you give it.
• Not for: Macs without Apple Silicon, since this is an MLX format copy.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: mlx
• Parameters: 4.2B
• Weights: about 16.8 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 381
• Likes: 2
• License: apache-2.0
• Based on: alibiserikbay/JevK5
• Tags: safetensors, qwen3_5_text, text-generation, apple-silicon, 4-bit, conversational, transformers, decision-model, system-one, jev-alternative, typed-decisions, self-hosted

Links

• Model: https://huggingface.co/mlx-community/JevK5-chat-mlx-4bit
• Files: https://huggingface.co/mlx-community/JevK5-chat-mlx-4bit/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download mlx-community/JevK5-chat-mlx-4bit

Then follow the usage in its model card. Nothing in this kit runs by itself.