Gemma 4 12B IT (LiteRT-LM)

Gemma 4 12B IT (LiteRT-LM)

Runs Gemma 4 12B on your own desktop with text, image and audio input, no internet needed. A LiteRT-LM build for developers who want private, on-device generation on Windows, macOS or Linux.

Made by litert-community on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Gemma 4 12B Instruct packaged in the LiteRT-LM format, ready to run on macOS, Linux and Windows. The model file takes text, images and audio, and it supports multi-token prediction for faster speculative decoding. Because it runs on your device, you can use it without an internet connection.

Who it suits: Developers and desktop-app builders who want a mid-sized model that runs locally and keeps data on the machine. It is meant for desktop use, with the web as a more limited option.

Industry | Who uses it | How they use it | Example
Software | Desktop app developers | Embed a local assistant in an app through LiteRT-LM | A writing helper that works offline
Privacy-focused work | Teams with sensitive text | Keep prompts and answers on their own machines | Summarizing internal notes without a cloud call
Product design | Prototype builders | Try text, image and audio input in one model | A demo that answers questions about a photo
Research | Edge AI researchers | Measure speed and memory on local hardware | Comparing decode speed across devices

You can skip it if your machine has little memory, or if you want a hosted model, since this one runs on your own hardware.

Where to use it:

• Windows, macOS and Linux desktops
• A web demo, in a more limited way
• Apps built on LiteRT-LM, the runtime this model targets

Use case | Who | What they get back
Offline chat assistant | Desktop developers | Text replies generated on the device
Image and audio questions | Prototype builders | Answers that use a picture or a sound clip
Fast local replies | App builders | Lower delay through multi-token prediction

How to set it up:

• Python, to install the Hugging Face client
• LiteRT-LM version 0.17 or later, which the card requires
• A desktop with enough memory, the benchmark GPU used about 7790 MB

1. Install the Hugging Face client.
2. Download the model files to your computer.
3. Run the model file with LiteRT-LM 0.17 or later.

pip install -U huggingface_hub
hf download litert-community/gemma-4-12B-it-litert-lm

How to use it: Run the model file through LiteRT-LM, using its desktop command-line tool or your own app. The source shows no command, so follow the LiteRT-LM guide. If memory runs short, set the max_num_tokens setting lower to shrink the context.

Tool | What it does
LiteRT-LM | Runs the model file and adds KV-cache handling, prompt templates and function calling
LiteRT | Provides the hardware acceleration underneath
max_num_tokens | Sets the context length, lower it when memory is tight

Example: You are on a plane with no network and ask the model to summarize a page of notes. It reads the text on your machine and returns a summary. On the benchmark GPU, the first token arrived in 0.3 seconds.

For newcomers and experts: If you are new, try the desktop command-line tool from the LiteRT-LM guide with a short text question before you add images or audio. Start with the default context length.

If you know the field: the benchmarks used 1024 prefill tokens, 256 decode tokens and a 4096-token context. The model can reach 128k context on devices with enough memory, and multi-token prediction cuts latency.

Your prompt
LiteRT-LM
XNNPack on CPU or ML Drift on GPU
Reply on device

Good to know: The card gives a Linux benchmark on an NVIDIA 4090 with 24 GB. Prefill ran at 3548 tokens per second and decode at 69 tokens per second.

Title: Memory on an NVIDIA 4090, in MB
Model file: 6883 MB
GPU memory: 7790 MB

• Inputs: text, images and audio
• Context: 4096 tokens in the benchmark, up to 128k with enough memory
• License: Apache 2.0
• Cost to run: none named
• Data it reaches: model files downloaded to your computer, nothing else named
• Not for: the web at full capability, since the card says web support is more limited

On Hugging Face, 13 November 2026

• Library: litert-lm
• Downloads in the last 30 days: 13,083
• Likes: 71
• License: apache-2.0
• Based on: google/gemma-4-12B-it
• Tags: litert-lm

Links

• Model: https://huggingface.co/litert-community/gemma-4-12B-it-litert-lm
• Files: https://huggingface.co/litert-community/gemma-4-12B-it-litert-lm/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download litert-community/gemma-4-12B-it-litert-lm

Then follow the usage in its model card. Nothing in this kit runs by itself.