H2O-Lightning-4B

H2O-Lightning-4B

Answers typed decision questions about a document, ticket, policy or conversation, and about its images, giving each option a probability. For teams that want a small model to judge records.

Made by h2oai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: H2O-Lightning-4B is a decision model from H2O.ai, built on Qwen3.5-4B. You give it a record such as a document, ticket, policy or conversation, plus any photos, screenshots, scans or charts that go with it, and ask a typed question. It gives each option a probability.

Who it suits: It suits teams that sort or judge records and want a number behind each call, not a paragraph of prose.

Industry | Who uses it | How they use it | Example
Customer support | Support teams | Pick a category for a ticket that comes with a screenshot | Choosing billing or technical for a ticket
Compliance | Policy reviewers | Check whether a statement about a document holds | Is this clause present, yes or no
Operations | Process owners | Place a record on a stated scale with a probability spread | Rating an item from one to five
Document handling | Back-office staff | Read scanned pages and charts alongside the text | Sorting scanned forms into named types

Skip it if you want open-ended chat or long writing, since the card describes typed decision answers only.

Where to use it:

• A vLLM server, either H2O's own build of vLLM 0.30.0 or stock vLLM 0.30.0
• The transformers library, which the model is listed for
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Choice question | Triage teams | The probability of each named option
Yes or no question | Reviewers | The probability that a statement is true
Score question | Quality teams | A level on your scale with its probability spread

How to set it up: You need these first.

• Python with pip, to download the files
• Room for the weights: about 9.3 GB in BF16, or about 2.3 GB at 4-bit by estimate
• vLLM 0.30.0, ideally H2O's per-request attention build with its shim file in front

1. Install the Hugging Face hub library with pip.
2. Download the model files with the hf command.
3. Start the server with the serve script from the model's folder.
4. If you cannot install H2O's vLLM build, switch the script to stock vLLM, as shown below.

pip install -U huggingface_hub
hf download h2oai/h2o-lightning-4b

How to use it: Start the server, then send it a record, any images and a typed question. The default start reads each request two-way, which the card recommends. The fallback reads one-way and reuses the prefix cache, so it handles more requests each second when busy.

bash h2o-lightning-4b/serve.sh

TWO_WAY=0 bash h2o-lightning-4b/serve.sh

Tool | What it does
choice | Selects exactly one answer from options you name
yes/no (noul) | Returns how likely it is that your statement is true
score | Returns a level on your scale, along with the probability of each level
serve.sh | Starts the model server, two-way reading by default

Example: Send a support ticket with a screenshot attached and ask a yes or no question, such as whether the customer reports a billing error. The model replies with the probability that the statement is true, so you can set your own cut-off.

For newcomers and experts: If you are new, think of it as a classifier you can talk to. Hand it a record and a question that has fixed answers, and it tells you how sure it is of each. A first small try is one yes or no question about a short ticket.

If you know the field, the card reports that two-way reading, with the shim in front of H2O's vLLM build, scores 206 of 231 public JevBench items. Stock one-way reading scores 202 and runs faster under load. The card also lists first place among 46 open-weight models on the ImageJevBench composite and second of 170 on the JevBench board. Those rankings come from the authors, not from independent checks.

Record plus images arrive
You add a typed question
Server reads the request two-way
Model scores each option
You get a probability per option

Good to know:

• Size: 4.7B parameters in Safetensors
• Other builds: other people have made 6 quantized and 9 fine-tuned versions
• Two ways to serve: the default is recommended and the fallback trades some score for throughput

Title: JevBench public items scored out of 231
Two-way reading: 206 items
One-way fallback: 202 items

• License: Apache 2.0
• Cost to run: none named, the source lists no API key, account or paid service
• Data it reaches: the records and images you send it, handled by your own server, and the source names no web or account access
• Not for: open-ended chat, since the card covers only typed decision questions

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 4.7B
• Weights: about 9.3 GB in BF16; about 2.3 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 10,530
• Likes: 107
• License: apache-2.0
• Based on: Qwen/Qwen3.5-4B
• Quantized versions: 6
• Fine-tunes: 9
• Tags: transformers, safetensors, qwen3_5, image-text-to-text, decision-model, decision-making, calibration, jevbench, multimodal, typed-decisions, conversational

Links

• Model: https://huggingface.co/h2oai/h2o-lightning-4b
• Files: https://huggingface.co/h2oai/h2o-lightning-4b/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download h2oai/h2o-lightning-4b

Then follow the usage in its model card. Nothing in this kit runs by itself.