Phi-4-multimodal-instruct

Phi-4-multimodal-instruct

Compact 5.6B model that takes text, images and audio and replies in text, with a 128K token context. For developers who want speech, vision and text in one small model.

Made by microsoft on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Phi-4-multimodal-instruct is Microsoft's open model that takes text, images and audio as input and answers in text. It has 5.6 billion parameters and a 128K token context. Its training added supervised tuning plus preference and human-feedback stages, aimed at following instructions and staying safe.

Who it suits: It suits developers who want one compact model for text, pictures and speech, especially where memory or response time is tight.

Industry | Who uses it | How they use it | Example
Software | App developers | Add image and speech understanding to an app | Reading text from a photo of a receipt
Customer service | Support teams | Transcribe and summarize calls | Turning a recorded call into a summary
Education | Learning app builders | Explain charts, tables and math shown in images | Walking through an equation from a screenshot
Research | Multimodal researchers | Use it as a building block for experiments | Comparing it with other small models

You can skip it if you only need plain text generation, since a text-only model would be lighter.

Where to use it: It runs in Python through the transformers library on Windows, macOS or Linux, and Microsoft offers hosted playgrounds to try it first.

• Python scripts that use transformers
• Playgrounds on Azure, GitHub, Nvidia and Hugging Face
• A separate ONNX version for other runtimes

Use case | Who | What they get back
Speech recognition | Developers | Text transcripts of English, Chinese and other audio
Speech translation and QA | App builders | Translated or answered text from spoken input
Image and chart understanding | Analysts | Descriptions, OCR text and answers about tables
Tool calling | Agent builders | Function calls chosen from a request

How to set it up: You need Python and the Hugging Face tool. Hugging Face lists the weights at about 11.1 GB in BF16, or an estimated 2.8 GB at 4-bit. The card's full usage section covers the rest.

• Python with transformers
• About 11.1 GB of disk for the BF16 weights
• A GPU for comfortable speed

1. Install the Hugging Face download tool.
2. Download the model files, or let transformers fetch them.
3. Follow the model card to load the model and its processor.

pip install -U huggingface_hub
hf download microsoft/Phi-4-multimodal-instruct

How to use it: Give the model text, an image, audio or a mix, and read its text reply. Language support differs by input type. The visible part of the card gives no run command, so follow its own usage section.

Tool | What it does
hf download | Fetches the model files to your computer
transformers | Loads the model for use in Python

Example: You give the model a recording of someone planning a trip and ask for a short plan. It listens to the audio and writes recommendations in text, which is one of the demos Microsoft shows.

For newcomers and experts: If you are new, try a playground first to see text, image and audio in one chat. Then download the files and follow the card to run it yourself.

If you know the field, this is a lightweight multimodal model built on the research behind Phi-3.5 and Phi-4. Text covers about two dozen languages, vision is English, and audio covers English, Chinese, German, French, Italian, Japanese, Spanish and Portuguese.

Text, image or audio input
Multimodal Phi-4 model
Reasoning over the combined input
Text output

Good to know: The card lists broad multilingual and multimodal use, both commercial and research, with attention to compute-limited and latency-bound settings.

• License: MIT
• Cost to run: no API key, account or paid service is named for local use, though the hosted playgrounds are separate services
• Data it reaches: the text, images and audio you provide, plus files downloaded from Hugging Face
• Not for: languages outside its lists, such as vision in languages other than English

On Hugging Face, 13 November 2026

• Task: automatic speech recognition
• Library: transformers
• Parameters: 5.6B
• Weights: about 11.1 GB in BF16; about 2.8 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 68,370
• Likes: 1,629
• License: mit
• Languages: ar, zh, cs, da, nl, en, fi, fr
• Quantized versions: 12
• Fine-tunes: 56
• Tags: transformers, safetensors, phi4mm, text-generation, code, audio, automatic-speech-recognition, speech-summarization, speech-translation, visual-question-answering, phi-4-multimodal, phi-4-mini

Links

• Model: https://huggingface.co/microsoft/Phi-4-multimodal-instruct
• Files: https://huggingface.co/microsoft/Phi-4-multimodal-instruct/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download microsoft/Phi-4-multimodal-instruct

Then follow the usage in its model card. Nothing in this kit runs by itself.