Qwen2-Audio-7B-GGUF

Qwen2-Audio-7B-GGUF

Runs Qwen2-Audio locally to chat by voice and analyze audio, including speech translation, transcription and sound detection. GGUF copies for Nexa-SDK, for people who want audio AI on their own device.

Made by NexaAI on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF copies of Qwen2-Audio, an audio-and-text model, packaged by NexaAI so it runs locally with Nexa-SDK. It takes audio and text as input, so you can talk to it without a separate speech recognition step. The card says it supports English, Chinese and major European languages. The model has 7.8B parameters.

Who it suits: It suits people who want to talk to an AI or have it analyze audio on their own computer instead of sending files to a service.

Industry | Who uses it | How they use it | Example
Productivity | Everyday users | Ask spoken questions and get answers | Ask a daily question by voice and read the reply
Media | Editors and producers | Transcribe or summarize recordings | Drop in an audio file and ask for a summary
Language | Translators | Translate speech | Play a clip and ask for a translation
Sound work | Audio hobbyists | Analyze music or background noise | Ask what is in a mixed recording

If you only need text answers, you can skip it, because it is built around audio input.

Where to use it: You use it through Nexa-SDK, in a terminal or in a local Streamlit interface, on a Windows, macOS or Linux computer.

• A terminal, with Nexa-SDK installed
• A local Streamlit interface, started with a flag
• Edge devices and laptops that can spare the memory

Use case | Who | What they get back
Voice chat | Everyday users | Answers and suggestions to spoken questions
Audio analysis | Analysts | Summaries, extracted information and transcripts
Sound detection | Hobbyists | A description of background noise, music and mixed audio

How to set it up: You need Nexa-SDK, an on-device inference framework that you can install as a Python package or an executable installer. The default q4_K_M version needs 4.2 GB of RAM.

• Nexa-SDK, installed from the project's install options
• About 4.2 GB of RAM for the default version
• Windows, macOS or Linux

1. Install Nexa-SDK.
2. Run the command in your terminal.
3. Add your audio file and an optional text prompt.

nexa run qwen2audio

How to use it: In the terminal, drag your audio file into the window, or type the file path on Linux, then add a text prompt to steer the analysis. Leave the prompt empty to send direct voice input. To use a local interface instead, add the flag shown below.

nexa run qwen2audio -st

Tool | What it does
nexa run qwen2audio | Runs the model in the terminal with the default q4_K_M version
nexa run qwen2audio -st | Opens a local Streamlit interface instead
audio file | Dragged in, or entered as a path on Linux
text prompt | Guides the analysis, or can be left empty

Example: You drag a recording into the terminal and type "Summarize this audio." The model listens to the clip and answers in text with a summary. Without a prompt, it treats the clip as a voice message and replies to it.

For newcomers and experts: If you are new to this, think of it as an assistant that can hear. A first small step is to install Nexa-SDK, run the default command and speak a short question through an audio file.

If you know the field, the model takes audio and text together and answers without an ASR step in front of it. Nexa-SDK serves GGUF quantizations, and the card says the default q4_K_M version is the one that runs from the plain command. Other quantizations are listed with their RAM needs on the project's page.

Audio file plus optional text prompt
Nexa-SDK loads the GGUF model
Model reads audio and text together
Text answer comes back

Good to know: The card says Qwen2-Audio beats earlier top models and its own predecessor across the tasks it tested, and it shows results in charts that are not reproduced here. To get only the raw GGUF files, Hugging Face lists these commands.

pip install -U huggingface_hub
hf download NexaAI/Qwen2-Audio-7B-GGUF

Title: Size in GB
4-bit weights, estimate: 3.9 GB
RAM for the default version: 4.2 GB

• License: Apache-2.0
• Cost to run: no API key or paid service is named, and the default version needs 4.2 GB of RAM
• Data it reaches: the download comes from Hugging Face, and the model works on the audio and text you give it on your own device
• Not for: work without audio, or devices with less than 4.2 GB of free RAM

On Hugging Face, 13 November 2026

• Task: audio text to text
• Parameters: 7.8B
• Weights: about 3.9 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 2,522
• Likes: 177
• License: apache-2.0
• Languages: en
• Tags: gguf, audio-text-to-text, chat, audio, conversational

Links

• Model: https://huggingface.co/NexaAI/Qwen2-Audio-7B-GGUF
• Files: https://huggingface.co/NexaAI/Qwen2-Audio-7B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download NexaAI/Qwen2-Audio-7B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.