nomic-embed-text-v1.5-GGUF

nomic-embed-text-v1.5-GGUF

Turns sentences into vectors on your own machine through llama.cpp, using compact GGUF files, for search and RAG questions. For developers who want local text embeddings.

Made by nomic-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: nomic-embed-text-v1.5-GGUF holds llama.cpp-compatible GGUF files of the nomic-embed-text-v1.5 embedding model. An embedding model turns text into a list of numbers, so you can compare sentences by meaning for search or retrieval. The repository offers many quantizations, from 48 MiB up to the full 262 MiB.

Who it suits: It suits developers who want text embeddings from llama.cpp on their own hardware, for example to embed user questions in a RAG app.

Industry | Who uses it | How they use it | Example
Software | Developers | Embed user questions in a RAG app | Embed a question before searching your documents
Search | Search engineers | Compare sentences by meaning | Embed a batch of sentences from a text file
Edge devices | Hobbyists | Run embeddings from a small file on a modest machine | Use the Q4_K_M file at 81 MiB
Research | NLP researchers | Compare quantizations with the original | Read the card's error table

If you work in Python with sentence-transformers, use the original model instead and skip these files.

Where to use it: You run it with the embedding example that comes with llama.cpp, from a terminal on Windows, macOS or Linux. The card names a llama.cpp commit from 2/15/2024 as compatible.

• llama.cpp, with its embedding example program
• A terminal on Windows, macOS or Linux
• A RAG or search app that calls llama.cpp for its vectors

Use case | Who | What they get back
Question embedding | RAG builders | A vector for each user question
Batch embedding | Search engineers | Vectors for many texts from one file
Small-footprint runs | Hobbyists | A compact quantized model that runs locally

How to set it up: You need llama.cpp at a version the card says works, Python for the download, and one of the GGUF files. The repository holds every quantization, so choose the one you want.

• llama.cpp with the embedding example
• Python and the Hugging Face hub package, for the download
• One GGUF file, such as the F16 one

1. Install the Hugging Face hub package.
2. Download the repository files.
3. Build or install llama.cpp with its embedding example.
4. Run the embedding command on your text.

pip install -U huggingface_hub
hf download nomic-ai/nomic-embed-text-v1.5-GGUF

How to use it: Start each text with a task prefix, such as search_query, because the card says the model needs one. This command computes a single embedding.

./embedding -ngl 99 -m nomic-embed-text-v1.5.f16.gguf -c 8192 -b 8192 --rope-scaling yarn --rope-freq-scale .75 -p 'search_query: What is TSNE?'

For several texts, put one per line in a file and pass it with the file flag. The total tokens must stay within the context length.

search_query: What is TSNE?
search_query: Who is Laurens Van der Maaten?

./embedding -ngl 99 -m nomic-embed-text-v1.5.f16.gguf -c 8192 -b 8192 --rope-scaling yarn --rope-freq-scale .75 -f texts.txt

Tool | What it does
-m | Names the GGUF model file to load
-c 8192 | Asks for the full 8192 token context
--rope-scaling yarn | The context extension method the example uses
-p | The text to embed, with its task prefix
-f | A file with several texts to embed

Example: You run the single command with the question about TSNE. It prints the embedding for that question. With a batch file, the example program shows only the first three embeddings.

For newcomers and experts: If you are new to this, an embedding is a list of numbers that stands for the meaning of a text, and GGUF is the file format llama.cpp reads. A first small step is to run the single-embedding command once.

If you know the field, llama.cpp opens these files with a 2048-token window unless you extend it. The full 8192 needs a context extension method such as the yarn one in the command, because the Dynamic NTK-Aware RoPE scaling of the Transformers model is not available there.

Text with a task prefix goes in
llama.cpp loads the GGUF file
The embedding program computes the vector
The embedding is printed

Good to know: The card measures how far each quantization drifts from the Sentence Transformers version, as mean squared error. Smaller files drift more. F32 is the closest, at 6.08e-11.

Title: Mean squared error against Sentence Transformers
Quant | Size | MSE
Q2_K | 48 MiB | 2.33e-03
Q3_K_M | 65 MiB | 8.26e-04
Q4_0 | 75 MiB | 6.32e-04
Q4_K_M | 81 MiB | 2.42e-04
Q5_K_M | 95 MiB | 6.55e-05
Q6_K | 108 MiB | 5.58e-05
Q8_0 | 140 MiB | 5.79e-06
F16 | 262 MiB | 4.21e-10

Title: File size in MiB
Q2_K: 48 MiB
Q3_K_M: 65 MiB
Q4_K_M: 81 MiB
Q5_K_M: 95 MiB
Q6_K: 108 MiB
Q8_0: 140 MiB
F16: 262 MiB
F32: 262 MiB

• License: Apache-2.0
• Limits: 2048 tokens of context by default, and the card lists English as its language
• Cost to run: no API key or paid service is named
• Data it reaches: the download comes from Hugging Face, and the model works on the text you give it on your own machine
• Not for: texts sent without a task prefix, or llama.cpp builds far newer than the card's compatibility date

On Hugging Face, 13 November 2026

• Task: sentence similarity
• Parameters: 137M
• Weights: about 68 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 522,653
• Likes: 142
• License: apache-2.0
• Languages: en
• Based on: nomic-ai/nomic-embed-text-v1.5
• Quantized versions: 1
• Tags: gguf, feature-extraction, sentence-similarity

Links

• Model: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF
• Files: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download nomic-ai/nomic-embed-text-v1.5-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.