MTEB-BR Leaderboard

MTEB-BR Leaderboard

Ranks text embedding models on native Brazilian Portuguese tasks, with 93 models scored on 22 tasks. A static leaderboard page for researchers and engineers choosing a model for Portuguese search.

Made by MTEB-BR on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: This is a static web page that ranks text embedding models on Brazilian Portuguese. It holds results for 93 models, 73 with open weights and 20 closed commercial APIs, scored on 22 tasks in 7 categories. Only data written or found in Portuguese is admitted, so machine-translated test sets are left out.

Who it suits: Researchers who compare embedding models, and engineers who need to pick one for search over Portuguese text.

Industry | Who uses it | How they use it | Example
Machine learning | NLP researchers | Compare embedding models on native Portuguese tasks | Check which model leads on the headline score
Software | Search and RAG engineers | Choose an embedding model for Portuguese documents | Shortlist open-weight models before testing
Domain search | Teams with Portuguese documents | Look at tasks that match their field, such as MedPT or BRTaxQA | Read the retrieval results for their domain
Academia | Model authors | Propose a new model for evaluation through the project's issue tracker | Send a model ID with its evaluation files

Skip it if you work only in English or need results on translated test sets.

Where to use it:

• A web browser, on the Space's page at Hugging Face
• A local folder, after downloading the files and opening index.html in a browser
• The project's GitHub issues, for proposing a new model

Use case | Who | What they get back
Choose an embedding model | ML engineers | A ranked view of 93 models on Portuguese tasks
Check one model's strengths | Researchers | Scores across task categories such as retrieval or clustering
Propose a model | Model authors | A route to get a model evaluated

How to set it up:

• Nothing at all for the online page
• A web browser
• Python and pip, only if you want to download a copy with the Hugging Face tools

1. Open the Space's page at Hugging Face to use it online.
2. For a copy, install the Hugging Face hub package.
3. Download the Space into a folder called leaderboard.
4. Move into that folder.
5. Open index.html in a web browser.

pip install -U huggingface_hub
hf download MTEB-BR/leaderboard --repo-type space --local-dir leaderboard
cd leaderboard

The page is plain HTML with a style sheet and image assets, so nothing has to be built or started.

How to use it: Open the page and read the scores. The headline figure is mean_22, the average over all 22 tasks. It is reported alongside bootstrap intervals, significance tests between models, and a measure of how well each task tells models apart. To add a model, open an issue in the project's GitHub repository with the model ID, its evaluation JSON files and a script that reproduces the run.

Example: You are building search over Portuguese documents and need an embedding model. You open the leaderboard, look at the headline mean_22 score and the retrieval tasks, and shortlist a few open-weight models to test on your own data.

For newcomers and experts: An embedding model turns text into numbers so that similar texts land close together, which is what search tools rely on. If you are new, open the page and start with the headline score to see how the models order.

If you know the field, the design choice is native data only. Machine-translated sets such as mMARCO and mkqa are excluded by construction. Models are compared with bootstrap intervals and paired significance tests, and IRT discrimination shows which tasks separate them.

Models run on 22 native tasks
Each task gets a score
mean_22 averages the 22 scores
Intervals and tests add context
The leaderboard lists the models

Good to know:

• License: apache-2
• Cost to run: none named
• Data it reaches: it names a results dataset on Hugging Face called MTEB-BR/mteb-pt-results
• Not for: translated or non-Portuguese benchmarks, since only native Brazilian Portuguese data is admitted

Title: Models evaluated by type
Open-weight models: 73 models
Closed commercial APIs: 20 models

On Hugging Face, 13 November 2026

• Likes: 126
• Built with: static
• License: apache-2.0

Links

• Space: https://huggingface.co/spaces/MTEB-BR/leaderboard
• Files: https://huggingface.co/spaces/MTEB-BR/leaderboard/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.