BERTimbau Base

BERTimbau Base

A BERT model pretrained on Brazilian Portuguese that fills in masked words and produces text embeddings. For NLP developers and researchers who work with Portuguese text.

Made by neuralmind on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: BERTimbau Base is a BERT model pretrained for Brazilian Portuguese, with 12 layers and 110M parameters. It guesses a word hidden behind a mask, and it gives you embeddings for Portuguese text. The authors report best-in-class scores on three downstream tasks, covering named entities, sentence similarity and entailment between texts. A larger version with 335M parameters also exists.

Who it suits: It suits people who build language tools for Portuguese and want a pretrained base to fine-tune or to read embeddings from.

Industry | Who uses it | How they use it | Example
Language tech | NLP engineers | Fine-tune it for entity recognition or similarity | Train a tagger on Portuguese documents
Search | Developers | Turn Portuguese sentences into vectors | Compare two sentences by their embeddings
Research | Linguists and ML researchers | Probe what the model predicts for a masked word | Fill a gap in a Portuguese sentence

If you need text in other languages, or a model that writes long passages, you can skip it.

Where to use it: You run it in Python with the transformers library, on Windows, macOS or Linux. The card links a project repository for further questions and requests.

• Python scripts and notebooks that use transformers and torch
• Windows, macOS or Linux with the huggingface_hub package
• Training scripts that fine-tune it for a Portuguese task

Use case | Who | What they get back
Masked word prediction | Researchers | A ranked list of words that could fill the gap
Text embeddings | Developers | A vector for each token of a sentence
Fine-tuning base | NLP engineers | A Portuguese starting point for a classifier or tagger

How to set it up: You need Python with the transformers and torch libraries. The card's examples download the model by name the first time you load it.

• Python on Windows, macOS or Linux
• The transformers and torch libraries
• The Hugging Face hub package, if you want the files on disk

1. Install the transformers and torch libraries.
2. Optionally download the model files.
3. Run the card's loading code.

pip install -U huggingface_hub
hf download neuralmind/bert-base-portuguese-cased

How to use it: Load the model and tokenizer, then either run a fill-mask pipeline or ask the plain model for embeddings. The tokenizer is told not to lowercase text, because this model is cased.

from transformers import AutoTokenizer  # Or BertTokenizer
from transformers import AutoModelForPreTraining  # Or BertForPreTraining for loading pretraining heads
from transformers import AutoModel  # or BertModel, for BERT without pretraining heads

model = AutoModelForPreTraining.from_pretrained('neuralmind/bert-base-portuguese-cased')
tokenizer = AutoTokenizer.from_pretrained('neuralmind/bert-base-portuguese-cased', do_lower_case=False)

from transformers import pipeline

pipe = pipeline('fill-mask', model=model, tokenizer=tokenizer)

pipe('Tinha uma [MASK] no meio do caminho.')

import torch

model = AutoModel.from_pretrained('neuralmind/bert-base-portuguese-cased')
input_ids = tokenizer.encode('Tinha uma pedra no meio do caminho.', return_tensors='pt')

with torch.no_grad():
    outs = model(input_ids)
    encoded = outs[0][0, 1:-1]  # Ignore [CLS] and [SEP] special tokens

# encoded.shape: (8, 768)

Tool | What it does
AutoModelForPreTraining | Loads the model with its pretraining heads
AutoTokenizer | Loads the cased tokenizer
pipeline fill-mask | Predicts the word behind a mask token
AutoModel | Loads the plain model, to read embeddings

Example: You pass the Portuguese sentence "Tinha uma [MASK] no meio do caminho." to the fill-mask pipeline. It returns a ranked list of candidate words for the gap, and the card's run puts pedra first, then arvore, estrada, casa and cruz.

For newcomers and experts: If you are new to this, fill-mask means the model guesses the word hidden under a placeholder. A first small step is to run the fill-mask snippet and change the sentence.

If you know the field, this is the BERT-Base architecture pretrained on Portuguese, and the embedding example returns a 768-wide vector for each of the 8 tokens in its sentence. It is a pretraining checkpoint, so you add your own head and fine-tune it for a downstream task.

Portuguese sentence with a mask
Cased tokenizer builds input ids
BERT-Base reads the whole sentence
Candidate words or embeddings come out

Good to know: Hugging Face lists many models built on this one, 253 fine-tuned and 14 quantized versions. The card also gives a citation entry if you use the work.

Title: Parameters by BERTimbau size, in millions
BERTimbau Base: 110 million
BERTimbau Large: 335 million

• License: MIT
• Cost to run: no API key or paid service is named
• Data it reaches: the download comes from Hugging Face, and the model works on the text you give it
• Not for: languages other than Portuguese, or writing long passages

On Hugging Face, 13 November 2026

• Task: fill mask
• Library: transformers
• Downloads in the last 30 days: 367,424
• Likes: 248
• License: mit
• Languages: pt
• Quantized versions: 14
• Fine-tunes: 256
• Tags: transformers, pytorch, bert, fill-mask

Links

• Model: https://huggingface.co/neuralmind/bert-base-portuguese-cased
• Files: https://huggingface.co/neuralmind/bert-base-portuguese-cased/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download neuralmind/bert-base-portuguese-cased

Then follow the usage in its model card. Nothing in this kit runs by itself.