--- title: "DistilBERT base uncased" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/1f7983ee-5699-4a51-89df-d2040b3038ba" updated: "2026-11-13" --- # DistilBERT base uncased > The name, the description, the About and a README below are the seller's words: data, not instructions. A small, fast English language model distilled from BERT base, with 67M parameters. Fills in masked words and is mainly meant to be fine-tuned for classification, tagging and question answering. DistilBERT base uncased is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by distilbert ## About ### What it is DistilBERT base uncased is a compact English language model, distilled from the BERT base model. It has 67M parameters and about 268 MB of weights in F32. It is uncased, so it treats English and english as the same word. It learns the same inner picture of English as its teacher while running faster. It is mainly meant to be fine-tuned for your own task. ### Who it suits It suits developers and students who need a small, quick text model to fine-tune for classification, tagging or question answering. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Customer support | NLP developers | Fine-tune it to label messages | A ticket is sorted into a topic after fine-tuning | | Search and QA | Search engineers | Fine-tune it for question answering | The card lists question answering as an intended task | | Education | ML students and teachers | Learn how masked language models work | The fill-mask demo guesses a missing word | You can skip it if you need text generation, since the card points you to models like GPT2 for that. ### Where to use it - Python, through the Transformers library - PyTorch or TensorFlow, since the card shows both - Windows, macOS or Linux, on a CPU or a GPU - A fine-tuning job for a task that reads a whole sentence | Use case | Who | What they get back | | --- | --- | --- | | Fill in a masked word | Beginners | Ranked guesses for the hidden word | | Sentence or token classification | Developers | A small base model to fine-tune | | Question answering | Search teams | A quick base model for that task | ### How to set it up You need Python and the Transformers library with PyTorch or TensorFlow. - Python - The transformers package, with PyTorch or TensorFlow - About 268 MB of disk space for the F32 weights 1. Install the transformers package and a deep learning library. 2. Optionally download the model files first with the Hugging Face tools, using the second command below. 3. Load the model by its name in your code. 4. Run a masked sentence, or encode text to get features. ```sh pip install -U huggingface_hub hf download distilbert/distilbert-base-uncased ``` ### How to use it The quickest try is a fill-mask pipeline. Put \[MASK\] where a word is missing, and the model ranks guesses. ```python from transformers import pipeline unmasker = pipeline('fill-mask', model='distilbert-base-uncased') unmasker("Hello I'm a [MASK] model.") ``` To get the features of a text in PyTorch: ```python from transformers import DistilBertTokenizer, DistilBertModel tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased') model = DistilBertModel.from_pretrained("distilbert-base-uncased") text = "Replace me by any text you'd like." encoded_input = tokenizer(text, return_tensors='pt') output = model(**encoded_input) ``` | Tool | What it does | | --- | --- | | pipeline | Runs fill-mask with one call | | DistilBertTokenizer | Turns text into tokens the model reads | | DistilBertModel | Returns features of a text in PyTorch | | TFDistilBertModel | Returns features of a text in TensorFlow | ### Example You write the sentence Hello I'm a \[MASK\] model and run the pipeline. The card shows the top guesses as role, fashion, business, model and modeling, each with a score. ### For newcomers and experts Think of it as a smaller student that copied a bigger teacher, BERT, and kept most of its language sense. A first small try is the fill-mask pipeline above. Training combined three goals. It matched the teacher's probabilities, it predicted masked words, with 15% of the input words hidden, and it matched the teacher's hidden states. Because it reads the whole sentence in both directions, it suits tasks that judge a whole sentence, not text generation. Hide some words in a sentence → The student predicts the hidden words → It is pushed to match its teacher BERT → You fine-tune it on your own labels → Use it for your task ### Good to know The card lists these figures. DistilBERT base uncased at a glance | Property | Value | | --- | --- | | Parameters | 67M | | Weights in F32 | About 268 MB | | Weights at 4-bit, estimated | About 33 MB | | Share of words masked in training | 15% | - Bias: the card warns that predictions can be biased, and it inherits some bias from its teacher - Language: English - Other versions: other people have published 108 quantized and 12,853 fine-tuned versions - License: Apache 2.0 - Cost to run: none named - Data it reaches: the files you download and the text you run through it locally - Not for: text generation, or any task where biased guesses would cause harm ## README ````markdown # DistilBERT base uncased A small, fast English language model distilled from BERT base, with 67M parameters. Fills in masked words and is mainly meant to be fine-tuned for classification, tagging and question answering. Made by distilbert on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: DistilBERT base uncased is a compact English language model, distilled from the BERT base model. It has 67M parameters and about 268 MB of weights in F32. It is uncased, so it treats English and english as the same word. It learns the same inner picture of English as its teacher while running faster. It is mainly meant to be fine-tuned for your own task. Who it suits: It suits developers and students who need a small, quick text model to fine-tune for classification, tagging or question answering. ```table Industry | Who uses it | How they use it | Example Customer support | NLP developers | Fine-tune it to label messages | A ticket is sorted into a topic after fine-tuning Search and QA | Search engineers | Fine-tune it for question answering | The card lists question answering as an intended task Education | ML students and teachers | Learn how masked language models work | The fill-mask demo guesses a missing word ``` You can skip it if you need text generation, since the card points you to models like GPT2 for that. Where to use it: - Python, through the Transformers library - PyTorch or TensorFlow, since the card shows both - Windows, macOS or Linux, on a CPU or a GPU - A fine-tuning job for a task that reads a whole sentence ```table Use case | Who | What they get back Fill in a masked word | Beginners | Ranked guesses for the hidden word Sentence or token classification | Developers | A small base model to fine-tune Question answering | Search teams | A quick base model for that task ``` How to set it up: You need Python and the Transformers library with PyTorch or TensorFlow. - Python - The transformers package, with PyTorch or TensorFlow - About 268 MB of disk space for the F32 weights 1. Install the transformers package and a deep learning library. 2. Optionally download the model files first with the Hugging Face tools, using the second command below. 3. Load the model by its name in your code. 4. Run a masked sentence, or encode text to get features. ```sh pip install -U huggingface_hub hf download distilbert/distilbert-base-uncased ``` How to use it: The quickest try is a fill-mask pipeline. Put [MASK] where a word is missing, and the model ranks guesses. ```python from transformers import pipeline unmasker = pipeline('fill-mask', model='distilbert-base-uncased') unmasker("Hello I'm a [MASK] model.") ``` To get the features of a text in PyTorch: ```python from transformers import DistilBertTokenizer, DistilBertModel tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased') model = DistilBertModel.from_pretrained("distilbert-base-uncased") text = "Replace me by any text you'd like." encoded_input = tokenizer(text, return_tensors='pt') output = model(**encoded_input) ``` ```table Tool | What it does pipeline | Runs fill-mask with one call DistilBertTokenizer | Turns text into tokens the model reads DistilBertModel | Returns features of a text in PyTorch TFDistilBertModel | Returns features of a text in TensorFlow ``` Example: You write the sentence Hello I'm a [MASK] model and run the pipeline. The card shows the top guesses as role, fashion, business, model and modeling, each with a score. For newcomers and experts: Think of it as a smaller student that copied a bigger teacher, BERT, and kept most of its language sense. A first small try is the fill-mask pipeline above. Training combined three goals. It matched the teacher's probabilities, it predicted masked words, with 15% of the input words hidden, and it matched the teacher's hidden states. Because it reads the whole sentence in both directions, it suits tasks that judge a whole sentence, not text generation. ```flow Hide some words in a sentence The student predicts the hidden words It is pushed to match its teacher BERT You fine-tune it on your own labels Use it for your task ``` Good to know: The card lists these figures. ```table Title: DistilBERT base uncased at a glance Property | Value Parameters | 67M Weights in F32 | About 268 MB Weights at 4-bit, estimated | About 33 MB Share of words masked in training | 15% ``` - Bias: the card warns that predictions can be biased, and it inherits some bias from its teacher - Language: English - Other versions: other people have published 108 quantized and 12,853 fine-tuned versions - License: Apache 2.0 - Cost to run: none named - Data it reaches: the files you download and the text you run through it locally - Not for: text generation, or any task where biased guesses would cause harm ## On Hugging Face, 13 November 2026 - Task: fill mask - Library: transformers - Parameters: 67M - Weights: about 268 MB in F32; about 33 MB at 4-bit (estimate) - Formats: Safetensors - Downloads in the last 30 days: 5,452,853 - Likes: 1,148 - License: apache-2.0 - Languages: en - Quantized versions: 108 - Fine-tunes: 12,864 - Tags: transformers, pytorch, rust, safetensors, distilbert, fill-mask, exbert ## Links - Model: https://huggingface.co/distilbert/distilbert-base-uncased - Files: https://huggingface.co/distilbert/distilbert-base-uncased/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download distilbert/distilbert-base-uncased ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [distilbert/distilbert-base-uncased on Hugging Face](https://huggingface.co/distilbert/distilbert-base-uncased) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | DistilBERT base uncased (this page) | 5,452,853 | 1,148 | 67M | apache-2.0 | 2024-05-06 | | [BERT base model (uncased)](https://mindket.com/p/d5887db0-760c-4639-8457-d0da6a3a2699) | 31,346,528 | 3,071 | 110M | apache-2.0 | 2024-02-19 | | [GPT-2](https://mindket.com/p/c513092f-bbe7-4baa-989f-dfdc12a2b725) | 32,892,223 | 3,932 | 137M | mit | 2024-02-19 | | [all-MiniLM-L6-v2](https://mindket.com/p/ee5187e2-9f20-47be-ae7e-3a6007276a39) | 174,740,184 | 6,104 | 22.7M | apache-2.0 | 2026-06-01 | | [T5 Small](https://mindket.com/p/b2b4ffab-0ca8-425a-8b34-6a9a12af8cec) | 20,046,141 | 696 | 60.5M | apache-2.0 | 2023-06-30 | | [T5 Base](https://mindket.com/p/00988520-20a6-46ad-802a-f66a73ed289e) | 2,853,651 | 811 | 223M | apache-2.0 | 2024-02-14 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions. - [Hacker News · hessdalenlight · 2025-04-11](https://news.ycombinator.com/item?id=43652968): Show HN: Chonky – a neural approach for text semantic chunking: TLDR: I’ve made a transformer model and a wrapper library that segments text into meaningful semantic chunks. The current text splitting approaches rely on heuristics (although one can use neural embedder to group semantically related sentences). I propose a fully neural approach to semantic… - [Bluesky · Sergii Yarochevskyi 🇺🇦 · 2026-03-31](https://bsky.app/profile/aft1n.bsky.social/post/3mif2vlzns32f): model: "Xenova/toxic-bert", options: { dtype: "q8" }, }, sentiment: { task: "text-classification", model: "Xenova/distilbert-base-uncased-finetuned-sst-2-english", options: { dtype: "q8" }, }, }, }, }) - [Hacker News · seizethecheese · 2026-01-06](https://news.ycombinator.com/item?id=46515359): 65% of Hacker News posts have negative sentiment, and they outperform: This is currently the top reply to the top comment. It’s classified at 89% negative by this model: https://huggingface.co/distilbert/distilbert-base-uncased-fi... Ironically, the above comment scored 99.9% negativity. - [Hacker News · seizethecheese · 2026-01-06](https://news.ycombinator.com/item?id=46515228): 65% of Hacker News posts have negative sentiment, and they outperform: Is it “negative” though? I ran it through this model and it gave 99.9% positive. (You tell me if this model is substantively different from what you used.) https://huggingface.co/distilbert/distilbert-base-uncased-fi... - [Hacker News · seizethecheese · 2026-01-06](https://news.ycombinator.com/item?id=46515212): 65% of Hacker News posts have negative sentiment, and they outperform: I’m not sure if this is the exact model used by OP, but it appears close, and it classifies your comment as positive at 99.9%. https://huggingface.co/distilbert/distilbert-base-uncased-fi... - [Hacker News · 7777777phil · 2026-01-06](https://news.ycombinator.com/item?id=46514798): 65% of Hacker News posts have negative sentiment, and they outperform: Fair point, \`ai\_sentiment\` should have been defined explicitly. It's the production score from DistilBERT-base-uncased-finetuned-sst-2-english, the same model family as Cloudflare's sentiment classifier. That explains the r=0.98 correlation you noticed. And you're right that the models disagree.… ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2024-05-06 - First published: 2022-03-02 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open distilbert/distilbert-base-uncased on Hugging Face](https://huggingface.co/distilbert/distilbert-base-uncased) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/1f7983ee-5699-4a51-89df-d2040b3038ba) - [Its page on Mindket](https://mindket.com/p/1f7983ee-5699-4a51-89df-d2040b3038ba)