Sarvam-30B

Sarvam-30B

A mixture-of-experts language model from Sarvam AI with 32.2B parameters and 2.4B active per token, built for practical use in English and many Indian languages. Needs an NVIDIA GPU for the card's sample code.

Made by sarvamai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Sarvam-30B is a mixture-of-experts language model from Sarvam AI with 32.2 billion parameters in total and 2.4 billion non-embedding parameters active for each token. The card aims it at practical deployment, with reasoning, coding and chat in Indian languages. It also says the model can handle multilingual voice calls that make tool calls.

Who it suits: Developers and researchers who want an open model for Indian languages and English, and who have an NVIDIA GPU. A larger sibling, Sarvam-105B, exists for people who want more capacity.

Industry | Who uses it | How they use it | Example
Customer service | Support teams | Build chat and voice assistants in Indian languages | Answer a customer question in Tamil
Software | Developers | Add a multilingual open model to an app | Generate replies in Hindi from a script
Education | Teachers and learners | Practice and ask questions in many languages | Explain a math step in Marathi
Research | AI researchers | Study a small-active-parameter expert model | Compare it with other models near its size

Anyone who needs strong repair of code in big repositories can skip it, since the card's own table shows other models ahead on that test.

Where to use it:

• Python with the Hugging Face transformers library, on a machine with an NVIDIA GPU
• Serving tools such as vLLM and SGLang, which the card's index points to
• Windows, macOS or Linux for downloading the files

Use case | Who | What they get back
Text generation | Developers | Replies sampled with the settings you choose
Multilingual chat | Indian-language users | Answers in languages such as Hindi, Bengali and Telugu
Tool-calling assistants | Agent builders | A model the card says can call tools during voice calls

How to set it up:

• Python with pip
• The torch and transformers libraries
• An NVIDIA GPU, since the card's code sends its inputs to the first CUDA device
• Enough memory, since the card record estimates about 129 GB in full precision and about 16.1 GB at 4-bit

1. Install the Hugging Face hub package.
2. Download the model files, or let transformers fetch them on the first run.
3. Run the card's Python code, which loads the tokenizer and the model.

pip install -U huggingface_hub
hf download sarvamai/sarvam-30b

How to use it: Load the model with the code below, then define the helper function. Call it with your prompt text, and it returns the decoded text, which includes your prompt followed by the model's continuation.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

model_name = "sarvamai/sarvam-30b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="auto")

def generate_text(
    prompt: str,
    max_new_tokens: int = 2048,
    temperature: float = 0.8,
    top_p: float = 0.95,
    repetition_penalty: float = 1.0,
) -> None:
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda:0")

    generation_config = GenerationConfig(
        max_new_tokens=max_new_tokens,
        repetition_penalty=repetition_penalty,
        temperature=temperature,
        top_p=top_p,
        do_sample=True,
    )

    with torch.no_grad():
        output_ids = model.generate(
            input_ids=inputs["input_ids"],
            attention_mask=inputs["attention_mask"],
            generation_config=generation_config,
        )
    return tokenizer.decode(output_ids[0], skip_special_tokens=True)

Setting | Default in the card's code | What it does
max_new_tokens | 2048 | Caps how many new tokens a reply can have
temperature | 0.8 | Controls how varied the wording is
top_p | 0.95 | Limits sampling to the most likely tokens
repetition_penalty | 1.0 | Leaves repeats unpenalized

Example: You load the model and call the helper with a short question in Hindi. After the weights finish loading, it returns the question plus the model's answer as one block of text.

For newcomers and experts: If you are new, start with the defaults and a short prompt, and watch the memory use on your GPU. Then change the temperature to see how the wording shifts.

Experts will note the card lists 19 layers, 128 experts with one shared expert, and top-6 routing for each token. It uses grouped key-value attention with 4 heads and a large rope theta of 8e6 for stable long context without RoPE scaling. The design favors speed and low memory over depth.

Title: Scores the card reports for Sarvam-30B
Math500: 97.0
MBPP: 92.7
HumanEval: 92.1
MMLU: 85.1
MMLU Pro: 80.0
Live Code Bench v6: 70.0

Load the tokenizer and model
Turn the prompt into tokens
Generate text with the sampling settings
Decode the tokens back into text

Good to know:

• Agent tests: in the card's table it trails some peers on SWE Bench Verified and the tau-squared benchmark, so test it on your own tasks
• Remote code: the loading line sets trust_remote_code to true, which runs code stored with the model, so read that code first
• Other versions: Hugging Face lists 35 quantized and 13 fine-tuned versions made by other people
• Languages: the card claims strong results across 22 Indian languages, and the page record lists 8 including English
• License: Apache 2.0
• Cost to run: none named, no API key or account, but a GPU with plenty of memory
• Data it reaches: it downloads files from Hugging Face, and your prompts stay in your own code
• Not for: machines without a suitable GPU, or heavy code-repair agent work where the card's table shows other models ahead

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 32.2B
• Weights: about 129 GB in F32; about 16.1 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 3,804
• Likes: 235
• License: apache-2.0
• Languages: en, hi, bn, ta, te, mr, gu, kn
• Quantized versions: 35
• Fine-tunes: 13
• Tags: transformers, safetensors, sarvam_moe, text-generation, conversational

Links

• Model: https://huggingface.co/sarvamai/sarvam-30b
• Files: https://huggingface.co/sarvamai/sarvam-30b/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download sarvamai/sarvam-30b

Then follow the usage in its model card. Nothing in this kit runs by itself.