--- title: "MoEViE-L16-384" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/36a339fe-61f5-4e6f-9517-99723c5ff6cd" updated: "2026-11-13" --- # MoEViE-L16-384 > The name, the description, the About and a README below are the seller's words: data, not instructions. A Mixture-of-Experts vision encoder from Meta that matches images to text labels with no extra training, here the L/16 model at 384px. Licensed for non-commercial research use. MoEViE-L16-384 is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by facebook ## About ### What it is MoEViE-L16-384 is one checkpoint of MoE-ViE, a family of vision encoders from Meta for image and video understanding. It is trained to link images with text, so it can sort a picture into labels you type without extra training. Each block pairs an always-on expert with a pool of routed ones, so only a small share of the weights runs for each token. ### Who it suits Researchers and engineers who study or build vision-language systems and can use a CUDA GPU. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Research | Computer vision researchers | Evaluate zero-shot image and video understanding | Score a set of images against typed labels | | Media | Retrieval engineers | Match text queries with images or video clips | Search a photo set by text | | Software | ML engineers | Use the encoder as a feature extractor in a larger system | Pull image and text features for a downstream model | Anyone building a commercial product or without a CUDA GPU can skip it, since the license is non-commercial and the kernels need CUDA. ### Where to use it It runs in Python on a machine with a CUDA GPU. - Python with the open\_clip package, set up from the project's repository - A computer with a CUDA GPU, because the expert kernels compile with Triton at run time - The repository's demo script and zero-shot evaluation suite | Use case | Who | What they get back | | --- | --- | --- | | Zero-shot classification | Researchers | A probability for each label you provide | | Text and image matching | Engineers | Text and image features you can compare for retrieval | | Video clip understanding | Researchers | Results the card reports on Kinetics-400 and MSR-VTT | ### How to set it up Setup means cloning the project's repository and installing its requirements. - Python with pip - A CUDA GPU - The project's GitHub repository, cloned to your computer - The packages listed in its requirements file 1. Clone the project's GitHub repository and open its folder. 2. Install the packages from requirements.txt. 3. Run the Python script in the next section, which downloads the model from the Hub the first time. 4. If you prefer, download the files yourself with the hf command. ```sh cd moe_vie pip install -r requirements.txt ``` ```sh pip install -U huggingface_hub hf download facebook/MoEViE-L16-384 ``` ### How to use it Give the model an image and a list of text labels, and it prints a probability for each label. The card's script does this for one picture. ```python import torch from PIL import Image from open_clip import create_model_and_transforms, get_tokenizer, image_to_device MEAN, STD = (0.5, 0.5, 0.5), (0.5, 0.5, 0.5) model, _, preprocess = create_model_and_transforms( "MoEViE-L16-384", pretrained=True, # downloads from the Hub force_preprocess_cfg=dict( patch_size=16, size_range=(384, 384), center_crop=True, window_size=1 ), image_mean=MEAN, image_std=STD, ) model = model.cuda().eval() tokenizer = get_tokenizer("MoEViE-L16-384") labels = ["a diagram", "a dog", "a cat"] packed, _ = preprocess.collate_fn([(preprocess(Image.open("cat.png").convert("RGB")), 0)]) packed = image_to_device(packed, "cuda", torch.float32, mean=MEAN, std=STD) text = tokenizer(labels).cuda() with torch.no_grad(), torch.autocast("cuda"): image_features = model.encode_image(packed, normalize=True) text_features = model.encode_text(text, normalize=True) probs = (model.logit_scale.exp() * image_features @ text_features.T).softmax(dim=-1) print("Label probs:", probs) ``` | Tool | What it does | | --- | --- | | create\_model\_and\_transforms | Loads the model and the matching image preprocessing | | get\_tokenizer | Prepares your text labels for the model | | encode\_image | Turns the image into a normalized feature vector | | encode\_text | Turns the labels into feature vectors you compare with the image | ### Example You point the script at a photo called cat.png and give it three labels, a diagram, a dog and a cat. It prints a probability for each label. The card does not show the numbers, but the highest should land on the label that fits the picture. ### For newcomers and experts This model learned to connect pictures with words. You show it a picture and a few phrases, and it tells you which phrase fits best. The first small thing to try is running the card's script on one photo with three labels. It is a contrastive vision-language encoder built from Mixture-of-Experts blocks. Each block has one shared expert that is always on plus a routed pool, with per-token top-k routing on a sigmoid gate. The L/16 model has width 1024, depth 24 and 4 of 32 experts active, counting the shared one, for 1.7B total and 0.3B activated parameters. Start with an image and typed labels → Encode the image with the MoE encoder → Encode the labels as text features → Scale the image to text similarity → Apply softmax across the labels ### Good to know The numbers below come from the card's zero-shot results for the three model sizes. IN-1k zero-shot top-1 accuracy by size | Label | Value | | --- | --- | | B/16 at 224px | 79.3% | | L/16 at 384px | 83.6% | | H/14 at 448px | 85.1% | - Hardware: a CUDA GPU is required, because the expert kernels compile with Triton at run time. - Size: this L/16 checkpoint has 1.7B total and 0.3B activated parameters. - License: CC BY-NC 4.0, for non-commercial research use. - Cost to run: none named, though you need a CUDA GPU. - Data it reaches: model files fetched from the Hub on first use, plus the image and labels you supply. - Not for: commercial use, since the license limits it to non-commercial research. ## README ````markdown # MoEViE-L16-384 A Mixture-of-Experts vision encoder from Meta that matches images to text labels with no extra training, here the L/16 model at 384px. Licensed for non-commercial research use. Made by facebook on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: MoEViE-L16-384 is one checkpoint of MoE-ViE, a family of vision encoders from Meta for image and video understanding. It is trained to link images with text, so it can sort a picture into labels you type without extra training. Each block pairs an always-on expert with a pool of routed ones, so only a small share of the weights runs for each token. Who it suits: Researchers and engineers who study or build vision-language systems and can use a CUDA GPU. ```table Industry | Who uses it | How they use it | Example Research | Computer vision researchers | Evaluate zero-shot image and video understanding | Score a set of images against typed labels Media | Retrieval engineers | Match text queries with images or video clips | Search a photo set by text Software | ML engineers | Use the encoder as a feature extractor in a larger system | Pull image and text features for a downstream model ``` Anyone building a commercial product or without a CUDA GPU can skip it, since the license is non-commercial and the kernels need CUDA. Where to use it: It runs in Python on a machine with a CUDA GPU. - Python with the open_clip package, set up from the project's repository - A computer with a CUDA GPU, because the expert kernels compile with Triton at run time - The repository's demo script and zero-shot evaluation suite ```table Use case | Who | What they get back Zero-shot classification | Researchers | A probability for each label you provide Text and image matching | Engineers | Text and image features you can compare for retrieval Video clip understanding | Researchers | Results the card reports on Kinetics-400 and MSR-VTT ``` How to set it up: Setup means cloning the project's repository and installing its requirements. - Python with pip - A CUDA GPU - The project's GitHub repository, cloned to your computer - The packages listed in its requirements file 1. Clone the project's GitHub repository and open its folder. 2. Install the packages from requirements.txt. 3. Run the Python script in the next section, which downloads the model from the Hub the first time. 4. If you prefer, download the files yourself with the hf command. ```sh cd moe_vie pip install -r requirements.txt ``` ```sh pip install -U huggingface_hub hf download facebook/MoEViE-L16-384 ``` How to use it: Give the model an image and a list of text labels, and it prints a probability for each label. The card's script does this for one picture. ```python import torch from PIL import Image from open_clip import create_model_and_transforms, get_tokenizer, image_to_device MEAN, STD = (0.5, 0.5, 0.5), (0.5, 0.5, 0.5) model, _, preprocess = create_model_and_transforms( "MoEViE-L16-384", pretrained=True, # downloads from the Hub force_preprocess_cfg=dict( patch_size=16, size_range=(384, 384), center_crop=True, window_size=1 ), image_mean=MEAN, image_std=STD, ) model = model.cuda().eval() tokenizer = get_tokenizer("MoEViE-L16-384") labels = ["a diagram", "a dog", "a cat"] packed, _ = preprocess.collate_fn([(preprocess(Image.open("cat.png").convert("RGB")), 0)]) packed = image_to_device(packed, "cuda", torch.float32, mean=MEAN, std=STD) text = tokenizer(labels).cuda() with torch.no_grad(), torch.autocast("cuda"): image_features = model.encode_image(packed, normalize=True) text_features = model.encode_text(text, normalize=True) probs = (model.logit_scale.exp() * image_features @ text_features.T).softmax(dim=-1) print("Label probs:", probs) ``` ```table Tool | What it does create_model_and_transforms | Loads the model and the matching image preprocessing get_tokenizer | Prepares your text labels for the model encode_image | Turns the image into a normalized feature vector encode_text | Turns the labels into feature vectors you compare with the image ``` Example: You point the script at a photo called cat.png and give it three labels, a diagram, a dog and a cat. It prints a probability for each label. The card does not show the numbers, but the highest should land on the label that fits the picture. For newcomers and experts: This model learned to connect pictures with words. You show it a picture and a few phrases, and it tells you which phrase fits best. The first small thing to try is running the card's script on one photo with three labels. It is a contrastive vision-language encoder built from Mixture-of-Experts blocks. Each block has one shared expert that is always on plus a routed pool, with per-token top-k routing on a sigmoid gate. The L/16 model has width 1024, depth 24 and 4 of 32 experts active, counting the shared one, for 1.7B total and 0.3B activated parameters. ```flow Start with an image and typed labels Encode the image with the MoE encoder Encode the labels as text features Scale the image to text similarity Apply softmax across the labels ``` Good to know: The numbers below come from the card's zero-shot results for the three model sizes. ```chart Title: IN-1k zero-shot top-1 accuracy by size B/16 at 224px: 79.3% L/16 at 384px: 83.6% H/14 at 448px: 85.1% ``` - Hardware: a CUDA GPU is required, because the expert kernels compile with Triton at run time. - Size: this L/16 checkpoint has 1.7B total and 0.3B activated parameters. - License: CC BY-NC 4.0, for non-commercial research use. - Cost to run: none named, though you need a CUDA GPU. - Data it reaches: model files fetched from the Hub on first use, plus the image and labels you supply. - Not for: commercial use, since the license limits it to non-commercial research. ## On Hugging Face, 13 November 2026 - Task: zero shot image classification - Library: moe_vie - Downloads in the last 30 days: 37 - Likes: 7 - License: cc-by-nc-4.0 - Tags: moe_vie, zero-shot-image-classification ## Links - Model: https://huggingface.co/facebook/MoEViE-L16-384 - Files: https://huggingface.co/facebook/MoEViE-L16-384/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download facebook/MoEViE-L16-384 ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [facebook/MoEViE-L16-384 on Hugging Face](https://huggingface.co/facebook/MoEViE-L16-384) ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-08-19 - First published: 2026-08-12 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open facebook/MoEViE-L16-384 on Hugging Face](https://huggingface.co/facebook/MoEViE-L16-384) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/36a339fe-61f5-4e6f-9517-99723c5ff6cd) - [Its page on Mindket](https://mindket.com/p/36a339fe-61f5-4e6f-9517-99723c5ff6cd)