exo

What it is

exo links your devices into one AI cluster. Its README says exo lets you run models larger than would fit on a single device, and with day-0 support for RDMA over Thunderbolt it makes them run faster as you add devices. Each device serves a dashboard and an API on port 52415. The API speaks the OpenAI Chat Completions, Claude Messages, OpenAI Responses and Ollama formats, so clients you already use can talk to it.

1. You install exo on each device
2. Devices on the same network and version find each other
3. exo splits a model across the devices it can see
4. Each device serves a dashboard and an API on port 52415
5. Your chat app or code sends requests to that address

Who it suits

People with more than one capable device, most clearly several Apple silicon Macs, who want to run a model too large for any one of them. The README’s dashboard picture shows four 512 GB M3 Ultra Mac Studios. On Linux the README says exo currently runs on CPU. The contributing guide says the codebase is actively evolving and that testing is mostly manual.

Where to use it

exo runs on macOS and Linux. Its README and API reference name these ways to reach it.

• The built-in dashboard at http://localhost:52415, for managing your cluster and chatting with models
• The OpenAI Chat Completions, Claude Messages, OpenAI Responses and Ollama endpoints, which the API reference says existing clients can use after a base URL change
• OpenWebUI and similar tools, through the Ollama endpoints
• exo-bench, which measures prefill and token generation speed
• Custom models from the Hugging Face hub, added through the /models/add endpoint

How to set it up

Get it saves a zip of Mindket’s setup notes in your downloads folder. It has a page for the app, each source install and the tasks around them, taken from exo’s README, API reference, contributing guide, uninstall script and pyproject.toml. On a Mac, the app is the README’s easiest route.

1. On a Mac, download EXO-latest.dmg, drag EXO to Applications and open it. Or use Homebrew.
2. Allow Local Network access when macOS asks, so exo can find your other devices.
3. Repeat on each other device, on the same network or joined by a Thunderbolt cable, and check the versions match.
4. Open http://localhost:52415 to see the dashboard.

brew install --cask exo

To run from source on macOS, the README gives these lines. On Linux it swaps the uv sync line for uv sync --extra mlx-cpu.

git clone https://github.com/exo-explore/exo
cd exo/dashboard && npm install && npm run build && cd ..
uv sync --extra mlx
uv run exo

How to use it

Chat in the dashboard, or call the API. The README walks through four steps with a small model, mlx-community/Llama-3.2-1B-Instruct-4bit.

1. Preview the placements for your model with GET /instance/previews.
2. Create the instance with POST /instance, using a placement from step 1, and wait for it with GET /instance/await. The create call is asynchronous.
3. Send a chat completion to /v1/chat/completions.
4. Delete the instance with DELETE /instance/YOUR_INSTANCE_ID when you are done.

curl -N -X POST http://localhost:52415/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "user", "content": "What is Llama 3.2 1B?"}
    ],
    "stream": true
  }'

The same server answers /v1/messages for the Claude format, /v1/responses for OpenAI Responses and /ollama/api/chat for Ollama. The README gives a curl line for each.

Example

You have two Macs and want them to work as one cluster. You install EXO on both with Homebrew and open it on each.

brew install --cask exo

You put both Macs on the same Wi-Fi or Ethernet network and check that they run the same version. The README says the second Mac shows up in the dashboard within a few seconds. If it doesn’t, the README’s checklist starts with the version, then the Local Network switch in System Settings on both Macs. To see the version in the app, choose About EXO from the menu bar icon or open Settings › General. A source install prints it with this line.

uv run exo --version

The app and a source checkout number their versions differently, so by default they don’t join each other. Give every device the same cluster name to mix them.

For newcomers and experts

If you are new. A cluster here means your own devices sharing the work on one model. exo reads the layout of your network, picks how to split the model and handles the rest. Start with the app on one Mac, add a second, and try the README’s small model before any large one. Models are big files, so check the free space on the drive that holds ~/.exo/models. The /state endpoint shows the instance IDs and what is deployed.

For experts. The README lists topology-aware auto parallel, which weighs device resources and the latency and bandwidth of each link. It also lists tensor parallelism, with a claimed speedup of up to 1.8x on 2 devices and 3.2x on 4. For RDMA over Thunderbolt 5 it claims a 99% cut in latency between devices. The inference backend is MLX, with MLX distributed for communication.

A preview lists a sharding, Pipeline in the README’s sample, and an instance_meta, MlxRing in the sample. exo-bench filters on ring, jaccl or both, and on pipeline, tensor or both. Cancelling works by closing the HTTP connection, or by calling /v1/cancel/{command_id}.

EXO_MODELS_DIRS adds writable model folders, tried in order after the default, and the first with enough free space is used. EXO_ZENOH_NAMESPACE does the same as --namespace. The --no-worker flag makes a coordinator-only node. Models are described by TOML model cards, and the contributing guide lists the required fields.

Good to know

The app and the dashboard work on your devices, and a few steps reach into your system or the internet.

• Thunderbolt network setup: it needs your administrator password. The app installs a small system service, creates a network location called exo, switches the Mac to it and turns off Thunderbolt Bridge. Fixed IP, DNS and proxy settings from your usual location don’t apply there
• Login: the welcome window has Open EXO at login ticked. Untick it there, or in Settings › General
• Versions: devices on different versions don’t join, so update every Mac in the cluster together
• Linux: the README says exo currently runs on CPU, with GPU support under development
• Your data: exo keeps it in ~/.exo on a Mac. Models go in ~/.exo/models, and the log is ~/.exo/exo_log/exo.log
• Internet: EXO checks for updates by itself, and models are fetched from Hugging Face. EXO_OFFLINE=true uv run exo runs without an internet connection, using only local models
• The API: the reference describes no sign-in or API key for its endpoints
• Custom models: trust_remote_code is off by default. Only turn it on for a model whose code you trust
• Install lines: the README’s Homebrew installer, uv installer (Linux) and rustup lines download a script and run it at once, so read them first
• Removing it: if you agreed to the Thunderbolt setup, moving EXO.app to the Trash or running brew uninstall leaves the Mac on the exo location with Thunderbolt Bridge off. Run the uninstall script with sudo to restore them. It also deletes ~/.exo, and keeps downloaded models unless you pass --delete-models
• License: Apache-2.0. The LICENSE file reads “Copyright 2025 Exo Technologies Ltd”