FreeLLMAPI

What it is

FreeLLMAPI is a gateway you run yourself. It puts the free tiers of many LLM providers behind one OpenAI-compatible address. You add your own provider keys, point your tools at the gateway, and a router picks a model for each request. The README says the router moves on to the next provider when one is rate limited. It also counts usage per key so you stay under each free-tier cap, and it keeps your keys encrypted. Besides the OpenAI format, the gateway answers in Anthropic’s Messages format and Gemini’s native format. It can also act as an MCP server, which fresh installs keep off until you switch it on under Agent compatibility on the Keys page. The maker says it is for personal experimentation and learning, not production.

1. Your tool calls one address
2. The router picks a model with a healthy key under its caps
3. It decrypts that key in memory and calls the provider
4. On a 429 or 5xx the key cools down and the next model runs
5. The answer comes back with an X-Routed-Via header

Who it suits

Developers, students and tinkerers who want to try many models without paying for each one, and who already use OpenAI-compatible tools, coding agents or SDKs. It is single-user by design, so it fits one person on one machine or on a trusted network.

Skip it if you need production reliability or one gateway for a whole team. The docs call it single-user, tell you to keep it off the internet and promise no SLA.

Where to use it

The docs give a setup route for each of these.

• Claude Code: a generator command, a launch command that writes no credentials to disk, or two environment variables
• Codex CLI: a generator command or a launch command, over the Responses API
• Gemini CLI: set by hand, through the native Gemini address
• Cursor: a command that prints a guide, with a public https address
• Cline, Roo Code, Continue, Aider, OpenCode, Goose, Qwen Code, Kilo Code and Crush: one setup command each
• OpenClaw, Hermes Agent, Pi, Reasonix, ZCode, MiMo Code and AtomCode: one setup command each
• Zed and JetBrains AI: through the gateway’s Ollama emulation
• OpenAI, Anthropic and Gemini SDKs, LangChain, LlamaIndex and curl: the base URL and your key
• ChatGPT: through the MCP server and a private tunnel

The gateway itself runs in Docker, as a desktop app with macOS and Windows installers, from a source checkout, or in Termux on Android.

How to set it up

Get it saves a zip of Mindket’s setup notes in your downloads folder, with the steps from the maker’s docs for each route and agent. The main route is the one-line installer, which needs Docker. On Windows, the install guide calls the desktop installer from Releases the easiest path.

1. Run the install line. It sets up ~/freellmapi, makes an encryption key, pulls the image and starts the container.
2. Open http://localhost:3001. A server install’s dashboard has an email and password account, so set it up or sign in, then add your provider keys on the Keys page.
3. Copy the unified API key on the Keys page. It is in the page’s header. Your tools use that key, not the provider keys.
4. Point your agent or SDK at the gateway. Use its setup command or its base URL.
5. Send one request and read X-Routed-Via. That header names the provider that answered.

curl -fsSL https://freellmapi.co/install.sh | bash

The README links the script so you can read it before you pipe it to bash. Re-running it is safe, because your .env and its encryption key are kept. For Claude Code, the clients page gives a dry run and then the real command.

export FREELLMAPI_API_KEY=<unified-key>   # or pass --api-key on each command
npx freellmapi setup-claude --url http://localhost:3001 --dry-run
npx freellmapi setup-claude --url http://localhost:3001

Docker Compose, a source checkout, the desktop app and Termux each have their own notes in the kit.

How to use it

Use the gateway like any OpenAI address. Set the base URL to http://localhost:3001/v1 and the key to your unified key. The model name picks the routing for that request. The desktop app serves on http://127.0.0.1:31415 by default and moves up a port if that one is taken. Its tray popover copies the URL and key, so desktop users put that address in place of localhost:3001.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="<unified-key>",
)

resp = client.chat.completions.create(
    model="auto",  # let the router pick; or "auto:fast", "auto:smart", a profile, or a model id
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)
print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

Every response carries an X-Routed-Via header with the provider and model that served it. When a request fell over, X-Fallback-Attempts and X-Fallback-Trail name each hop that failed and why. The API reference shows this shape:

X-Fallback-Attempts: 2
X-Fallback-Trail: groq/llama-3.3-70b key1=rate_limited; google/gemini-2.5-flash key2=timeout

The dashboard also has a Playground. It sends a chat through the router and prints the provider, the model id and the latency on the message.

Example

You use Claude Code and want to try free models behind it. You run the generator with its dry run first, then for real. The docs say it backs up the existing config and merges into it, so your other settings stay.

export FREELLMAPI_API_KEY=<unified-key>   # or pass --api-key on each command
npx freellmapi setup-claude --url http://localhost:3001 --dry-run
npx freellmapi setup-claude --url http://localhost:3001

Claude Code takes the server root, because it adds the Messages path itself. On the Keys page’s Anthropic tab you send each of its model families to auto or to a model you pin. The docs name four families: default, opus, sonnet and haiku. When the first model is rate limited, the router cools that key down and retries the next model in your chain, so the agent keeps going.

For newcomers and experts

If you are new. Many AI providers offer a small free allowance so developers can try their models. Each one has its own sign-up, its own key and its own limits. FreeLLMAPI keeps all those keys in one place and gives your tools a single address, so you do not change settings when one allowance runs out. The unified key is the one key your tools use for the gateway. Start with the desktop app or the Docker line, add one provider key, and send a test message in the dashboard’s Playground.

For experts. The architecture page says the router scores reliability, speed, capability and headroom with a Thompson-sampling bandit. It enforces per-key rate and token counters with a persistent ledger, and it fails over across up to 20 attempts within a time budget. Conversations stay on one model for 30 minutes. An opt-in handoff note tells a new model it is taking over when a switch happens mid-chat.

FREELLMAPI_CONTEXT_HANDOFF=on_model_switch

Prompt compression and a response cache are opt-in too. Embeddings route by model family and never fail over to another model, because vectors from different models do not mix. A provider key can carry monthly request and token caps that reset at 00:00 UTC on the first of each month. The docs call that a usage guard, not an exact billing limit.

Good to know

FreeLLMAPI runs on your machine, and a few paths reach out.

• Personal use: the README says it is for personal experimentation and learning, not production. Free tiers carry no SLA and variable latency, and the top models have the lowest daily caps, so answers get weaker late in the UTC day
• Provider terms: each provider’s terms still apply when traffic goes through the gateway. The review on the architecture page rates some providers as caution or avoid. Its rules of thumb are one account per provider, no reselling and no sharing your endpoint
• Exposure: the docs call it single-user with no multi-tenant auth and say not to expose it to the internet. The Docker container’s port is published on 127.0.0.1 only, unless you start it with HOST_BIND=0.0.0.0. Cursor’s row asks for a public https address, so read that guide first
• Keys: provider keys are encrypted with AES-256-GCM in SQLite and decrypted in memory for each request. Keep the same ENCRYPTION_KEY and data volume when you upgrade, or the stored keys cannot be decrypted
• Outbound calls: requests go from your machine to the providers you enabled. The router also pulls a signed model catalog from freellmapi.co twice a day. The README says the catalog server never sees your prompts, completions or provider keys. The desktop app checks Releases for updates, and with Automatic update check on it downloads new releases in the background. A Premium key is validated against freellmapi.co’s license service
• Premium: a paid live catalog exists. Free installs get the monthly snapshot, so a model reaches them 30 days after it joins the live feed. The router stays MIT-licensed and free
• The installer: the one-line install pipes a script from freellmapi.co into bash
• Other makers’ software: the docs pair some agents with install lines for other makers’ tools (OpenClaw, Hermes Agent, Pi and Reasonix) and a run line for DeepSeek’s dsh tool (npx @deepseek-ai/dsh web). The kit copies those lines but does not cover the tools, and the Hermes line pipes a script from hermes-agent.nousresearch.com into bash
• Removal: removing the app never removes your data folder, and docker compose down -v drops the Docker volume
• License: MIT. The LICENSE file reads Copyright (c) 2026 Tashfeen Ahmed