Billion Context

What it is

Billion Context is a proxy that runs on your computer between a coding agent and the model it calls. The model gets four tools to manage its own history: compress, decompress, search_context and acp_status. It folds a range of old messages into a summary, and it can unfold that range or search the summaries when exact detail matters. Its maker, ranxianglei, describes it as a context-compression plugin for month-long single sessions, small context windows (a 100K window is enough) and token savings (5x fewer tokens). Those figures are the maker’s claims.

1. Your agent sends its request to the local proxy
2. The proxy adds the context tools and a growth nudge
3. The model folds old messages into a summary
4. The proxy sends the shorter request to the real API
5. Decompress brings exact detail back when needed

Who it suits

Developers who leave a coding agent on one task for hours or days and watch old tool output fill the window. It also suits people on models with small windows. People who pay per token get a cache report that shows what each fold saved. Skip it if your sessions are short, or if you do not want a local proxy that updates itself. Exact recall of one sentence buried in a huge document is also weaker: the maker’s own benchmark loses ground there.

Where to use it

The README names 21 clients in its table and lists launchers for three more. The topic dsh-plugin on the repository points at dsh, an open-source project from DeepSeek AI for running agents. A client connects by one of three routes.

• Native plugin: bili plugin install plus the client’s name, then you start the client as usual. Offered for pi, omp, OpenCode, dsh, Kimi Code, Hermes, ZCode and Claude Code
• Launcher: bili plus the client’s name starts the proxy and the client together, and edits no real config file
• URL prefix: put http://localhost:8787/bili/ in front of the client’s base URL, for any client that lets you edit it

How to set it up

Get it saves a zip of Minda’s setup notes in your downloads folder, with each client’s steps from ranxianglei’s README and docs. The main route is one npm install, then one command for your client.

1. Install the package. On Linux and macOS use the user-level prefix, and on Windows run the plain command.
2. Run bili --version, and add ~/.local/bin to your PATH if the command is not found.
3. Run the launcher or the native install for your client, and restart the client after a native install.

npm install -g billion-context --prefix=~/.local

npm install -g billion-context

The launcher is the lightest route. It edits no config, and traffic goes direct again the moment you stop running the command.

bili claude

Claude Code can also add it from inside a session, as a plugin marketplace.

/plugin marketplace add ranxianglei/billion-context
/plugin install billion-context@billion-context
/billion-context:bili-setup

How to use it

Work in your client as before. The model decides when to compress. By default the proxy nudges it as the conversation grows, in steps of about 50K tokens. It also folds on its own as a hard backstop when a request alone would overflow the window. The first fold comes 50K tokens past the starting content, so a large system prompt and tool list push it later.

In OpenCode, dsh and Hermes, typing /acp shows the status panel for the current conversation. The /acp-cache command prints a cache report with a health verdict and the net tokens saved per fold. The maker’s rule of thumb: compression itself costs 2 percent or less, and a healthy session sits at 95 to 97 percent cache hits. A local page at http://localhost:8787 adds logs and a config editor.

Example

You use Claude Code with an API key and want proof that the proxy answers before a long session, so you start it in one terminal and ask for its health from another.

# terminal 1
bili

# terminal 2
curl -s http://localhost:8787/__bili/health

The README shows this answer, naming the upstream the proxy forwards to:

{"ok":true,"upstream":"https://api.anthropic.com"}

Then you point Claude Code at the proxy and send a message. In this manual mode CONFIGURATION.md also asks you to set CLAUDE_CODE_AUTO_COMPACT_WINDOW to bili’s window for your model, so Claude Code’s own auto-compact does not fire first.

export ANTHROPIC_BASE_URL="http://localhost:8787/bili/https://api.anthropic.com"
claude

As the conversation grows, the log at ~/.local/state/billion-context/bili.log shows a line for each round and a compress event, in the README’s format:

[acp-usage] round N input=X cached=Y (cache hit Z%)

For newcomers and experts

If you are new. A model can only read so much at once, and a long coding session eventually passes that limit, after which the README says the session degrades or dies. Billion Context lets the model fold old parts into summaries and unfold them again. Start small. Try the launcher with the one agent you use daily, on a throwaway project.

For experts. With a native plugin or a launcher, the agent registers the tools and runs compress itself. On the plain URL route the proxy runs compress on its side. Session state is keyed on the conversation id the client gives, such as Claude Code’s x-claude-code-session-id header. A client that sends none gets prefix affinity, which reattaches a request only when its history matches a stored session byte for byte.

By default the model your agent already uses writes the summaries. An optional external summary chain in the config names up to 16 other models, each with its own key. It never falls back to the main model when a reference does not resolve.

That run sits in the repository’s benchmarks folder, which calls it an independent third-party evaluation of version 0.1.202 with DeepSeek V4.1 Flash as the base model. It names no evaluator and is published in the maker’s own repository, so read it as the maker’s claim. The same write-up reports 71.1 percent lower cost than full context for independent sessions, and a 1.48M-token test where full context failed on all 199 items. Its trade-off line says exact needle questions fall from 69.0 percent for full context to 27.0 percent.

Good to know

The proxy sees every prompt and reply you route through it. It also changes itself over time.

• Where requests go: to the model API your client already uses. Billion Context has no model of its own, and the README and CONFIGURATION.md name no server run by the maker in that path
• Your keys: the Authorization or API-key header is forwarded untouched. The README says the raw key is never stored, only a 12-character hash
• What it stores: raw messages and summaries under ~/.local/share/billion-context/ as plain JSON plus a SQLite index, unencrypted unless you set BILI_ENCRYPTION_KEY
• Local web page: http://localhost:8787 listens on loopback with no login. By default every bili instance on a machine shares one store, and any of them can read every session
• External summary chain: text goes to the providers you name, with keys kept in a private credentials file
• TLS interception: some launchers, and the proxy setup for clients such as ZCode, decrypt the HTTPS traffic to your model host so the proxy can read it. They use a root CA that bili makes at ~/.local/share/billion-context/ca/root-ca.pem. CONFIGURATION.md says it is not a system-wide install, only the client you point at it trusts it, and only listed model hosts are decrypted while other traffic passes through still encrypted. A root CA can sign certificates, so keep that folder as private as a password file. Deleting it makes bili create a new one. The Claude Code launcher needs no CA

• Self-update: it checks npm every 3 minutes and installs a newer version on its own after a random delay of 15 minutes to 48 hours. ACP_AUTO_UPDATE=0 turns that off. A separate advisory check can force-install a named version even then, and BILI_ADVISORY_CHECK=0 turns it off
• Other fetches: the update policy file from the repository, a plain GET that CONFIGURATION.md says sends no query parameters or device id, and the blog panel, refreshed from GitHub about once an hour
• No login on shared use: the --host 0.0.0.0 flag and the Docker image listen with no authentication
• Limits: the README’s Status section calls the project early and says to expect rough edges. The master branch can break, so install from npm. Compression is lossy, as the exact-recall drop above shows
• Removal: bili plugin remove undoes a native install. The README gives no command that removes the package or its folders
• License: MIT plus an Additional Term. GitHub shows no standard license because of it, and the npm entry lists MIT. Any product or service that includes the software and that end users see or use, commercial or open source, must credit billion-context with a link on its home page, documentation or About page. Server-side use with no user-facing surface can carry the notice in its shipped documentation. As we read it, setting this up for your own coding is not such a product, but read the LICENSE file before you ship something on top of it. Minda’s listing is notes only and ships none of the software