ChatCLI ships a native context compression layer that dramatically cuts the tokens consumed by the bulky payloads an agent reads (search results, logs, diffs, JSON, code) and generates (response verbosity) — without ever losing information and with no external dependency: everything is pure standard-library Go, keyless, no cgo, no trained model, no network.
The layer is reversible by design: lossy reduction only happens when the original is first written to a local store (CCR) and a marker is embedded in the prompt. The model recovers the original verbatim with @recall. Below a size threshold the output is byte-identical to before.
Why it matters
A single @search over a large codebase, a verbose go test ./..., or a JSON API response can inject tens of thousands of mostly-redundant tokens into the context. Without compression that volume:
- Saturates the context window and forces aggressive compaction (which loses information).
- Multiplies cost on every ReAct turn.
- Busts the provider’s prefix cache when volatile content enters raw.
The compression layer attacks this at the source — keeping only what the model needs to act and sending the rest to the reversible store.
How it works
A ContentRouter detects the content type (or trusts a hint from the originating tool) and routes to the right compressor. Each compressor is deterministic and type-specific:
Code is never compressed automatically. Dropping the body of a file the agent is about to edit would be harmful. code-ast only runs when explicitly requested (@compress with hint=code). Prose only auto-fires on web content (@webfetch/@websearch/@wikipedia) — reference material — never on local file reads.
CCR — Contextual Compression Retrieval
When a compressor drops part of a payload, the full original is written to a local store and a <<ccr:HASH>> marker is embedded in the output. The store is:
- Content-addressed (key = SHA-256 hash of the content) → natural dedup: identical content is stored once.
- Bounded (size cap via LRU + TTL) and crash-safe (no corruptible index — the directory is the index).
- Boundary-validated: keys are validated as fixed-width hex before becoming a filesystem path (no path traversal).
If the model needs the dropped detail, it calls @recall and gets the byte-identical original.
Zero-loss compaction
CCR is not limited to the compression router: every lossy compaction path in the agent loop archives before it cuts. The layer exposes an Archive operation — verbatim store write, no compression heuristics — used by:
- Microcompact: old tool results degraded to head+tail previews or one-line summaries embed a
<<ccr:KEY>> marker pointing at the archived original. Markers carry forward across levels — when a preview later becomes a summary, the marker survives on the summary.
- Emergency payload shrinking (context recovery): when a proxy/WAF body cap forces message content to be hard-truncated, each message is archived once before its first cut and the shrunk content keeps a recall marker.
Archive is idempotent by construction: content already carrying a CCR marker is refused (its original is stored under that key), so repeated compaction rounds never duplicate store entries. The net effect: no compaction path in ChatCLI discards bytes irrecoverably while the layer is enabled — the conversation window stays small, the knowledge stays reachable.
Compresses a payload on demand. Accepts {"content":"...","hint":"auto|log|search|diff|json|code|prose"} and returns the reduced form with the original preserved in CCR. The {"cmd":"stats"} subcommand reports session savings.
Recovers the full original from a <<ccr:KEY>> marker. Lenient by design — it accepts the bare 16-hex key, the full marker, and the paste-back variants models actually produce (ccr:KEY without angle brackets, single brackets, uppercase hex, a marker buried in surrounding text, alias JSON fields like marker or id). Multiple markers in one call return each original in a labeled section, with per-key misses reported in place. A single key still returns the original byte-identical when it fits one tool result. Use it when the compressed view omitted something you need.Long originals come back in pages. The agent loop caps every tool result (CHATCLI_TOOL_RESULT_MAX_CHARS, 20,000 characters by default), so an original larger than that comes back one page at a time, cut at a line break, with a footer giving the total and the exact call for the next page. offset and max_chars page through it; match returns only the lines that contain a text, numbered, so the model can find the part it needs without reading everything:Several keys in one call share one page. @recall markers appear anywhere ChatCLI archived bytes before a lossy rewrite — compressed tool outputs, microcompacted old tool results, emergency-shrunk messages, and aged-out skill blocks (the skill lifecycle collapses stale mid-loop skill guidance to a stub carrying its marker). One tool recovers all of them.
Automatic compression across all modes
Output-token reduction
Complementary to input compression, ChatCLI reduces the tokens the model generates:
- Verbosity steering — a static (cache-friendly) directive injected into the cached system-prompt prefix tells the model to drop preamble, restatement and ceremony and lead with the answer/action. Levels:
full (off), concise (default), minimal.
- Effort routing (opt-in) — a keyless complexity classifier lowers reasoning effort on trivial prompts. It only lowers effort for clearly-trivial prompts and only when no effort was already chosen — it never overrides a skill/user choice and never raises effort, so it cannot degrade a hard task.
Control at runtime via /config output (see below).
Image compression (vision)
Images are shrunk before reaching vision-capable models — see the Vision Input page. In short: downscale the longest edge to 1568px (what providers already do server-side, so it is token-equivalent) + re-encode photos as JPEG, preserving transparency (PNG), never inflating the payload. Keyless, pure-Go.
Configuration
/config compression
/config output
Environment variables
Each turn’s savings surface in the chat footer (e.g. 🗜 12K saved); the session total lives in /config compression stats, alongside the @recall hit-rate.
Guarantees (never degrade)
- A result that is irreversible or does not shrink → verbatim passthrough (the router rejects it).
- No CCR available (lossless mode / no store) → lossy compressors drop nothing.
- Idempotent: already-compressed content (carrying a marker) is not re-compressed.
- Tool errors go verbatim so the model can debug them.
To see the gains in practice: run a large @search/grep or a verbose go test in /agent, observe the compressed output + the <<ccr:...>> marker, then @recall to confirm the byte-identical original, and /config compression stats for session savings.
