Skip to main content

Session Pruning

Session pruning trims old tool results from the in-memory context right before each LLM call. It never rewrites stored session history: the database keeps every entry, and only the request view is trimmed.

When it runs

  • When mode: "cache-ttl" is enabled and the last Anthropic call for the session is older than ttl.
  • Only affects the messages sent to the model.
  • Only active for Anthropic API calls (and OpenRouter Anthropic models).
  • For best results, match ttl to your model cacheRetention policy (short = 5m, long = 1h).
  • Pruning decisions are sticky: once a TTL window expires and a prune round runs, the trimmed tool results stay trimmed for the rest of the session. The prune round is recorded per session and is not undone by later requests within the same session; a restart re-derives the same trimmed view from the recorded decisions.

Smart defaults (Anthropic)

  • OAuth or setup-token profiles: enable cache-ttl pruning and set heartbeat to 1h.
  • API key profiles: enable cache-ttl pruning, set heartbeat to 30m, and default cacheRetention: "short" on Anthropic models.
  • If you set any of these values explicitly, WednesdayAI does not override them.

What this improves (cost + cache behavior)

  • Why prune: Anthropic prompt caching only applies within the TTL. If a session goes idle past the TTL, the next request re-caches the full prompt unless you trim it first.
  • What gets cheaper: pruning reduces the cacheWrite size for that first request after the TTL expires.
  • Why the trimmed view sticks: once a prune round runs, follow‑up requests keep the trimmed tool results so they do not re-cache the full history again.
  • What it does not do: pruning doesn’t add tokens or “double” costs; it only changes the request view, never stored history.

What can be pruned

  • Only toolResult messages.
  • User + assistant messages are never modified.
  • The last keepLastAssistants assistant messages are protected; tool results after that cutoff are not pruned.
  • If there aren’t enough assistant messages to establish the cutoff, pruning is skipped.
  • Tool results containing image blocks are skipped (never trimmed/cleared).

Context window estimation

Pruning uses an estimated context window (chars ≈ tokens × 4). The base window is resolved in this order:
  1. models.providers.*.models[].contextWindow override.
  2. Model definition contextWindow (from the model registry).
  3. Default 200000 tokens.
If agents.defaults.contextTokens is set, it is treated as a cap (min) on the resolved window.

Mode

cache-ttl

  • Pruning only runs if the last Anthropic call is older than ttl (default 5m).
  • When it runs: same soft-trim + hard-clear behavior as before.

Soft vs hard pruning

  • Soft-trim: only for oversized tool results.
    • Keeps head + tail, inserts ..., and appends a note with the original size.
    • Skips results with image blocks.
  • Hard-clear: replaces the entire tool result with hardClear.placeholder.

Tool selection

  • tools.allow / tools.deny support * wildcards.
  • Deny wins.
  • Matching is case-insensitive.
  • Empty allow list => all tools allowed.

Interaction with other limits

  • Built-in tools already truncate their own output; session pruning is an extra layer that prevents long-running chats from accumulating too much tool output in the model context.
  • Tool results are additionally capped at 400,000 text characters in memory (raw bytes are stored out-of-band when database write-through is enabled). The cap truncates text only and keeps image blocks. When a result is later trimmed to fit the context window, its images keep at most half the trimmed budget and over-budget images are replaced with [N image(s) omitted from context]. These limits are separate from pruning and apply regardless of contextPruning.
  • Compaction is separate: compaction appends a summary entry to stored history, while pruning only changes the request view (its decisions stay sticky for the session). See /concepts/compaction.

Defaults (when enabled)

  • ttl: "5m"
  • keepLastAssistants: 3
  • softTrimRatio: 0.3
  • hardClearRatio: 0.5
  • minPrunableToolChars: 50000
  • softTrim: { maxChars: 4000, headChars: 1500, tailChars: 1500 }
  • hardClear: { enabled: true, placeholder: "[Old tool result content cleared]" }

Examples

Default (off):
Enable TTL-aware pruning:
Restrict pruning to specific tools:
See config reference: Gateway Configuration

Reset, delete, and where old history goes

Stored history is append-only and kept forever by default. Resetting (/new, /reset) or deleting a session removes only its index row (a tombstone); the conversation rows stay in the database and stay readable — enforce-mode retention only retires the index row as well, it never deletes conversation rows (see Session Management / Maintenance). The legacy JSONL archival machinery (per-session *.jsonl.reset.*/*.deleted.* archives and session.archival in-session trimming) is retired: SQL session storage never writes transcript files, so core no longer invokes it. The archiveSessionTranscript plugin-SDK export is deprecated and unused by core: calls reject unsupported file trimming without touching the filesystem. It is removed in the next release. sessions.delete ignores its deprecated deleteTranscript parameter and returns an empty deprecated archived list; neither reset nor delete moves legacy sources. Only the importer moves verified sources into <state-dir>/legacy-session-archive/<agentId>/ (see Upgrading from JSON session files).

Troubleshooting

Pruning is enabled but tool results are not being trimmed Pruning only fires for mode: "cache-ttl" when the last Anthropic call for the session is older than ttl (default 5m). If requests arrive within the TTL window, pruning intentionally skips so the cached prompt stays valid. Wait for the session to go idle past ttl, then send another message. Also check that contextPruning.mode is set in agents.defaults (not a top-level key) and that the gateway has restarted since the config change. Tool results with images are never pruned even when context is large This is expected behavior. Pruning skips tool results containing image blocks (they are never soft-trimmed or hard-cleared by pruning). Separately, when an image-bearing result is trimmed to fit the context window, its images keep at most half the trimmed budget and over-budget images are replaced by an [N image(s) omitted from context] note. If image-heavy tool results are inflating context, consider capping image output at the tool level or reducing how many images are returned per tool call. A reset or deleted session still has conversation rows in the database That is expected: reset and delete remove only the index row (a tombstone); stored history is append-only and never deleted. The rows stay readable, and nothing serves them to the model once the index row is gone. /compact ran but the stored entry count grew Also expected: compaction appends a summary entry and keeps every pre-compaction entry, so the database only grows. Trimming happens in the request view, never in storage. Related: Session management · Compaction · Gateway configuration