Skip to main content

Session Consumer Claims

The session-consumer claim ledger is a durable obligation-tracking primitive that records which ended sessions each registered consumer still owes work on. It is the foundation of the Cognitive Substrate program’s session-lifecycle reliability layer (sub-project 0s).

How it works

When a session ends (replaced via /new or /reset), a core-owned reconciler sweep materializes claim rows for every registered consumer that declares interest in session.end effects. Each claim represents an outstanding obligation: “consumer K still needs to process session S.” Conversation history is never deleted (ADR 0098); the ledger exists so consumers can resume exactly where they left off. The gateway sweep runs on a sweepCadenceMs timer and performs two operations per tick: (1) reconcileSessionConsumerClaims reads one row per session_id past each consumer’s watermark, keyset-paged on (updated_at, session_id), and materializes a claim only once the session has ended, meaning it can no longer receive turns under its reset policy: superseded by a newer session_id on its key (/new, reset), or past the daily reset boundary or idle reset window of every configured reset policy, so the longest window wins (default: the next 04:00). A live session is never claimed early; the watermark stays just below it until it ends (ADR 0079). Turns that land on a session after its claim become a follow-up claim segment: the claim’s segment is 1, 2, … and fromSeq/toSeq bound its transcript, so key captures on (sessionId, segment); (2) reapExpiredLeases resets crashed/expired claims — claims whose attempt_count is still under maxAttempts go back to pending, claims at or over the budget transition to dlq (and emit a dlq_terminal diagnostic once). Consumers claim work via api.claimSessionWork() (loader-bound consumerId, fail-closed authz, limit fixed at 1), process it, and acknowledge via api.ackSessionWork(). The claim/lease pattern mirrors ctx_storage_outbox (FOR UPDATE SKIP LOCKED on Postgres, BEGIN IMMEDIATE on SQLite) with a 30s lease and exponential backoff retry.

Agent ID scoping

Claims are scoped to the agents this gateway serves: every agent in agents.list (ADR 0078 D8, ADR 0079). Consumer ids are agent-qualified (<agent>:<plugin>:<effectKind>), each agent keeps its own watermark, and claimSessionWork claims across the agents in rotation with a per-row agent guard. The sessionConsumerRegistry is built from manifest data before the plugin loop starts; the claim store and agent binding resolve at claim time. When no agent ID is configured (single-agent setups), the registry is not built and claims are unscoped — the SQL agent_id filter becomes a wildcard. The gateway sweep warns once and returns early when no agent ID is configured, preventing claim materialization for the wrong agent. Duplicate plugin IDs in the manifest registry are deduplicated using first-seen precedence, matching the loader’s seenIds logic.

Architecture

Schema (two tables)

  • session_consumer_claims — the claim ledger. PK claim_id (uuid). Unique idempotency key (session_id, consumer_id, effect_kind, schema_version). Statuses: pending | claimed | processing | processed | dlq. The live state machine is pending → claimed → processed | dlq (the reaper moves over-budget claims to dlq; under-budget expired leases return to pending). processing is a documented placeholder retained for forward compatibility — no transition sets it today. Tracks lease owner, lease until, attempt count, last progress timestamp, last error.
  • session_consumer_watermarks — per-consumer per-lane watermark. PK (consumer_id, lane_id). The watermark advances to max(updated_at) of materialized sessions. Disable (enabled = 0) preserves the watermark; only uninstall (purgeConsumer) drops it.
Lane is derived in-memory from session_key (format agent:{agentId}:{channel}:{method}:{peerId}) — there is no lane_id column on ctx_sessions (D0s-10). The updated_at column (last activity) serves as the effective session-end timestamp.

Backend implementations

Postgres timestamp columns (dlq_at, lease_until, …) are bigint epoch-ms. node-postgres sends parameters untyped, so a statement whose only typed anchor is the parameter — e.g. dlq_at = CASE WHEN ... THEN $2 ELSE NULL END — resolves the expression to text and fails with column "dlq_at" is of type bigint but expression is of type text. Cast such parameters explicitly (THEN CAST($2 AS BIGINT)); a bare assignment (dlq_at = $2) takes the column type directly and needs no cast. SQLite is dynamically typed and unaffected.

Plugin manifest extension

  • effectKind — the effect this consumer processes (default: session.end).
  • subCursors — bounded sub-cursor suffixes (max 8), each ^[a-z][a-z0-9-]*$.
  • consumeScopes — manifest-declared authz scopes (fail-closed: mismatch → null). Must be an array of strings; a bare string is rejected at manifest validation time.
  • consumeLanes — lane scopes the consumer is authorized for. Wildcard "*" authorizes any lane; absent (omitted) defaults to the single-user "" lane. An explicit empty array means “no lanes authorized” and the consumer is skipped during the sweep. Requesting an unauthorized lane at claim time is fail-closed (returns null). Must be an array of strings; a bare string is rejected at manifest validation time.

SDK surface

The legacy openclaw/plugin-sdk specifier still loads (ADR 2057). Read denials from the session store surface as StorageReadAccessError with a code and a retryable flag: session_not_visible (the owner row is not visible yet; retryable: true, so deferSessionWork the claim behind a nonzero retryAfterMs — acking failed would make the same drain re-claim it immediately and burn the attempt budget toward dlq) or session_foreign_owner (the session belongs to a different agent; retryable: false, straight to dlq). Read helpers such as transcript reads return empty results on those errors; only retry/ack decisions should branch on the flag. The not-visible defer must stay bounded: an owner row that never becomes visible (e.g. a permanently failed primary write) would otherwise be deferred forever, since deferSessionWork is attempt-neutral and never reaches dlq. Defer only while the claim is younger than a visibility window measured from createdAt (the claim row’s materialization time, created_at; learning-core uses 10 minutes — the owner row normally becomes visible within ~250 ms of the primary write). Do not measure from since, which is the consumer scan watermark, not materialization time. Past the window — or when the backend does not record createdAt, so the age is unknown — ack failed with retryable: true and let the normal attempt budget take the claim to dlq.

Deferring a claim without spending an attempt

api.deferSessionWork(claimId, leaseOwner, { retryAfterMs }) returns a claim to pending without counting the attempt against its retry budget. Use it when the work cannot run yet and retrying would only burn attempts toward dlq: a store outage after the claim, an enqueue that recovery admission refuses, or work that must wait for the next UTC day.
  • Every claim carries its own lease token in leaseOwner (<process>:<token>). Pass it explicitly; there is no claim-id fallback. The update only applies while the claim is still claimed by that token.
  • A deferred claim is pending again with its attempt given back and retryAfterMs as a not-before time: nothing claims it earlier, and the reaper leaves it alone. Deferring it any number of times never moves it to dlq.
  • A defer (or a late ack) from an owner whose lease was reaped and reclaimed by another handler returns "stale" and changes nothing.
Log the processing error before acking a failure. If ackSessionWork itself throws (a store bug, a transient backend error), the claim is never marked — the lease expires, the reaper resets it, and the hook warning on the next sweep only shows the ack failure, not the real processing error. Log the processing error via api.logger.warn first so it is never masked by a failing ack. consumerId is not a parameter — it is loader-bound from record.id (the plugin’s manifest id), the effect kind, and an optional /<subCursor> suffix, in the form ${manifestId}:${effectKind}[/<subCursor>]. The :effectKind suffix avoids consumerId collisions between consumers of different effect kinds declared by the same plugin. This is an irreversible contract (ADR 0026, D0s-3).

onSessionWorkAvailable hook

Plugins can register a callback that the gateway sweep invokes after reconciliation and lease-reap on each tick. Use it to drive claim/ack loops without running a separate timer:
The hook receives { effectKind: string } — the same effectKind the consumer declared in its manifest. The sweep calls the hook for each effect kind after materializing claim rows, so the plugin should drain work in a loop until null (as in the example above). The hook is additive (spec D2) — plugins that track their own timers can skip it and call claimSessionWork directly.

ClaimedSessionWork shape

workspaceDir is recorded when the ended session is materialized: api.runtime.resolveLearningWorkspaceDir(agentId) over the live config. Claims materialized before the column existed read null; resolve them with the same runtime method, and never fall back to a guessed directory. agentId is the agent whose session was claimed: the gateway’s runtime agent or any agent in agents.list, never a value from the plugin manifest. One gateway serves every configured agent, so a single claimSessionWork call can return a claim for any of them; branch on agentId (and resolve its workspace from workspaceDir) rather than assuming the runtime agent.

Wildcard lane claiming

When a consumer declares consumeLanes: ["*"], omitting laneId from claimSessionWork() (or passing undefined) claims work from any materialized lane — the SQL wildcard check (lane_id IS NULL) matches all lanes. Passing an explicit laneId scopes the claim to that lane even when the consumer is wildcard-authorised.

Config keys

Diagnostic events

Emitted via emitDiagnosticEvent():
  • dlq_terminal — fires once, at the DLQ transition (when the lease reaper moves a claim to dlq because attempt_count >= maxAttempts). Guarded by rowCount (Postgres) / changes (SQLite) to prevent duplicate events when the MARK_SCC_FAILED update is a no-op (stale lease).
  • skipped_backlog — fires on the sweep that sees a consumer’s watermark go from disabled to enabled; count is how many sessions the consumer skipped while it was disabled (bounded to one page, so a lower bound for larger backlogs). Payload carries consumerId and count, no sessionId.

Key design decisions

  • D0s-2 (irreversible): Idempotency key (session_id, consumer_id, effect_kind, schema_version).
  • D0s-3 (irreversible): consumerId is loader-bound, not a parameter.
  • D0s-4 (irreversible, superseded): Prune transitions from age-based to obligation-gated. The prune gate and the age-based prune path were removed entirely (ADR 0098: conversation history is never deleted); the claim ledger now only tracks resumable obligations.
  • D0s-7: jsonl-fs is degraded (no crash-durability); claim/lease ops throw.
  • D0s-10: No lane_id/ended_at on ctx_sessions; lane derived from session_key.
See ADR 0026 (dev-docs/adr/0041-session-consumer-claim-ledger.md) for the full irreversible-decision record.

Limitations (hardening items)

  1. retryBaseMs is resolved from config but not yet wired through to the claim store’s retry-delay scheduling (hardcoded backoff in the reconcile/lease layer). leaseMs and maxAttempts are wired through (src/gateway/server.impl.ts).
  2. dlq_terminal diagnostics are emitted through the global event system, not the session-consumer ring buffer, so sessions.claims.status does not surface them. skipped_backlog appears in its recentEvents.