Logging
WednesdayAI logs in two places:- File logs (JSON lines) written by the Gateway.
- Console output shown in terminals and the Control UI.
Where logs live
By default, the Gateway writes a rolling log file under:/tmp/openclaw/openclaw-YYYY-MM-DD.log
The date uses the gateway host’s local timezone.
Note:You can override this in/tmpis periodically purged by the OS. For persistent log retention, usewednesdayai logs --followfor live output, or override the log path to a stable location such as~/.openclaw/logs/openclaw.log.
~/.openclaw/openclaw.json:
How to read logs
CLI: live tail (recommended)
Use the CLI to tail the gateway log file:--follow reads the log file directly and keeps
following across rotation and Gateway restarts. With gateway.mode: "remote", --url, or
--rpc it keeps one persistent Gateway RPC session instead. For a long-running terminal tail,
the lightweight wednesdayai-logs bin does the same local follow with a small, flat memory
footprint. See wednesdayai logs.
Output modes:
- TTY sessions: pretty, colorized, structured log lines.
- Non-TTY sessions: plain text.
--json: line-delimited JSON (one log event per line).--plain: force plain text in TTY sessions.--no-color: disable ANSI colors.
type-tagged objects:
meta: stream metadata (file, cursor, size, andsource:fileorgateway)log: parsed log entrynotice: truncation / rotation hintsraw: unparsed log line
--rpc), the CLI prints a short hint to run:
Control UI (web)
The Control UI’s Logs tab tails the same file usinglogs.tail.
See /web/control-ui-v2 for how to open it.
Channel-only logs
To filter channel activity (WhatsApp/Telegram/etc), use:Log formats
File logs (JSONL)
Each line in the log file is a JSON object. The CLI and Control UI parse these entries to render structured output (time, level, subsystem, message).Rotation and retention
The main Gateway file log rotates before an append would push the active file overlogging.maxFileBytes. Defaults are:
logging.maxFileBytes:10485760bytes (10 MB)logging.maxBackups:5- Backup compression: gzip
.1.gz, older backups move
to higher numbers, and backups beyond the retention count are deleted. If
logging.maxBackups is 0, WednesdayAI truncates the active file at rotation
time and keeps no backup.
Other durable diagnostic and audit logs use the same shared file-log controls
through logging.fileLogs. These controls do not apply to session transcripts
or conversation storage.
Console output
Console logs are TTY-aware and formatted for readability:- Subsystem prefixes (e.g.
gateway/channels/whatsapp) - Level coloring (info/warn/error)
- Optional compact or JSON mode
logging.consoleStyle.
Configuring logging
All logging configuration lives underlogging in ~/.openclaw/openclaw.json.
Log levels
logging.level: file logs (JSONL) level.logging.consoleLevel: console verbosity level.
silent → fatal → error → warn → info (default) → debug → trace
Setting a level records that level and everything less verbose than it, so debug also
records info, warn, error and fatal. silent disables the sink entirely. Invalid values
are ignored with a warning and the default is kept.
You can override both via the OPENCLAW_LOG_LEVEL environment variable (e.g. OPENCLAW_LOG_LEVEL=debug). The env var takes precedence over the config file, so you can raise verbosity for a single run without editing openclaw.json. You can also pass the global CLI option --log-level <level> (for example, wednesdayai --log-level debug gateway run), which overrides the environment variable for that command.
--verbose only affects console output; it does not change file log levels.
File-log limits
Uselogging.fileLogs.defaults to set a global budget for durable file logs
outside sessions and conversations. Use logging.fileLogs.profiles when one log
needs a tighter or looser policy.
primary: the main Gateway log.cron.run: cron run JSONL logs.config.audit: config write audit records.hooks.command: bundled command logger output.agents.rawStream: raw-stream diagnostic JSONL.agents.modelPayload: model payload debug log JSONL (off by default; see below).agents.cacheTrace: cache-trace debug JSONL.extensions.learningCore: learning-core JSONL records.extensions.voiceCall.calls: voice-call legacy call records.
cron.run, extensions.learningCore, and
extensions.voiceCall.calls, the file profile field is ignored so readers and
recovery paths keep using the canonical log path. Tune their size, backup,
compression, queue, and line-retention fields instead.
Console styles
logging.consoleStyle:
pretty: human-friendly, colored, with timestamps.compact: tighter output (best for long sessions).json: JSON per line (for log processors).
pretty and compact, structured metadata passed with a log call is appended as a
dimmed key=value suffix after the message (the same fields the json style and JSONL
file log already include). Operators who read console or journald output can see those
fields without switching styles. Formatting rules, truncation, and omitted values are
documented in Gateway logging.
On json, each line is one object carrying time, level, subsystem, and message
alongside the metadata fields. Those four envelope fields always win: metadata using the
same key cannot overwrite what a log shipper routes on.
Redaction
Tool summaries can redact sensitive tokens before they hit the console:logging.redactSensitive:off|tools(default:tools)logging.redactPatterns: list of regex strings to override the default set
Authorization: Bearer headers, URL userinfo passwords (postgres://user:pass@host/db — only the
password is masked; scheme, user and host survive), PEM private-key blocks, and common token
prefixes (sk-…, ghp_…, Telegram bot tokens, and similar).
Redaction affects console output only and does not alter file logs.
Model payload logging (debug escape hatch)
diagnostics.modelPayloadLog is an off-by-default, bounded, rotating local capture of model
request payloads, written through logging.fileLogs (profile agents.modelPayload).
It is a short-lived debugging tool, not a telemetry path. For tokens, cost, latency, and spans use
diagnostics OpenTelemetry / Langfuse (below) — those are the supported, production-grade signals.
Enable payload logging only while reproducing a provider-side payload bug, then turn it off.
Safe by default, in layers:
- Off unless
enabled: true. With no config and no env var, nothing is written and no file is created. - When
enabled: true, only provider-neutral usage summaries (tokens, latency, provider/model) are written, never message content. - Raw request bodies require a second explicit flag,
rawPayloads: true. Even then content is redacted via the shared redactor, and sections are stripped unlesscaptureContentopts them in (the system prompt stays off even whencaptureContentistrue).
usageSummaries(defaulttrue): write usage/cost/latency summary events when enabled.rawPayloads(defaultfalse): the separate opt-in for full request bodies.captureContent(defaultfalse):truefor a default set, or an object to gate per type (inputMessages,outputMessages,toolInputs,toolOutputs,systemPrompt,toolDefinitions).providers(default all): optional allowlist; when non-empty, only listed providers are logged.file(default$OPENCLAW_STATE_DIR/logs/model-payload.jsonl): JSONL output path.maxQueuedBytes: hard cap on in-memory queued bytes before entries are dropped (with a throttled warning). Rotation, retention, and compression follow theagents.modelPayloadprofile and thelogging.fileLogscontrols above.
OPENCLAW_MODEL_PAYLOAD_LOG and
OPENCLAW_MODEL_PAYLOAD_LOG_FILE. The older OPENCLAW_ANTHROPIC_PAYLOAD_LOG[_FILE] names are
deprecated aliases, still honoured with a one-time deprecation warning.
Diagnostics + OpenTelemetry
Diagnostics are structured, machine-readable events for model runs and message-flow telemetry (webhooks, queueing, session state). They do not replace logs; they exist to feed metrics, traces, and other exporters. Diagnostics events are emitted in-process, but exporters only attach when diagnostics + the exporter plugin are enabled.OpenTelemetry vs OTLP
- OpenTelemetry (OTel): the data model + SDKs for traces, metrics, and logs.
- OTLP: the wire protocol used to export OTel data to a collector/backend.
- WednesdayAI exports via OTLP/HTTP (protobuf) today.
Signals exported
- Metrics: counters + histograms (token usage, message flow, queueing).
- Traces: spans for model usage + webhook/message processing.
- Logs: exported over OTLP when
diagnostics.otel.logsis enabled. Log volume can be high; keeplogging.leveland exporter filters in mind.
Diagnostic event catalog
Model usage:model.usage: tokens, cost, duration, context, provider/model/channel, session ids.
webhook.received: webhook ingress per channel.webhook.processed: webhook handled + duration.webhook.error: webhook handler errors.message.queued: message enqueued for processing.message.processed: outcome + duration + optional error.
queue.lane.enqueue: command queue lane enqueue + depth/active/queued counts.queue.lane.dequeue: command queue lane dequeue + wait time and depth/active/queued counts.queue.lane.task.completed: command queue lane task completion with duration and outcome.session.state: session state transition + reason.session.stuck: session stuck warning + age.run.attempt: run retry/attempt metadata.diagnostic.heartbeat: aggregate counters (webhooks/queue/session).
sqlite_ctx_worker.restart: SQLite session-storage worker crash, respawn, or give-up (reason:crash|respawn|give_up, plusmessage). In-processonDiagnosticEventlisteners (plugin SDK) receive this event; the bundleddiagnostics-otelexporter records it as theopenclaw.sqlite_ctx_worker.restartcounter, tagged withreason.
taskDiagnostic / dreamDiagnostic):
task.started/task.completed/task.failed/task.preemptedtask.budget_deferred/task.budget_ceiling_reached/task.preemption_ceiling_reachedtask.boundary_violation/task.pipeline_disabled/task.idle_monitor_disabledtask.plugin_pass_rejected/task.runlog_retentiontask.recovery_exhausted— boot recovery moved a run todlq(attempt exhaust or age stall)dream.cycle_started/dream.cycle_completed/dream.pass_completed/dream.stage_skippeddream.contemplation_unconfigured/dream.contemplation_skipped/dream.contemplation_faileddream.rate_capped/dream.jsonl_parse_skipped/dream.fetch_faileddream.crystallisation_candidate
Enable diagnostics (no exporter)
Use this if you want diagnostics events available to plugins or custom sinks:Diagnostics flags (targeted logs)
Use flags to turn on extra, targeted debug logs without raisinglogging.level.
Flags are case-insensitive and support wildcards (e.g. telegram.* or *).
- Flag logs go to the standard log file (same as
logging.file). - Output is still redacted according to
logging.redactSensitive. - Full guide: /diagnostics/flags.
Export to OpenTelemetry
Diagnostics can be exported via thediagnostics-otel plugin (OTLP/HTTP). This
works with any OpenTelemetry collector/backend that accepts OTLP/HTTP.
- You can also enable the plugin with
wednesdayai plugins enable diagnostics-otel. protocolcurrently supportshttp/protobufonly.grpcis ignored.- Metrics include token usage, cost, context size, run duration, and message-flow counters/histograms (webhooks, queueing, session state, queue depth/wait, lane active/queued/duration pressure).
- Traces/metrics can be toggled with
traces/metrics(default: on). Traces include model usage spans plus webhook/message processing spans when enabled. - Set
headerswhen your collector requires auth. - Environment variables supported:
OTEL_EXPORTER_OTLP_ENDPOINT,OTEL_EXPORTER_OTLP_TRACES_ENDPOINT,OTEL_EXPORTER_OTLP_METRICS_ENDPOINT,OTEL_EXPORTER_OTLP_LOGS_ENDPOINT,OTEL_SERVICE_NAME,OTEL_EXPORTER_OTLP_PROTOCOL, andOTEL_SDK_DISABLED.
Exported metrics (names + types)
Model usage:openclaw.tokens(counter, attrs:openclaw.token,openclaw.channel,openclaw.provider,openclaw.model)openclaw.cost.usd(counter, attrs:openclaw.channel,openclaw.provider,openclaw.model)openclaw.run.duration_ms(histogram, attrs:openclaw.channel,openclaw.provider,openclaw.model)openclaw.context.tokens(histogram, attrs:openclaw.context,openclaw.channel,openclaw.provider,openclaw.model)
openclaw.webhook.received(counter, attrs:openclaw.channel,openclaw.webhook)openclaw.webhook.error(counter, attrs:openclaw.channel,openclaw.webhook)openclaw.webhook.duration_ms(histogram, attrs:openclaw.channel,openclaw.webhook)openclaw.message.queued(counter, attrs:openclaw.channel,openclaw.source)openclaw.message.processed(counter, attrs:openclaw.channel,openclaw.outcome)openclaw.message.duration_ms(histogram, attrs:openclaw.channel,openclaw.outcome)
openclaw.queue.lane.enqueue(counter, attrs:openclaw.lane)openclaw.queue.lane.dequeue(counter, attrs:openclaw.lane)openclaw.queue.lane.task.completed(counter, attrs:openclaw.lane,openclaw.outcome)openclaw.queue.lane.active(histogram, attrs:openclaw.lane)openclaw.queue.lane.queued(histogram, attrs:openclaw.lane)openclaw.queue.lane.duration_ms(histogram, attrs:openclaw.lane,openclaw.outcome)openclaw.queue.depth(histogram, attrs:openclaw.laneoropenclaw.channel=heartbeat)openclaw.queue.wait_ms(histogram, attrs:openclaw.lane)openclaw.session.state(counter, attrs:openclaw.state,openclaw.reason)openclaw.session.stuck(counter, attrs:openclaw.state)openclaw.session.stuck_age_ms(histogram, attrs:openclaw.state)openclaw.run.attempt(counter, attrs:openclaw.attempt)
stt-tts Voice mode; realtime engine sessions are not recorded
in this ring; attrs: provider as <stt>:<tts>, never transcript text or audio):
openclaw.talk.turn.end_of_turn_ms,openclaw.talk.turn.pre_model_ms,openclaw.talk.turn.model_first_token_ms,openclaw.talk.turn.tts_first_byte_ms,openclaw.talk.turn.gateway_added_ms,openclaw.talk.turn.stop_to_first_audio_ms,openclaw.talk.turn.barge_in_stop_ms(histograms, ms; a stage is recorded only when the turn reached it)openclaw.talk.turn.false_interruption,openclaw.talk.turn.confirmed_interruption,openclaw.talk.turn.interruption_candidate(counters)
Exported spans (names + key attributes)
openclaw.model.usageopenclaw.channel,openclaw.provider,openclaw.modelopenclaw.sessionKey,openclaw.sessionIdopenclaw.tokens.*(input/output/cache_read/cache_write/total)
openclaw.webhook.processedopenclaw.channel,openclaw.webhook,openclaw.chatId
openclaw.webhook.erroropenclaw.channel,openclaw.webhook,openclaw.chatId,openclaw.error
openclaw.message.processedopenclaw.channel,openclaw.outcome,openclaw.chatId,openclaw.messageId,openclaw.sessionKey,openclaw.sessionId,openclaw.reason
openclaw.session.stuckopenclaw.state,openclaw.ageMs,openclaw.queueDepth,openclaw.sessionKey,openclaw.sessionId
openclaw.task.*/openclaw.dream.*(one span per diagnostic type)- Task:
openclaw.task.action,.id,.run_id,.stage,.attempts,.reason,.error,.duration_ms - Dream:
openclaw.dream.action,.cycle_fingerprint,.stage,.pass_id,.reason,.cap - Error status on
task.failed,task.recovery_exhausted,task.boundary_violation,dream.contemplation_failed,dream.fetch_failed - Full tables: Task and dream spans
- Task:
Sampling + flushing
- Trace sampling:
diagnostics.otel.sampleRate(0.0–1.0, root spans only). - Metric export interval:
diagnostics.otel.flushIntervalMs(min 1000ms).
Protocol notes
- OTLP/HTTP endpoints can be set via
diagnostics.otel.endpointorOTEL_EXPORTER_OTLP_ENDPOINT. - Per-signal OTLP/HTTP endpoints can be set with
diagnostics.otel.tracesEndpoint,diagnostics.otel.metricsEndpoint, anddiagnostics.otel.logsEndpoint; matchingOTEL_EXPORTER_OTLP_*_ENDPOINTvariables can also override signal routing. - If the endpoint already contains
/v1/tracesor/v1/metrics, it is used as-is. - If the endpoint already contains
/v1/logs, it is used as-is for logs. diagnostics.otel.logsenables OTLP log export for the main logger output.- At gateway startup,
diagnostics-otellogs a redacted exporter summary with resolved signal URLs, header presence,OTEL_SDK_DISABLED, preloaded SDK state, and whether the NodeSDK started. Use that line to distinguish “no traces configured” from “traces routed somewhere else.”
Log export behavior
- OTLP logs use the same structured records written to
logging.file. - Respect
logging.level(file log level). Console redaction does not apply to OTLP logs. - High-volume installs should prefer OTLP collector sampling/filtering.
Troubleshooting tips
- Gateway not reachable? Run
wednesdayai doctorfirst. - Logs empty? Check that the Gateway is running and writing to the file path
in
logging.file. - Need more detail? Set
logging.leveltodebugortraceand retry. - Set
debugbut the file log looks no more detailed? Upgrade to the latest release. In releases up to and including 0.4.10 the levels more verbose thaninfowere applied inverted, sodebugbehaved aswarnandtraceaserror, and the extra records were discarded on the way to the file.infowas unaffected, which is why the default always looked correct.