Observability

Find what your workspace ran, why it failed, what it cost, and what is still waiting on a human.

A CLI, MCP, or web call that changes workspace state, spends, or reaches a provider records an operation event; so does every provider API request and durable run. A dry run records one too — it is a real call with a real outcome, just without spend. Plain reads (listing tables, listing runs) do not. Observability is the read-only lens over that record, workspace-wide and correlated by trace id. A run page answers "what happened inside this one run"; this answers "what happened in my workspace".

Start here when something did not happen

oxygen observability runs --status failed --json       # which runs broke
oxygen observability events --status error --json      # which calls broke
oxygen observability approvals --json                  # what is blocked on a person

All three are list-only. Each run and approval item carries decide_hint (the exact command that resolves it), hint_kind, and web_url. Deciding or retrying always routes back through the primitive that owns the work.

The web feed

Sidebar Knowledge → Observability, route /observability, page header Operation events. No beta toggle — every workspace has it.

Three rows of filter pills: Status (All, Success, Error, Queued, Skipped), Type (All, Operations, Provider calls), Source (All, CLI, Web, MCP, Provider). The feed lists Time, Operation, Status, Duration. Click a row to expand it: Source, Command, Route, Method, Tool, Provider, Operation, Trace, Run — a provider call swaps Route for Path and adds Credential — plus Credits est. and Credits actual when recorded, over two JSON panels (input/output for a provider call, details/result for an operation). Errors print their code and message on the row itself.

Filters are URL state, so responses deep-link straight into the view: /observability?status=error&source=provider, ?trace_id=<id>, ?run_id=<id>.

The web page shows the event feed only. The runs and approvals lenses are CLI and MCP today.

Copy debug prompt

When anything failed recently, a banner appears above the feed ("3 failed runs · 12 recent failed events") with a Copy debug prompt button. It copies the failed runs and events plus the Oxygen commands that investigate them. Paste it into Claude Code, Claude Desktop, Codex, or any agent holding your CLI or MCP server.

Filter values

Command	Flag	Values
`observability events`	`--status`	`success`, `error`, `completed`, `failed`, `completed_with_errors`, `queued`, `skipped`, `blocked`
`observability events`	`--type`	`operation`, `provider_request`
`observability events`	`--source`	`cli`, `web`, `mcp`, `provider`
`observability runs`	`--status`	`running`, `queued`, `waiting_approval`, `failed`, `completed`, `canceled` (comma-separated; defaults to active + failed)
`observability runs`	`--source`	`workflow`, `table_run`
`observability approvals`	`--source`	`workflow`, `publishing`, `message_review`, `inbox_draft`, `collab_request`

Anything outside those values is rejected with invalid_request, so a typo fails loudly instead of returning an empty, healthy-looking list.

runs and approvals return at most --limit items per source (default 25, max 100). A capped source is marked sources[].capped in the JSON, and printed on stderr when you omit --json — raise --limit or narrow --source.

Drill into one failure

oxygen observability events --run-id <run-id> --json — every operation and provider call belonging to that run.
oxygen runs get <run-id> --json — status, totals, event stream.
oxygen table-runs items <run-id> --status failed --json, then oxygen table-runs provider-summary <run-id> --json for provider attempts and credit capture/release.
oxygen cells inspect <table> <row-id> <column> --json — the value one item produced and where it came from.
oxygen workflows failures --json — failed workflow runs and trigger-scheduler failures.
Nothing recorded at all? oxygen worker queue-stats --json and oxygen worker failures --json show whether the work was ever claimed.

Every CLI and MCP failure carries its trace id in error.details.trace_id. oxygen observability events --trace-id <id> --json reassembles that one request end to end.

What it cost

Operation and provider events carry per-call creditsEstimate and creditsActual. The runs lens reports cost.credits as null — per-run credit rollups are not aggregated yet. For run-level spend use oxygen table-runs provider-summary <run-id> --json, and oxygen billing usage --meter credits --json for the ledger.

Retrying

A retry re-executes the failed items and spends credits again. Read the item error first. Items with an unconfirmed external effect stay blocked until you verify the destination and pass --approved-effect-unknown — see Runs.

MCP

oxygen_observability_events, oxygen_observability_runs, and oxygen_observability_approvals take the same filters and return widgets. They are not in the default tool list a fresh client loads — reconnect with ?toolset=workflows (or ?toolset=full), or find them with oxygen_capabilities_search.

Runs — one run in depth: items, modes, retries.
Approvals — how a pending decision is resolved.
Pricing reference — the ledger categories and transaction types behind the numbers above.

On this page