Skip to content

Feature: verifiable dreamer run progress (pending count + per-task status) #272

Description

@canomogollon

Short description

Surface verifiable progress for on-demand /ctx-dream runs so users can track backlog draining (e.g. unclassified memories) instead of relying on the end-of-run completion summary alone.

What problem does this solve?

The Dreamer runs a suite of tasks as child sessions, but the only feedback is a summary notification after the run completes. During long-running tasks there is no way to observe:

  • how many items (memories/notes) remain to be processed in the current task;
  • whether the run is making progress or stuck;
  • which task is currently active.

Concretely, on a project with ~6.3k memories, map-memories and classify-memories abort with incomplete: N selected memories remain after the 20-minute default timeout_minutes. The message says N remains, but there is no increasing/decreasing signal across repeated on-demand runs, no per-task breakdown, and no percentage — so a user cannot tell whether repeated /ctx-dream invocations are converging or stuck on a prompt/issue (see closed bug #232 where classify-memories was silently using the wrong system prompt; a progress counter would have made that visible immediately).

Proposed solution

  1. Persist per-task progress in the shared SQLite DB (a dream_progress table or columns on dream_runs): current task name, items processed, items remaining, started/finished timestamps, and status (pending/running/done/failed) for each task of the active run.
  2. Expose it via a read-only surface: either a magic-context status CLI subcommand, a TUI sidebar section, or a Dashboard "Dream runs" progress view. The CLI is the most portable across OpenCode/Pi and trivially scriptable.
  3. Emit progress during the run: the runner writes rows as each task iterates its selected items (e.g. every batch or every N items), so a watcher can poll and see the number decrease.

Alternatives considered

  • Parsing the completion summary: insufficient — it only arrives after the run, carries no delta, and no partial info if a task aborts.
  • Log scraping (/tmp/opencode/magic-context/magic-context.log): works but is fragile, harness-specific, and untyped.
  • A status callback hook: viable for dashboards but leaves CLI users (who trigger /ctx-dream most) without a way to poll.

Area

Memory / historian / dreamer

Additional context

  • The dreamer.tasks config already supports per-task timeout_minutes (raised to 90 for map-memories/classify-memories on our side); a progress view would let users right-size these timeouts with evidence.
  • Related closed bug: "classify-memories" task uses my system prompt #232 — "classify-memories task uses my system prompt" (a progress counter surfaced the anomaly).
  • Inspiration: CI runners show live job progress; the Dreamer is effectively a background job runner and would benefit from the same observability.
  • We run a small monitoring script that queries memories WHERE classified_at IS NULL before/after each run to approximate progress; a first-class counter would replace this workaround.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions