[BUG] Cron Job Silent Failures — 61% of Scheduled Tasks Fail Silently

[Da-Mikey]

Summary

14 of 23 session dump files (61%) are from cron job executions, and every single one records max_retries_exhausted with a provider error (timeout/connection). The cron jobs run on schedule but silently fail — no notification reaches the user, no partial work is preserved, and the failure is invisible without auditing session dumps.

Evidence

• 14/14 cron dumps are failure states
• All with failure_category: "timeout" or network errors
• Affected jobs: evolution-introspection, evolution-research, evolution-funnel
• The entire self-improvement pipeline has not completed successfully in multiple days
• Cron system reports jobs as "ran" because the agent was invoked — no outcome tracking

Root Cause

Cron jobs execute the agent but have no post-run outcome verification. If the agent fails mid-conversation, no failure signal is emitted — the job appears to have "run" from the scheduler perspective. No structured failure log exists.

Recommended Fix

1. Add mandatory cron outcome report step: after max_retries_exhausted, write a structured failure JSON to a known path
2. Add cron watchdog that alerts if >=3 consecutive runs of any job produce no deliverable output
3. Log failure_category and retry_count to a cron-failures.jsonl that subsequent runs can consume
4. Consider a "last_successful_run" TTL per job that triggers alert if exceeded

Generated by evolution-introspection cron job — abstracted from local session analysis, no raw content or PII exposed.

[Lexus2016]

Blocked: dead code / missing integration — PR #475 added cron failure persistence (good) and a build_cron_failure_digest() helper, but that helper is never invoked by any real user-interaction path, so the failure digest does not actually surface to the user. The issue's core goal is to make silent cron failures visible to the operator, not just store them on disk.

To fix and re-open a PR:

1. Wire build_cron_failure_digest() into the actual user interaction entry point (e.g., run_agent.py / CLI session start, or the gateway message handler that starts a non-cron turn) so that when a user next interacts with the agent, recent un-acked cron failures are prepended to the context or printed in the CLI.
2. Keep the persistence and config key as-is; add a small integration test that exercises the end-to-end path from a failed cron run to the digest appearing in a simulated user turn.
3. Ensure ack timestamps are updated only when the digest is actually delivered to the user, not merely computed.

Re-open a PR once the digest has a real call site and a test proves it reaches the user.

[Lexus2016]

Rework PR opened: #476 wires the existing cron failure digest into the CLI user turn and adds integration tests proving it reaches the model.

Closes #433.