Add a review-before-send Report a problem diagnostic flow

[areycruzer]

Summary

AO should add an explicit, review-before-send Report a problem flow that helps users share useful diagnostics without dumping private repo data, prompts, terminal transcripts, or secrets.

This is not a request to make telemetry broader or more automatic. The best first step is a user-controlled support/reporting flow: generate a redacted diagnostic summary, let the user preview it, then let them choose where it goes.

Why this feels worth doing now

When AO breaks, the useful evidence is scattered across places the average user will not know how to collect:

• Electron renderer errors and PostHog exception breadcrumbs
• daemon status and structured Go logs
• local SQLite telemetry events
• crash dumps under Electron user data
• ao doctor output
• current route/surface and daemon readiness state
• project/session context, which must be redacted or hashed carefully

That makes bug reports either too thin ("it broke") or too risky (users paste raw logs with local paths, repo names, prompts, tokens, branch names, issue bodies, etc.). A guided report flow would help maintainers get actionable reports while keeping users in control of what leaves their machine.

Prior art / research notes

A few mature open-source apps seem to converge on the same pattern:

• VS Code separates usage telemetry, error telemetry, and crash reporting behind explicit telemetry levels (all, error, crash, off) and documents what each level means: https://code.visualstudio.com/docs/configure/telemetry
• Firefox explains what crash reports can contain and gives users a case-by-case submission path instead of making every report opaque: https://support.mozilla.org/en-US/kb/crash-report
• Electron provides crashReporter, including local-only collection with uploadToServer: false, so desktop apps can gather crash evidence without auto-uploading it: https://www.electronjs.org/docs/latest/api/crash-reporter
• Sentry / PostHog-style tools are useful for grouping exceptions and breadcrumbs, but they still depend on data scrubbing and user-feedback context to turn an exception into an actionable bug: https://docs.sentry.io/product/user-feedback/ and https://posthog.com/docs/error-tracking/capture
• GitHub issue forms work best when the app can generate a structured, pasteable diagnostic block instead of asking users to manually find logs: https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms

The common thread: don't ship a scary "send logs" button. Give users a clear preview, safe defaults, and multiple destinations.

Current AO state

AO already has good pieces to build on:

• Renderer telemetry is documented as automatic PostHog usage/error reporting: docs/telemetry.md
• Renderer telemetry already redacts local paths and local URLs before sending: frontend/src/renderer/lib/telemetry.ts
• Renderer exceptions are captured through captureRendererException: frontend/src/renderer/lib/telemetry.ts
• Electron-owned data, including crash dumps, is already pinned under ~/.ao/electron: frontend/src/main.ts
• The daemon has local/remote telemetry sink wiring: backend/internal/daemon/telemetry_wiring.go
• Backend 5xx and panic paths already produce structured fingerprints instead of raw payload dumps: backend/internal/httpd/log.go, backend/internal/httpd/recover.go

The missing piece is a user-facing bridge from "something went wrong" to "here is a safe, useful report I can share".

Proposed product flow

Add Report a problem in a visible place such as:

• Help menu / app menu
• Settings
• error boundary screen after a renderer crash
• daemon error/recovery UI

The dialog could ask:

• What are you reporting?
  • Bug
  • Feature request
  • General feedback
  • Question / setup help
• What happened?
• What did you expect?
• Optional repro steps
• Optional screenshot attachment
• Include diagnostics? default: on
• Include recent logs? default: off or limited preview
• Include crash dump metadata? default: off

Then offer destinations:

• Copy GitHub issue body
• Open GitHub issue with a prefilled body
• Copy Discord-ready summary
• Copy email-ready report
• Export support bundle as a local .zip

That lets the same flow support maintainers, Discord triage, email support, and contributors filing a bug/feature/feedback issue.

Safe diagnostic bundle: include by default

A first version could include only low-risk structured data:

• AO app version and build mode
• OS / arch
• install source if known
• Electron/Node/runtime versions where easy
• daemon status (ready, stopped, starting, error code/message)
• current UI route surface, not the full local route if it contains IDs
• hashed project/session IDs
• recent telemetry event names, levels, fingerprints, and timestamps
• recent typed API error codes and request IDs
• ao doctor summary, redacted
• last N daemon log lines after redaction, with a preview gate
• renderer error name/surface/source, not full user data

Do not include by default

These should require explicit opt-in or stay excluded entirely:

• repository file contents
• prompts or agent messages
• full terminal transcript
• issue bodies / PR review bodies
• API keys, env vars, tokens, cookies
• raw local paths
• raw project names if they came from local folders
• raw crash minidumps
• full SQLite database
• full daemon log history

Acceptance criteria for a first useful PR

A narrow first PR could be:

• Add a Report a problem entry point in Settings or Help
• Generate a redacted diagnostic object locally
• Show a preview before anything is copied/exported
• Provide buttons for:
  • Copy GitHub issue body
  • Copy Discord summary
  • Export local diagnostic .json or .zip
• Reuse the existing telemetry sanitization rules where possible
• Add tests for path/token/local URL redaction
• Document exactly what is included and excluded

A later PR can decide whether to add direct upload, direct GitHub issue creation, richer crash dump support, or app-menu integration.

Why this should not be automatic upload first

AO operates directly around source code, local terminals, worktrees, agents, GitHub/Linear/Jira context, and credentials. Even with good sanitizers, an automatic log uploader would be harder to trust and easier to reject in review.

A preview-first local export flow is safer, easier to merge, and still immediately useful for maintainers.

Duplicate / overlap check

I checked open and closed issues/PRs for diagnostic bundle, support bundle, report problem, report issue, telemetry feedback, crash logs, and related terms. I did not find a duplicate.

Nearby but different:

• [Feature]: Add GitHub issue templates for bug reports and feature requests #2210 is about GitHub issue templates. This proposal is about an in-app report/diagnostic bundle flow. They would complement each other: issue templates define the shape, AO fills the useful diagnostic block.

Suggested implementation areas

Likely frontend/Electron first:

• frontend/src/main.ts / preload IPC for diagnostic collection
• frontend/src/renderer/lib/telemetry.ts sanitizers, or a shared redaction helper extracted from it
• Settings/help UI for the report dialog
• optional backend endpoint later if daemon-local telemetry/log summaries should be fetched through the daemon instead of Electron

Likely backend support later:

• expose a safe diagnostic summary endpoint from the daemon
• list recent local telemetry events with payload allowlisting
• expose ao doctor --json style diagnostics for the desktop flow

Priority

P2 / medium enhancement.

This is not blocking core agent workflows, but it would make every future bug report, Discord triage, email support request, and contributor PR much easier to handle safely.