diff --git a/README.md b/README.md new file mode 100644 index 0000000..c61d04b --- /dev/null +++ b/README.md @@ -0,0 +1,165 @@ +# urunc CI Dashboard + +A real-time dashboard for monitoring urunc CI test status. + +**Live Dashboard**: https://urunc-dev.github.io/ci-dashboard/ + +## Features + +### Dashboard UI + +- **Dual-period stats** — separate stat cards for the last 24 hours and all-time totals (success rate, runs executed, passed, failed) +- **Workflow table** — sortable by name, last run conclusion, success rate, avg duration, and weather history +- **7-run weather history** — colored dots showing the last 7 run outcomes at a glance (oldest → newest) +- **Filter chips** — instantly filter by: All, Failing, Passing, Critical, Recent Failure, Recent Run, No Runs +- **Expandable drawer** — click any workflow row to reveal: + - **Runs table** — all recent runs with conclusion, date, duration, attempt number, and direct GitHub link + - **Structured log viewer** — failure signals grouped by category with keyword highlighting + - **Per-workflow charts** — avg job duration, pass/fail per run, duration trend +- **Overview charts panel** — activated via the Charts chip: + - Overall passed vs failed + - Avg duration per workflow + - Execution time last 7 runs per workflow + - Success rate by workflow + - **Auto-refresh indicator** — Data updates every cronjob run +---------- + +## Architecture + +![Architecture](architecture.excalidraw.png) + +## Repository Layout + +``` +. +├── .github/ +│ └── workflows/ +│ └── update-stats.yml # Cronjob, commits stats.json +├── main.go # Data fetching, log analysis, stats generation +├── notify.go # GitHub issue/comment notification engine +├── config.yaml # All configuration +├── index.html # Dashboard markup +├── styles.css # Dark/light theme, table, chart, log viewer styles +├── script.js # Dashboard logic — filters, charts, drawer, log renderer +└── stats.json # Auto-generated by cronjob +``` + +---------- + +## Configuration + +All configuration lives in `config.yaml`. + +### Settings + +```yaml +settings: + source_repo: "urunc-dev/urunc" # Repo to monitor + max_runs_per_workflow: 40 # Runs to fetch per workflow + recent_runs_in_output: 40 # Runs included in stats.json +``` +### Notifications + +```yaml +notify: + enabled: true + target_repo: "your-org/your-repo" # Where issues are opened + label: "ci-failure" # Label applied to opened issues +``` + +### Log analysis + +```yaml +log_analysis: + max_signals_per_job: 40 + + noise_patterns: # Lines matching these are dropped before analysis + - "sudo process started" + - "/usr/lib/gcc/" + + categories: # Lower priority number = more important + - name: "Crash / Timeout" + priority: 1 + patterns: ["panic:", "test timed out", "deadlock"] + + - name: "Network Failure" + priority: 2 + patterns: ["connection reset by peer", "i/o timeout"] + + - name: "Test Failure" + priority: 3 + patterns: ["--- fail:", "expected", "received"] + + - name: "Build Failure" + priority: 4 + patterns: ["make: ***", "##[error]", "compilation terminated"] + + - name: "Fatal Runtime Error" + priority: 5 + patterns: ["level=fatal", "fatal: "] +``` + +The frontend log viewer maps these category names to colored badges automatically (`Crash / Timeout` → red, `Network Failure` → blue, `Test Failure` → orange, `Build Failure` → purple, `Fatal Runtime Error` → red). + +### Workflows + +```yaml +workflows: + - name: "ci" + description: "Main CI workflow" + critical: true # critical=true workflows trigger issue notifications + # and show a red "critical" badge in the dashboard + - name: "scorecard" + description: "Security scorecard" + critical: false # monitored in dashboard, no issue opened on failure +``` +---------- + +## How It Works + +1. **Github cronjob** fetches `workflows_raw.json` and `runs_raw.json` from the GitHub API using `gh CLI`, then progress with go app. +2. For each configured workflow, the Go pipeline: + - Fetches job-level data for recent runs + - For failed jobs, downloads and time-windows the raw log, strips noise, categorises signals by priority + - Falls back to check-run annotations if the log has expired + - Builds a `WorkflowSummary` with weather history, failure rate, avg duration, and log snippets +3. Writes `stats.json` and commits it back to the repo +4. For each `critical: true` workflow with a recent failure, the notifier opens or updates a GitHub issue +5. GitHub Pages serves `index.html` which fetches `stats.json` and renders the dashboard +---------- + + +## Local Development + +```bash +# Fetch raw data (requires gh CLI and GITHUB_TOKEN in environment) +export GITHUB_TOKEN=your_token + +gh api repos/urunc-dev/urunc/actions/workflows \ + --paginate > workflows_raw.json + +# Fetch runs for a specific workflow ID +gh api "repos/urunc-dev/urunc/actions/workflows//runs?per_page=40" \ + > runs_raw.json + +# Run the pipeline +go run . + +# Preview the dashboard +python3 -m http.server 8080 +open http://localhost:8080 +``` +---------- + + +## Contributing + +1. **Add new tests/modify failure/notify**: Edit `config.yaml` and add desirable configurations +2. **UI changes**: Modify `index.html`, `style.css`, or `app.js` +3. **Cronjob data processing**: Modify `github/workflows/updates.yaml` + +---------- + +## License + +Apache 2.0 - See [LICENSE](LICENSE) \ No newline at end of file diff --git a/architecture.excalidraw.png b/architecture.excalidraw.png new file mode 100644 index 0000000..944bca8 Binary files /dev/null and b/architecture.excalidraw.png differ