Skip to content

Commit 39342f8

Browse files
committed
feat: add /codeboarding-feedback command to capture PR feedback via PostHog
On an issue_comment whose first word is /codeboarding-feedback, the guard forwards the comment text and PR context to PostHog as an explicit codeboarding_feedback_submitted event, reacts to the comment, and exits before any checkout, engine setup, or LLM-key handling — so feedback never runs the analysis path. - scripts/submit_feedback.py: stdlib-only sender. Strips the command token (newlines preserved), caps at feedback_max_chars, sends one event, swallows all send failures, and never prints the feedback body. Mirrors Core's PostHog key/host defaults and DO_NOT_TRACK / CODEBOARDING_TELEMETRY opt-outs. - action.yml: feedback_command + feedback_max_chars inputs; feedback branch and env in the guard; a 👍 Acknowledge feedback step gated on feedback_received. - README: Feedback command subsection, Inputs rows, and opt-out docs that state plainly this is explicit user feedback (text + PR context), not anonymous telemetry. - tests/test_submit_feedback.py: 26 stdlib unittest cases (extraction, multiline, truncation, opt-outs, host override, JSON shape, no-leak).
1 parent 3168018 commit 39342f8

4 files changed

Lines changed: 452 additions & 0 deletions

File tree

README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,29 @@ This table mirrors the engine and may lag it. The source of truth is the engine'
154154

155155
The command needs the `issue_comment` trigger and runs from your default branch (a GitHub rule), so it only works once the workflow is merged there. On-demand runs on fork PRs are refused, so fork code is never analyzed with your secrets.
156156

157+
### Feedback command
158+
159+
In review workflows that include `issue_comment`, anyone whose comment reaches the action can send product feedback with:
160+
161+
```text
162+
/codeboarding-feedback <message>
163+
```
164+
165+
The action sends the message text and PR context (repository, PR number, comment URL, commenter) to CodeBoarding via PostHog, then reacts 👍 to the comment. This is **explicit, user-submitted feedback, not anonymous telemetry** — unlike [Core's telemetry](https://github.com/CodeBoarding/CodeBoarding/blob/main/TELEMETRY.md), it intentionally includes the text you wrote and who wrote it. It runs no analysis, checks out no code, and sends no source. The branch exits before engine setup, so a feedback comment is cheap and never starts an LLM job. Change the keyword via `feedback_command`, or cap the text length with `feedback_max_chars` (default `4000`).
166+
167+
Because the feedback path runs no PR-head code, it isn't restricted to trusted collaborators the way the `/codeboarding` analysis command is — your workflow's own `if:` condition decides who can reach it. The quick-start workflow already restricts comment triggers to `OWNER`/`MEMBER`/`COLLABORATOR`; keep that condition if you only want trusted feedback. The default `startsWith(github.event.comment.body, '/codeboarding')` filter already matches `/codeboarding-feedback`, so no workflow change is needed to enable it.
168+
169+
Opt out by setting either in your workflow `env:` (workflow- or job-level):
170+
171+
```yaml
172+
env:
173+
CODEBOARDING_TELEMETRY: 'false' # CodeBoarding-specific switch
174+
# or the cross-tool standard:
175+
DO_NOT_TRACK: '1'
176+
```
177+
178+
When disabled, the feedback comment is acknowledged but nothing is sent. Override the destination with `CODEBOARDING_POSTHOG_KEY` / `CODEBOARDING_POSTHOG_HOST` (same names Core uses).
179+
157180
## Keep your architecture versioned (sync mode)
158181

159182
With `mode: sync`, the action analyzes the pushed commit and commits the results back to the branch (as `codeboarding[bot]`), so your architecture analysis stays versioned in git and tracks the code instead of drifting from it:
@@ -244,6 +267,8 @@ Be aware that `contents: write` is repo-wide — GitHub does not scope it to a b
244267
| `parsing_model` | both | `google/gemini-3.1-flash-lite-preview` | Parsing model. OpenRouter default shown; other providers use their own engine default. |
245268
| `comment_header` | review | `Architecture review` | Heading for the PR comment. |
246269
| `trigger_command` | review | `/codeboarding` | Slash command for trusted on-demand runs. |
270+
| `feedback_command` | review | `/codeboarding-feedback` | Slash command that sends a PR comment's text as [explicit product feedback](#feedback-command) to CodeBoarding via PostHog. Runs no analysis; opt out with `CODEBOARDING_TELEMETRY=false` / `DO_NOT_TRACK=1`. |
271+
| `feedback_max_chars` | review | `4000` | Max feedback characters sent from `feedback_command`; longer text is truncated and flagged. |
247272
| `cta_base_url` | review | empty | Click-proxy base URL: deep-links the editor link into VS Code/Cursor and adds a "get the extension" link (tracks owner/repo/pr). Empty links to the extension listing instead (GitHub strips `vscode:`/`cursor:` from comments). |
248273
| `webview_base_url` | review | `https://app.codeboarding.org` | Hosted webview base URL. The PR comment adds an "explore in browser" link to this PR's head-vs-base diff. Needs `commit_head_analysis` (same-repo PRs only); omitted on forks. Set empty to disable. |
249274
| `commit_head_analysis` | review | `true` | Commit the generated head `.codeboarding/analysis.json` (+ health report) to the PR branch so the webview can read it at the head SHA. Same-repo PRs only (the token is read-only on forks). |

action.yml

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,14 @@ inputs:
7070
description: 'Review mode: slash-command that triggers the action from a PR comment (issue_comment event). A comment whose first word is this runs the diagram on-demand.'
7171
required: false
7272
default: '/codeboarding'
73+
feedback_command:
74+
description: 'Review mode: slash-command for submitting explicit feedback to CodeBoarding. The command and following text are sent to CodeBoarding via PostHog (not anonymous telemetry, and no analysis runs). Opt out with CODEBOARDING_TELEMETRY=false or DO_NOT_TRACK=1.'
75+
required: false
76+
default: '/codeboarding-feedback'
77+
feedback_max_chars:
78+
description: 'Review mode: maximum number of feedback characters sent from /codeboarding-feedback (the text is truncated past this).'
79+
required: false
80+
default: '4000'
7381
mode:
7482
description: 'What the action does. "review" (default): post a Mermaid architecture-diff comment on the PR (pull_request / issue_comment events). "sync": analyze on push and commit the architecture (analysis.json + rendered docs) to target_branch, keeping it versioned and current (the baseline review mode diffs against). Events: push / workflow_dispatch / schedule. Run the two modes from separate workflow files with least-privilege permissions each.'
7583
required: false
@@ -142,6 +150,26 @@ runs:
142150
COMMENT_BODY: ${{ github.event.comment.body }}
143151
AUTHOR_ASSOC: ${{ github.event.comment.author_association }}
144152
TRIGGER: ${{ inputs.trigger_command }}
153+
# Feedback path (/codeboarding-feedback): a lightweight branch that sends
154+
# the comment text to PostHog and exits before any checkout/engine setup.
155+
# The script reads CODEBOARDING_POSTHOG_KEY/HOST and the opt-outs
156+
# DO_NOT_TRACK / CODEBOARDING_TELEMETRY straight from the caller's env: a
157+
# composite action inherits the calling workflow/job `env:` as real env
158+
# vars, so those need no re-mapping here. POSTHOG_KEY/HOST below are a
159+
# redundant alias the script only falls back to if the canonical vars are
160+
# unset — kept as a hint that the destination is overridable.
161+
FEEDBACK_COMMAND: ${{ inputs.feedback_command }}
162+
FEEDBACK_MAX_CHARS: ${{ inputs.feedback_max_chars }}
163+
POSTHOG_KEY: ${{ env.CODEBOARDING_POSTHOG_KEY }}
164+
POSTHOG_HOST: ${{ env.CODEBOARDING_POSTHOG_HOST }}
165+
SENDER_LOGIN: ${{ github.event.sender.login }}
166+
SENDER_ID: ${{ github.event.sender.id }}
167+
COMMENT_ID: ${{ github.event.comment.id }}
168+
COMMENT_URL: ${{ github.event.comment.html_url }}
169+
REPOSITORY_ID: ${{ github.event.repository.id }}
170+
RUN_ATTEMPT: ${{ github.run_attempt }}
171+
ACTION_PATH: ${{ github.action_path }}
172+
ACTION_REF: ${{ github.action_ref || github.sha }}
145173
EVENT: ${{ github.event_name }}
146174
REPOSITORY: ${{ github.repository }}
147175
PR_NUMBER_PULL: ${{ github.event.pull_request.number }}
@@ -243,6 +271,18 @@ runs:
243271
# word is the trigger; the payload lacks SHAs so we query the API.
244272
[ -n "$ISSUE_PR_URL" ] || skip "Comment is on a plain issue, not a PR."
245273
FIRST_WORD="$(printf '%s' "$COMMENT_BODY" | tr -d '\r' | awk 'NR==1{print $1; exit}')"
274+
# Feedback command: forward the comment text to PostHog, then stop BEFORE
275+
# the trusted-association check, checkout, engine setup, and LLM key.
276+
# Sending user-written feedback runs no PR code, so it doesn't need the
277+
# collaborator gate the analysis path does — the workflow's own `if:`
278+
# decides who can reach this step. The script swallows its own failures
279+
# and the guard has no `set -e`, so feedback can never block the action.
280+
if [ "$FIRST_WORD" = "$FEEDBACK_COMMAND" ]; then
281+
python3 "$ACTION_PATH/scripts/submit_feedback.py"
282+
echo "skip=true" >> "$GITHUB_OUTPUT"
283+
echo "feedback_received=true" >> "$GITHUB_OUTPUT"
284+
exit 0
285+
fi
246286
[ "$FIRST_WORD" = "$TRIGGER" ] || skip "Comment does not start with '$TRIGGER'."
247287
# SECURITY (pwn-request guard): issue_comment runs in the base repo WITH
248288
# secrets for ANY commenter. Only a trusted collaborator may trigger an
@@ -281,6 +321,21 @@ runs:
281321
} >> "$GITHUB_OUTPUT"
282322
echo "Resolved PR #$PR_NUMBER (base=$BASE_REPO@$BASE_SHA head=$HEAD_REPO@$HEAD_SHA) via $EVENT"
283323
324+
# Feedback exits the guard with skip=true, so the analysis "Acknowledge
325+
# command" step below (gated on skip != 'true') never fires for it; this one
326+
# reacts instead, keyed on the feedback_received flag the guard set.
327+
- name: Acknowledge feedback
328+
if: steps.guard.outputs.feedback_received == 'true'
329+
shell: bash
330+
env:
331+
GH_TOKEN: ${{ inputs.github_token }}
332+
REPOSITORY: ${{ github.repository }}
333+
COMMENT_ID: ${{ github.event.comment.id }}
334+
run: |
335+
# 👍 react to the feedback comment so the user knows it was received.
336+
gh api -X POST "repos/${REPOSITORY}/issues/comments/${COMMENT_ID}/reactions" \
337+
-f content='+1' >/dev/null 2>&1 || true
338+
284339
- name: Acknowledge command
285340
if: steps.guard.outputs.skip != 'true' && github.event_name == 'issue_comment'
286341
shell: bash

scripts/submit_feedback.py

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
"""Submit explicit user feedback (/codeboarding-feedback) to PostHog.
2+
3+
Standard-library only, on purpose: this runs in the action's guard phase, before
4+
the engine checkout and any dependency install, so it must not import third-party
5+
packages. Unlike Core's anonymous telemetry, this event intentionally carries the
6+
user-written feedback text and PR context — that difference is documented in the
7+
README. All sending failures are swallowed; feedback must never break a PR.
8+
"""
9+
10+
from __future__ import annotations
11+
12+
import json
13+
import os
14+
import sys
15+
import urllib.error
16+
import urllib.request
17+
18+
# Public PostHog ingest key — the same write-only project key Core ships.
19+
DEFAULT_POSTHOG_KEY = "phc_BQWpoXuPYQhW7mPWQcRv4yzSfuoAmh48EmXuUpeXPUB2"
20+
DEFAULT_POSTHOG_HOST = "https://us.i.posthog.com"
21+
DEFAULT_COMMAND = "/codeboarding-feedback"
22+
DEFAULT_MAX_CHARS = 4000
23+
EVENT_NAME = "codeboarding_feedback_submitted"
24+
SOURCE = "github_action_feedback"
25+
26+
27+
def telemetry_disabled(env: dict) -> bool:
28+
"""Mirror Core's opt-out: DO_NOT_TRACK or CODEBOARDING_TELEMETRY=false."""
29+
if env.get("DO_NOT_TRACK", "").strip().lower() in ("1", "true", "yes"):
30+
return True
31+
return env.get("CODEBOARDING_TELEMETRY", "true").strip().lower() == "false"
32+
33+
34+
def resolve_key(env: dict) -> str:
35+
return (env.get("CODEBOARDING_POSTHOG_KEY") or env.get("POSTHOG_KEY") or DEFAULT_POSTHOG_KEY).strip()
36+
37+
38+
def resolve_host(env: dict) -> str:
39+
host = (env.get("CODEBOARDING_POSTHOG_HOST") or env.get("POSTHOG_HOST") or DEFAULT_POSTHOG_HOST).strip()
40+
return host.rstrip("/") or DEFAULT_POSTHOG_HOST
41+
42+
43+
def resolve_command(env: dict) -> str:
44+
return (env.get("FEEDBACK_COMMAND") or "").strip() or DEFAULT_COMMAND
45+
46+
47+
def resolve_max_chars(env: dict) -> int:
48+
try:
49+
n = int((env.get("FEEDBACK_MAX_CHARS") or "").strip())
50+
except ValueError:
51+
return DEFAULT_MAX_CHARS
52+
return n if n > 0 else DEFAULT_MAX_CHARS
53+
54+
55+
def extract_feedback(comment_body: str, command: str) -> str:
56+
"""Return everything after the leading command token, newlines preserved.
57+
58+
The command is the first whitespace-delimited token of the comment. Only that
59+
one token is removed; the remainder (including any later lines) is kept
60+
verbatim, then outer whitespace is trimmed. Returns "" when the comment does
61+
not actually start with the command, or carries no text after it.
62+
"""
63+
body = (comment_body or "").replace("\r\n", "\n").replace("\r", "\n").lstrip()
64+
if not body:
65+
return ""
66+
parts = body.split(None, 1) # split once on the first run of whitespace
67+
if parts[0] != command:
68+
return ""
69+
return parts[1].strip() if len(parts) > 1 else ""
70+
71+
72+
def cap_feedback(text: str, max_chars: int) -> tuple[str, int, bool]:
73+
"""Return (capped_text, original_length, truncated)."""
74+
original_length = len(text)
75+
truncated = original_length > max_chars
76+
return (text[:max_chars] if truncated else text), original_length, truncated
77+
78+
79+
def _first(env: dict, *names: str) -> str:
80+
for name in names:
81+
value = (env.get(name) or "").strip()
82+
if value:
83+
return value
84+
return ""
85+
86+
87+
def distinct_id(env: dict) -> str:
88+
sender_id = _first(env, "SENDER_ID")
89+
if sender_id:
90+
return f"github-user:{sender_id}"
91+
return f"github-run:{_first(env, 'RUN_ID', 'GITHUB_RUN_ID')}"
92+
93+
94+
def build_properties(env: dict, command: str, feedback_text: str, feedback_length: int, truncated: bool) -> dict:
95+
props: dict = {
96+
"source": SOURCE,
97+
"command": command,
98+
"feedback_text": feedback_text,
99+
"feedback_length": feedback_length,
100+
"feedback_truncated": truncated,
101+
}
102+
optional = {
103+
"repository": _first(env, "REPOSITORY"),
104+
"repository_id": _first(env, "REPOSITORY_ID"),
105+
"pr_number": _first(env, "PR_NUMBER", "ISSUE_NUMBER"),
106+
"comment_id": _first(env, "COMMENT_ID"),
107+
"comment_url": _first(env, "COMMENT_URL"),
108+
"author_association": _first(env, "AUTHOR_ASSOC", "AUTHOR_ASSOCIATION"),
109+
"sender_login": _first(env, "SENDER_LOGIN"),
110+
"sender_id": _first(env, "SENDER_ID"),
111+
"run_id": _first(env, "RUN_ID", "GITHUB_RUN_ID"),
112+
"run_attempt": _first(env, "RUN_ATTEMPT", "GITHUB_RUN_ATTEMPT"),
113+
"action_ref": _first(env, "ACTION_REF", "GITHUB_ACTION_REF", "GITHUB_SHA"),
114+
}
115+
props.update({key: value for key, value in optional.items() if value})
116+
return props
117+
118+
119+
def build_payload(env: dict) -> dict | None:
120+
"""Build the PostHog event payload, or None when there is nothing to send."""
121+
command = resolve_command(env)
122+
feedback_text, feedback_length, truncated = cap_feedback(
123+
extract_feedback(env.get("COMMENT_BODY", ""), command), resolve_max_chars(env)
124+
)
125+
if not feedback_text:
126+
return None
127+
return {
128+
"api_key": resolve_key(env),
129+
"event": EVENT_NAME,
130+
"distinct_id": distinct_id(env),
131+
"properties": build_properties(env, command, feedback_text, feedback_length, truncated),
132+
}
133+
134+
135+
def post(payload: dict, host: str, timeout: int = 10) -> int:
136+
"""POST one event to PostHog's ingest endpoint; return the HTTP status."""
137+
request = urllib.request.Request(
138+
f"{host}/i/v0/e/",
139+
data=json.dumps(payload).encode("utf-8"),
140+
headers={"Content-Type": "application/json"},
141+
method="POST",
142+
)
143+
with urllib.request.urlopen(request, timeout=timeout) as response:
144+
return response.status
145+
146+
147+
def main(env: dict | None = None) -> int:
148+
env = os.environ if env is None else env
149+
150+
if telemetry_disabled(env):
151+
print("Feedback disabled via DO_NOT_TRACK / CODEBOARDING_TELEMETRY; not sending.")
152+
return 0
153+
154+
payload = build_payload(env)
155+
if payload is None:
156+
print("No feedback text after the command; nothing to send.")
157+
return 0
158+
if not payload["api_key"]:
159+
print("No PostHog key configured; skipping feedback send.")
160+
return 0
161+
162+
truncated = payload["properties"].get("feedback_truncated")
163+
try:
164+
status = post(payload, resolve_host(env))
165+
print(f"Feedback submitted (HTTP {status}, truncated={truncated}).")
166+
except urllib.error.HTTPError as exc:
167+
print(f"Feedback endpoint returned HTTP {exc.code}; ignoring.")
168+
except urllib.error.URLError as exc:
169+
print(f"Feedback endpoint unreachable ({type(exc.reason).__name__}); ignoring.")
170+
except Exception as exc: # never let feedback break the action
171+
print(f"Feedback send failed ({type(exc).__name__}); ignoring.")
172+
return 0
173+
174+
175+
if __name__ == "__main__":
176+
sys.exit(main())

0 commit comments

Comments
 (0)