ci: comment a run-anywhere command to try the PR template artifact (#491)

* ci: comment a run-anywhere command to try the PR template artifact

The PR Template Artifact job builds a scaffold with this PR's SDK tarballs
but leaves it in the artifacts UI with no obvious way to consume it. Add a
sticky PR comment that prints a self-contained command — gh run download of
this run's artifact, unzip, then databricks apps init --template — with the
run id, repo, artifact name, and output dir all baked in, so it can be pasted
into any folder without a repo checkout.

Reuses the existing github-script + marker-based upsert pattern (bundle-size,
breaking-change). Adds job-level pull-requests: write for the comment.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

* ci: restore id-token permission on pr-template-artifact job

Job-level permissions replace the top-level set rather than merging, so
adding pull-requests: write dropped the top-level id-token: write that
setup-jfrog-npm needs for its OIDC token (ACTIONS_ID_TOKEN_REQUEST_TOKEN
unbound). List all three at the job level.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

* ci: skip try-template comment on fork PRs

Fork PRs get a read-only GITHUB_TOKEN, so the github-script upsert would
403 and fail the whole pr-template-artifact job for an author who can't
fix it. Guard the comment step with the same head-repo check bundle-size.yml
and eval-pr-comment.yml already use.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

* ci: merge eval link and template command into one PR comment

Combine the evals-monitor link and the try-this-template command into a
single sticky comment with two delimited sections instead of two separate
comments. Add a shared upsert-pr-comment-section.cjs helper: each workflow
owns one section id and rewrites only its block, preserving the other's
content (a push reruns the template section without wiping the eval link).

The two writers are separated in time — the eval link posts instantly on PR
open, the template command posts after the ~5-min build — so concurrent
writes to the shared comment don't occur. Bundle-size and breaking-change
comments are intentionally left separate for now.

Replaces the single-purpose upsert-template-artifact-comment.cjs and
rewires eval-pr-comment.yml onto the shared helper.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

* ci: fold eval link into the template artifact comment

Drop the separate Eval PR Comment workflow and the section-merge helper;
the pr-template-artifact job now posts a single comment containing both the
evals-monitor link and the run-anywhere template command. One writer, one
comment, plain marker-based upsert — no cross-workflow coordination.

Tradeoff: the eval link now rides the ~5-min build and is skipped on
docs-only PRs (where the artifact job doesn't run and evals are moot),
instead of posting instantly on every PR open.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

* ci: give the PR-bot comment a header and parallel section titles

Add an overall '🤖 AppKit PR bot' header and a '### 🔬 Run evals' title so the
evals link matches the template section's heading instead of being a bare
blockquote.

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

---------

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Co-authored-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Author: MarioCadenas

.github/scripts/upsert-pr-bot-comment.cjs

@@ -0,0 +1,45 @@
+/**
+ * Upserts the sticky "PR bot" comment built by the pr-template-artifact job:
+ * the evals-monitor link plus the run-anywhere command to scaffold an app from
+ * this PR's template artifact.
+ *
+ * Invoked via `actions/github-script`. The comment body is read from the file
+ * at COMMENT_PATH (built by the workflow step so the run id, artifact name, and
+ * output dir are interpolated from CI context). The body carries the marker
+ * below as its first line, which we reuse to find and update an existing
+ * comment instead of posting a new one on every push.
+ */
+
+const fs = require("node:fs");
+
+const MARKER = "<!-- pr-bot -->";
+
+module.exports = async ({ github, context }) => {
+  const { owner, repo } = context.repo;
+  const issue_number = context.issue.number;
+  const body = fs.readFileSync(process.env.COMMENT_PATH, "utf-8");
+
+  const comments = await github.paginate(github.rest.issues.listComments, {
+    owner,
+    repo,
+    issue_number,
+    per_page: 100,
+  });
+  const existing = comments.find((c) => c.body?.includes(MARKER));
+
+  if (existing) {
+    await github.rest.issues.updateComment({
+      owner,
+      repo,
+      comment_id: existing.id,
+      body,
+    });
+  } else {
+    await github.rest.issues.createComment({
+      owner,
+      repo,
+      issue_number,
+      body,
+    });
+  }
+};

.github/workflows/ci.yml

@@ -171,6 +171,13 @@ jobs:
     runs-on:
       group: databricks-protected-runner-group
       labels: linux-ubuntu-latest
+    # Job-level permissions REPLACE (not merge with) the top-level set, so all
+    # three must be listed: pull-requests: write for the sticky comment step,
+    # and id-token: write which setup-jfrog-npm needs for its OIDC token.
+    permissions:
+      contents: read
+      pull-requests: write
+      id-token: write
 
     steps:
       - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
@@ -226,6 +233,55 @@ jobs:
           name: appkit-template-${{ steps.version.outputs.version }}
           path: appkit-template-${{ steps.version.outputs.version }}.zip
 
+      # Build the PR-bot comment: a link to the evals-monitor app, plus a
+      # self-contained, run-anywhere command that downloads this run's artifact
+      # and scaffolds an app from it. Every value (run id, repo, artifact name,
+      # output dir) is baked in so the reader can paste it into any folder — no
+      # repo checkout or specific cwd required.
+      - name: Build PR-bot comment
+        env:
+          VERSION: ${{ steps.version.outputs.version }}
+          RUN_ID: ${{ github.run_id }}
+          REPO: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+        run: |
+          ARTIFACT="appkit-template-${VERSION}"
+          OUT="appkit-pr-${PR_NUMBER}"
+          EVALS_URL="https://evals-monitor-6051921418418893.staging.aws.databricksapps.com/prs/appkit/${PR_NUMBER}"
+          {
+            echo "<!-- pr-bot -->"
+            echo "## 🤖 AppKit PR bot"
+            echo ""
+            echo "### 🔬 Run evals"
+            echo ""
+            echo "Start an eval for this PR from the evals-monitor app: [**Go to Evals Monitor →**](${EVALS_URL})"
+            echo ""
+            echo "### 📦 Try this PR's app template"
+            echo ""
+            echo "Scaffolds a new app from this PR's SDK build. Run it in **any** folder (requires the GitHub CLI — \`gh auth login\` — and the Databricks CLI):"
+            echo ""
+            echo '```bash'
+            echo "gh run download ${RUN_ID} -R ${REPO} -n ${ARTIFACT} -D ${OUT} \\"
+            echo "  && unzip -o \"${OUT}/${ARTIFACT}.zip\" -d \"${OUT}\" \\"
+            echo "  && databricks apps init --template \"${OUT}\""
+            echo '```'
+            echo ""
+            echo "The template pins \`@databricks/appkit\` and \`@databricks/appkit-ui\` to tarballs built from this branch, so the scaffolded app runs against this PR's code."
+          } > pr-bot-comment.md
+
+      # Fork PRs get a read-only GITHUB_TOKEN, so commenting would 403 and fail
+      # a job the author can't fix. Skip for forks — the command is still in the
+      # job log above. Matches bundle-size.yml.
+      - name: Upsert PR-bot comment
+        if: github.event.pull_request.head.repo.full_name == github.repository
+        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+        env:
+          COMMENT_PATH: pr-bot-comment.md
+        with:
+          script: |
+            const upsert = require('./.github/scripts/upsert-pr-bot-comment.cjs');
+            await upsert({ github, context });
+
   docs-build:
     name: Docs Build
     needs: detect-changes