Skip to content

ci(l1): consolidate Slack failure webhooks into a single secret#6526

Merged
ilitteri merged 1 commit intomainfrom
ci/route-failure-alerts-to-interno-channel
May 4, 2026
Merged

ci(l1): consolidate Slack failure webhooks into a single secret#6526
ilitteri merged 1 commit intomainfrom
ci/route-failure-alerts-to-interno-channel

Conversation

@avilagaston9
Copy link
Copy Markdown
Contributor

Motivation

Consolidate CI failure notifications onto a single Slack webhook.

Description

Replace ETHREX_L1_SLACK_WEBHOOK and ETHREX_L2_SLACK_WEBHOOK with ETHREX_INTERNO_SLACK_WEBHOOK in:

  • .github/workflows/common_failure_alerts.yaml — collapse the L1/L2 branching into a single webhook.
  • .github/workflows/daily_snapsync.yaml — update both jobs.
  • .github/workflows/daily_hive_report.yaml — collapse the two-webhook format() call into a single webhook.

The ETHREX_INTERNO_SLACK_WEBHOOK repo secret has been created.

Checklist

  • Updated STORE_SCHEMA_VERSION (crates/storage/lib.rs) if the PR includes breaking changes to the Store requiring a re-sync. — N/A

ETHREX_INTERNO_SLACK_WEBHOOK across the three workflows that use them.

In common_failure_alerts.yaml the L1/L2 branching in the Select Slack
webhook step is collapsed to a single line. In daily_hive_report.yaml
the two-webhook format() call is collapsed to a single webhook. In
daily_snapsync.yaml both SLACK_WEBHOOK_URL_FAILURE references are
updated.
@github-actions github-actions Bot added the L1 Ethereum client label Apr 24, 2026
@avilagaston9 avilagaston9 marked this pull request as ready for review April 24, 2026 20:25
@avilagaston9 avilagaston9 requested a review from a team as a code owner April 24, 2026 20:25
Copilot AI review requested due to automatic review settings April 24, 2026 20:25
@ethrex-project-sync ethrex-project-sync Bot moved this to In Review in ethrex_l1 Apr 24, 2026
@github-actions
Copy link
Copy Markdown

🤖 Claude Code Review


PR #6526 Review — ci(l1): consolidate Slack failure webhooks into a single secret

Overview

This PR replaces two separate Slack webhook secrets (ETHREX_L1_SLACK_WEBHOOK and ETHREX_L2_SLACK_WEBHOOK) with a single unified secret (ETHREX_INTERNO_SLACK_WEBHOOK) across three CI workflow files. It's a pure CI/infrastructure change with no production code impact.


Analysis

Correctness

  • The substitution is applied consistently across all three affected files: common_failure_alerts.yaml, daily_hive_report.yaml, and daily_snapsync.yaml.
  • In common_failure_alerts.yaml, the select_webhook step's shell logic (which branched on whether the workflow name contained "L2") is correctly collapsed — that branching logic is now obsolete since both channels receive the same webhook.
  • In daily_hive_report.yaml, the format('{0} {1}', L1_WEBHOOK, L2_WEBHOOK) call (which sent to two channels) is replaced by a single webhook. This is a behavior change: previously, both the L1 and L2 Slack channels received the daily Hive report; now only the unified channel does. Confirm this is intentional and that no stakeholders relying on the L2-channel report will be missed.
  • In daily_snapsync.yaml, both matrix jobs (lighthouse and prysm) are updated symmetrically — no inconsistency.

Security

  • No secrets are hardcoded. The migration from old secrets to the new one is the right approach.
  • The ETHREX_INTERNO_SLACK_WEBHOOK repo secret is noted as already created; verify it is scoped appropriately (repo-level is fine for this use case).
  • Old secrets (ETHREX_L1_SLACK_WEBHOOK, ETHREX_L2_SLACK_WEBHOOK) should be removed from the repository's secret store after this merges to avoid accumulating stale, potentially valid credentials.

Potential Issue — daily_hive_report.yaml dual-send behavior

The previous format('{0} {1}', ...) pattern passed a space-separated list of two webhook URLs, implying the notification script accepted multiple webhooks and sent to both. Replacing it with a single URL changes the fan-out from 2 to 1. This is likely intentional (per the PR description), but it's worth double-checking the notification script's behavior to ensure a single URL is still handled correctly with no off-by-one or parsing edge case.

Readability / Maintainability

  • Removing the L1/L2 branch in common_failure_alerts.yaml reduces dead logic and is a clear improvement.
  • The PR description is accurate and complete. No issues there.

Summary

This is a clean, low-risk CI housekeeping change. The only item worth confirming before merge:

  1. Intentional single-channel fan-out in daily_hive_report.yaml — the Hive report now goes to one channel instead of two; make sure all intended audiences are still covered by ETHREX_INTERNO_SLACK_WEBHOOK.
  2. Retire the old secrets (ETHREX_L1_SLACK_WEBHOOK, ETHREX_L2_SLACK_WEBHOOK) from the repo secret store post-merge to avoid stale credentials.

No correctness, security, or logic bugs found. Approve once Point 1 is confirmed.


Automated review by Claude (Anthropic) · sonnet · custom prompt

@github-actions
Copy link
Copy Markdown

🤖 Kimi Code Review

This PR consolidates Slack webhook configurations from separate L1/L2 channels into a single internal channel. The changes are straightforward, but there are operational considerations:

Critical: Secret Configuration

  • Ensure ETHREX_INTERNO_SLACK_WEBHOOK is configured in repository secrets before merging. If missing, all three workflows will fail at runtime when attempting to send notifications.

Behavioral Change - L2 Notification Loss

  • .github/workflows/common_failure_alerts.yaml (lines 39-44): L2-specific workflow failures previously routed to ETHREX_L2_SLACK_WEBHOOK will now all go to the unified interno channel. Confirm the L2 team no longer requires separate alerting.
  • .github/workflows/daily_hive_report.yaml (lines 256-263): Scheduled runs previously notified both L1 and L2 channels simultaneously. The new logic sends only to the interno channel. Verify this reduced notification coverage is intentional.

Code Quality

  • .github/workflows/common_failure_alerts.yaml (lines 36-44): The step name select_webhook is now misleading since it no longer performs conditional selection. Consider renaming to set_webhook or inlining the environment variable directly in subsequent steps.
  • .github/workflows/daily_hive_report.yaml (line 256): The environment variable SLACK_WEBHOOKS (plural) now receives a single webhook string instead of space-separated values. Ensure the downstream notification script handles this correctly (most Slack action implementations do).

Security

  • No issues. Secrets remain properly referenced via ${{ secrets.XXX }} without exposure in logs.

Recommendation: Add a PR comment or commit message documenting that this change intentionally removes L1/L2 notification segregation in favor of a unified channel.


Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR consolidates CI failure Slack notifications by replacing the separate L1/L2 failure webhooks with a single repository secret (ETHREX_INTERNO_SLACK_WEBHOOK) across the relevant GitHub Actions workflows.

Changes:

  • Updated Daily Snapsync workflows to use ETHREX_INTERNO_SLACK_WEBHOOK for failure notifications.
  • Simplified Daily Hive Report Slack posting to use a single webhook instead of formatting two.
  • Removed L1/L2 branching logic in common failure alerts and always selects the consolidated webhook.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
.github/workflows/daily_snapsync.yaml Switch failure Slack webhook env var to ETHREX_INTERNO_SLACK_WEBHOOK for both snapsync jobs.
.github/workflows/daily_hive_report.yaml Replace two-webhook format() expression with the consolidated webhook for scheduled runs.
.github/workflows/common_failure_alerts.yaml Remove workflow-name-based L1/L2 webhook selection and always use the consolidated webhook.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Apr 24, 2026

Greptile Summary

This PR consolidates three CI workflows from two separate Slack webhooks (ETHREX_L1_SLACK_WEBHOOK / ETHREX_L2_SLACK_WEBHOOK) to a single ETHREX_INTERNO_SLACK_WEBHOOK secret. All changes are straightforward substitutions with no logic side-effects.

Confidence Score: 5/5

Safe to merge — purely a secret-name consolidation with no behavioral changes beyond routing notifications to one channel instead of two.

All three files make targeted, correct substitutions. The only finding is a P2 style suggestion to remove a now-trivial intermediate step in common_failure_alerts.yaml.

No files require special attention.

Important Files Changed

Filename Overview
.github/workflows/common_failure_alerts.yaml Replaces L1/L2 webhook branching with a single ETHREX_INTERNO_SLACK_WEBHOOK; the intermediate select_webhook step is now a trivial pass-through.
.github/workflows/daily_hive_report.yaml Collapses two-webhook format() call into a single ETHREX_INTERNO_SLACK_WEBHOOK; the for-loop still works correctly with one webhook.
.github/workflows/daily_snapsync.yaml Updates SLACK_WEBHOOK_URL_FAILURE in both lighthouse and prysm jobs from ETHREX_L1_SLACK_WEBHOOK to ETHREX_INTERNO_SLACK_WEBHOOK.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[CI Workflow Fails] --> B{common_failure_alerts}
    B -->|before| C[Branch on L1 vs L2 name]
    C --> D[ETHREX_L1_SLACK_WEBHOOK]
    C --> E[ETHREX_L2_SLACK_WEBHOOK]
    B -->|after| F[ETHREX_INTERNO_SLACK_WEBHOOK]

    G[daily_hive_report] -->|before| H["format(L1, L2) → two webhooks"]
    G -->|after| I[ETHREX_INTERNO_SLACK_WEBHOOK]

    J[daily_snapsync lighthouse/prysm] -->|before| K[ETHREX_L1_SLACK_WEBHOOK on failure]
    J -->|after| L[ETHREX_INTERNO_SLACK_WEBHOOK on failure]

    D -.->|consolidated| F
    E -.->|consolidated| F
    H -.->|consolidated| I
    K -.->|consolidated| L
Loading
Prompt To Fix All With AI
This is a comment left during a code review.
Path: .github/workflows/common_failure_alerts.yaml
Line: 36-40

Comment:
**Intermediate step now a no-op**

The `select_webhook` step no longer branches on anything — it just echoes a constant secret. Consider removing the step and passing the secret directly to `notify_workflow_failure.sh` as an env var, which simplifies the job.

```suggestion
      - name: Post failure to Slack
        env:
          REPO: ${{ github.repository }}
          WORKFLOW_NAME: ${{ github.event.workflow_run.name }}
          CONCLUSION: ${{ github.event.workflow_run.conclusion }}
          RUN_HTML_URL: ${{ github.event.workflow_run.html_url }}
          RUN_ID: ${{ github.event.workflow_run.id }}
          HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
          FAILED_JOBS: ${{ steps.failed_jobs.outputs.names }}
          SLACK_WEBHOOK: ${{ secrets.ETHREX_INTERNO_SLACK_WEBHOOK }}
        run: |
          bash .github/scripts/notify_workflow_failure.sh "$SLACK_WEBHOOK"
```
(This also avoids routing the secret through `GITHUB_OUTPUT`.)

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "Replace ETHREX_L1_SLACK_WEBHOOK and ETHR..." | Re-trigger Greptile

Comment on lines 36 to 40
- name: Select Slack webhook
id: select_webhook
run: |
WF_NAME="${{ github.event.workflow_run.name }}"
if echo "$WF_NAME" | grep -Eiq 'L2'; then
echo "webhook=${{ secrets.ETHREX_L2_SLACK_WEBHOOK }}" >> "$GITHUB_OUTPUT"
else
echo "webhook=${{ secrets.ETHREX_L1_SLACK_WEBHOOK }}" >> "$GITHUB_OUTPUT"
fi
echo "webhook=${{ secrets.ETHREX_INTERNO_SLACK_WEBHOOK }}" >> "$GITHUB_OUTPUT"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Intermediate step now a no-op

The select_webhook step no longer branches on anything — it just echoes a constant secret. Consider removing the step and passing the secret directly to notify_workflow_failure.sh as an env var, which simplifies the job.

Suggested change
- name: Select Slack webhook
id: select_webhook
run: |
WF_NAME="${{ github.event.workflow_run.name }}"
if echo "$WF_NAME" | grep -Eiq 'L2'; then
echo "webhook=${{ secrets.ETHREX_L2_SLACK_WEBHOOK }}" >> "$GITHUB_OUTPUT"
else
echo "webhook=${{ secrets.ETHREX_L1_SLACK_WEBHOOK }}" >> "$GITHUB_OUTPUT"
fi
echo "webhook=${{ secrets.ETHREX_INTERNO_SLACK_WEBHOOK }}" >> "$GITHUB_OUTPUT"
- name: Post failure to Slack
env:
REPO: ${{ github.repository }}
WORKFLOW_NAME: ${{ github.event.workflow_run.name }}
CONCLUSION: ${{ github.event.workflow_run.conclusion }}
RUN_HTML_URL: ${{ github.event.workflow_run.html_url }}
RUN_ID: ${{ github.event.workflow_run.id }}
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
FAILED_JOBS: ${{ steps.failed_jobs.outputs.names }}
SLACK_WEBHOOK: ${{ secrets.ETHREX_INTERNO_SLACK_WEBHOOK }}
run: |
bash .github/scripts/notify_workflow_failure.sh "$SLACK_WEBHOOK"

(This also avoids routing the secret through GITHUB_OUTPUT.)

Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/common_failure_alerts.yaml
Line: 36-40

Comment:
**Intermediate step now a no-op**

The `select_webhook` step no longer branches on anything — it just echoes a constant secret. Consider removing the step and passing the secret directly to `notify_workflow_failure.sh` as an env var, which simplifies the job.

```suggestion
      - name: Post failure to Slack
        env:
          REPO: ${{ github.repository }}
          WORKFLOW_NAME: ${{ github.event.workflow_run.name }}
          CONCLUSION: ${{ github.event.workflow_run.conclusion }}
          RUN_HTML_URL: ${{ github.event.workflow_run.html_url }}
          RUN_ID: ${{ github.event.workflow_run.id }}
          HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
          FAILED_JOBS: ${{ steps.failed_jobs.outputs.names }}
          SLACK_WEBHOOK: ${{ secrets.ETHREX_INTERNO_SLACK_WEBHOOK }}
        run: |
          bash .github/scripts/notify_workflow_failure.sh "$SLACK_WEBHOOK"
```
(This also avoids routing the secret through `GITHUB_OUTPUT`.)

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@github-actions
Copy link
Copy Markdown

🤖 Codex Code Review

Findings

  1. daily_snapsync.yaml and daily_snapsync.yaml switch failure notifications to a new secret, but notify_snapsync_run.sh exits successfully if either webhook env var is empty. Because it checks SLACK_WEBHOOK_URL_FAILURE before branching on OUTCOME, a missing ETHREX_INTERNO_SLACK_WEBHOOK suppresses even successful snapsync notifications. I’d add an explicit secret-presence assertion in the workflow, or change the script to require only the webhook needed for the actual outcome.

  2. common_failure_alerts.yaml and daily_hive_report.yaml have the same migration risk, but with silent failure modes. notify_workflow_failure.sh treats an empty webhook as success, so alerts can disappear while the job stays green. In the Hive workflow, the && ... || ... expression will fall through to TEST_CHANNEL_SLACK if the new secret is empty, which can misroute a scheduled production report to the test channel. If this secret is not already provisioned everywhere, this change is brittle.

Open Questions

  • Is dropping the old L1/L2 fan-out for the scheduled Hive report intentional? daily_hive_report.yaml now sends only one copy instead of two.

No Rust/EVM/consensus code is touched here. Outside the secret-provisioning risk above, the workflow edits are straightforward.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@ilitteri ilitteri enabled auto-merge April 27, 2026 14:48
@ilitteri ilitteri added this pull request to the merge queue May 4, 2026
Merged via the queue into main with commit 08e614e May 4, 2026
90 checks passed
@ilitteri ilitteri deleted the ci/route-failure-alerts-to-interno-channel branch May 4, 2026 14:18
@github-project-automation github-project-automation Bot moved this from In Review to Done in ethrex_l1 May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

L1 Ethereum client

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants