Skip to content

ci: send Slack notifications on CI failures #2911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
anakin87 merged 8 commits into from
Mar 6, 2026
Merged

ci: send Slack notifications on CI failures #2911

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8135180
try notify Slack on failure
anakin87 Mar 5, 2026
f36875c
separate job
anakin87 Mar 5, 2026
22efcdc
try slim
anakin87 Mar 5, 2026
fe9b349
extend change to all workflows
anakin87 Mar 5, 2026
f02a7f8
simplify
anakin87 Mar 5, 2026
9f62ea0
refinement
anakin87 Mar 5, 2026
6af3e6d
fix
anakin87 Mar 5, 2026
f77dfcb
one blank line
anakin87 Mar 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 0 additions & 28 deletions .github/actions/send_failure/action.yml
View file
Open in desktop

This file was deleted.

10 changes: 3 additions & 7 deletions .github/workflows/CI_license_compliance.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ on:
- cron: "0 0 * * *" # every day at midnight

env:
CORE_DATADOG_API_KEY: ${{ secrets.CORE_DATADOG_API_KEY }}
PYTHON_VERSION: "3.10"
EXCLUDE_PACKAGES: "(?i)^(azure-identity|fastembed|ragas|tqdm|psycopg|mistralai|pgvector).*"

Expand Down Expand Up @@ -90,11 +89,8 @@ jobs:
if: ${{ always() }}
run: echo "${{ steps.license_check_report.outputs.report }}"

- name: Send event to Datadog for nightly failures
- name: Notify Slack on nightly failure
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations license compliance nightly failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}

slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
Comment on lines +92 to +96
Copy link
Copy Markdown
Contributor

@bogdankostic bogdankostic Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In all other workflow files we have the notification step as a separate job.

Copy link
Copy Markdown
Member Author

@anakin87 anakin87 Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in other workflows, we are using a separate job for this reason

to make sure we get a single notification when a matrix workflow fails (not one for each element in the matrix), I created a new lightweight job (uses ubuntu-slim)

In this case, we don't need a matrix: the license check only needs to run on a single operating system and with one Python version. Where possible, it's better to have a step instead of a job because GitHub does not need to spin up a new container.

13 changes: 7 additions & 6 deletions .github/workflows/aimlapi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,10 +71,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:cov-retry

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/amazon_bedrock.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,10 +92,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/amazon_sagemaker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/anthropic.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/arcadedb.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/astra.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,10 +73,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/azure_ai_search.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/azure_doc_intelligence.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/chroma.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,10 +73,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/cohere.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/cometapi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,10 +71,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:cov-retry

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/deepeval.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/elasticsearch.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/faiss.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/fastembed.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/firecrawl.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/github.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/google_genai.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
13 changes: 7 additions & 6 deletions .github/workflows/hanlp.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,11 @@ jobs:
hatch -e test env run -- uv pip install git+https://github.com/deepset-ai/haystack.git@main
hatch run test:unit

- name: Send event to Datadog for nightly failures
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/send_failure
notify-slack-on-failure:
needs: run
if: failure() && github.event_name == 'schedule'
runs-on: ubuntu-slim
steps:
- uses: deepset-ai/notify-slack-action@v1
with:
title: |
Core integrations nightly tests failure: ${{ github.workflow }}
api-key: ${{ secrets.CORE_DATADOG_API_KEY }}
slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL_NOTIFICATIONS }}
Loading
Loading