ci: split OSP integration tests to nightly + add ASan/UBSan

PR CI was burning runner time on 40 OSP integration workflows that
each spin up multiple matrix jobs, install a Debian container, install
.debs, and run upstream test suites -- on every push. That's the
runner-throttling we've been hitting. Move all of that to nightly.

OSP workflows -> reusable + dispatch-only
=========================================
40 workflows converted from `on: pull_request + push` to
`on: workflow_call + workflow_dispatch`. PRs no longer trigger them.
The `wait_for_smoke` job inside each is removed -- nightly doesn't
have a smoke gate (smoke gates the open-PR fast feedback loop, not
scheduled runs). Upstream matrices restored where Phase A had trimmed
them:
  - curl: curl_ref back to [curl-8_4_0, curl-7_88_1]
  - openssh: openssh_ref back to [V_10_0_P2, V_9_9_P1]
  - git-ssh-dr: key_type back to all four, iterations back to 10

The 40 OSPs: bind9, cjose, curl, debian-package, git-ssh-dr, grpc,
hostap, iperf, krb5, libcryptsetup, libeac3, libfido2, libhashkit2,
libnice, liboauth2, librelp, libssh2, libtss2, libwebsockets, net-snmp,
nginx, openldap, opensc, openssh, openvpn, pam-pkcs11, ppp,
python3-ntp, qt5network5, rsync, socat, sscep, sssd, stunnel, systemd,
tcpdump, tnftp, tpm2-tools, x11vnc, xmlsec.

New nightly orchestrator (.github/workflows/nightly-osp.yml)
============================================================
`schedule: 0 6 * * *` + workflow_dispatch. Fans out all 40 OSP
workflows in parallel via `uses:` and aggregates results in a
`notify` job that:
  - Always runs (`if: always()`) so failures don't suppress the report.
  - Parses `toJSON(needs)` to build pass/fail lists with jq:
      to_entries[] | select(.value.result != "success") | "\(.key) (\(.value.result))"
    (the `[]` stream is load-bearing -- `map(...)` then `.[].key`
    inside a string template is malformed jq.)
  - Posts a green/red Slack attachment to SLACK_WEBHOOK_URL, with
    `curl -fsS` so HTTP errors actually fail the workflow.
  - Writes the same summary to $GITHUB_STEP_SUMMARY so the run page
    is readable even when SLACK_WEBHOOK_URL isn't set.
  - SLACK_WEBHOOK_URL is read at JOB-level env so the step `if:` can
    see it. Step-level env is not in scope for that step's own `if:`.

ASan + UBSan workflow (.github/workflows/sanitizers.yml)
========================================================
Builds OpenSSL, wolfSSL, and wolfProvider from source under
-fsanitize=address,undefined -fno-omit-frame-pointer
-fno-sanitize-recover=all -static-libasan, then runs do-cmd-tests.sh
against the instrumented binaries. ASAN_OPTIONS and UBSAN_OPTIONS set
to halt on first hit so we don't drown in cascades. Versions come
from _discover-versions.yml. Gated on smoke. Runs on PR.

wait_for_smoke kept where it matters
====================================
After the OSP move, the PR-triggered workflows that build wolfProvider
all gate on smoke: simple, cmdline, fips-ready, openssl-version,
seed-src, multi-compiler, sanitizers. codespell stays ungated (it
doesn't build wolfprov).

Requires repo secret SLACK_WEBHOOK_URL for the Slack push to fire;
absent it the workflow still runs and writes the summary to the job
output.

Author: aidangarske

.github/workflows/bind9.yml

@@ -1,50 +1,23 @@
 name: Bind9 Tests
 
-# START OF COMMON SECTION
+# OSP integration test for Bind9 Tests. Runs nightly via the
+# Nightly OSP Suite orchestrator (.github/workflows/nightly-osp.yml)
+# or manually via workflow_dispatch. NOT triggered on PR/push --
+# PR CI focuses on smoke + simple + cheap internal checks.
+
 on:
-  push:
-    branches: [ 'master', 'main', 'release/**' ]
-  pull_request:
-    branches: [ '*' ]
-    types: [opened, synchronize, reopened, ready_for_review]
-    paths-ignore:
-      - '**.md'
-      - 'docs/**'
-      - 'LICENSE*'
-      - '.github/ISSUE_TEMPLATE/**'
-      - '.github/dependabot.yml'
-      - '.gitignore'
-      - 'AUTHORS'
-      - 'COPYING'
-      - 'README*'
-      - 'CHANGELOG*'
+  workflow_call: {}
+  workflow_dispatch: {}
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
-# END OF COMMON SECTION
-
 jobs:
-  wait_for_smoke:
-    name: Wait for smoke
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
-    runs-on: ubuntu-22.04
-    timeout-minutes: 35
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          fetch-depth: 1
-      - uses: ./.github/actions/wait-for-smoke
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-
   discover_versions:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     uses: ./.github/workflows/_discover-versions.yml
 
   build_wolfprovider:
-    needs: [wait_for_smoke, discover_versions]
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
+    needs: discover_versions
     uses: ./.github/workflows/build-wolfprovider.yml
     with:
       wolfssl_ref: ${{ matrix.wolfssl_ref }}
@@ -60,7 +33,6 @@ jobs:
         replace_default: [ true ]
 
   test_bind:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     runs-on: ubuntu-22.04
     needs: [build_wolfprovider, discover_versions]
     container:

.github/workflows/cjose.yml

@@ -1,50 +1,23 @@
 name: cjose Tests
 
-# START OF COMMON SECTION
+# OSP integration test for cjose Tests. Runs nightly via the
+# Nightly OSP Suite orchestrator (.github/workflows/nightly-osp.yml)
+# or manually via workflow_dispatch. NOT triggered on PR/push --
+# PR CI focuses on smoke + simple + cheap internal checks.
+
 on:
-  push:
-    branches: [ 'master', 'main', 'release/**' ]
-  pull_request:
-    branches: [ '*' ]
-    types: [opened, synchronize, reopened, ready_for_review]
-    paths-ignore:
-      - '**.md'
-      - 'docs/**'
-      - 'LICENSE*'
-      - '.github/ISSUE_TEMPLATE/**'
-      - '.github/dependabot.yml'
-      - '.gitignore'
-      - 'AUTHORS'
-      - 'COPYING'
-      - 'README*'
-      - 'CHANGELOG*'
+  workflow_call: {}
+  workflow_dispatch: {}
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
-# END OF COMMON SECTION
-
 jobs:
-  wait_for_smoke:
-    name: Wait for smoke
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
-    runs-on: ubuntu-22.04
-    timeout-minutes: 35
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          fetch-depth: 1
-      - uses: ./.github/actions/wait-for-smoke
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-
   discover_versions:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     uses: ./.github/workflows/_discover-versions.yml
 
   build_wolfprovider:
-    needs: [wait_for_smoke, discover_versions]
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
+    needs: discover_versions
     uses: ./.github/workflows/build-wolfprovider.yml
     with:
       wolfssl_ref: ${{ matrix.wolfssl_ref }}
@@ -60,7 +33,6 @@ jobs:
         replace_default: [ true ]
 
   test_cjose:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     runs-on: ubuntu-22.04
     needs: [build_wolfprovider, discover_versions]
     # Run inside Debian Bookworm to match packaging environment

.github/workflows/cmdline.yml

@@ -25,7 +25,21 @@ concurrency:
 # END OF COMMON SECTION
 
 jobs:
+  wait_for_smoke:
+    name: Wait for smoke
+    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
+    runs-on: ubuntu-22.04
+    timeout-minutes: 35
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+      - uses: ./.github/actions/wait-for-smoke
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
   cmdtest_test:
+    needs: wait_for_smoke
     if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     name: Command line test
     runs-on: ubuntu-22.04

.github/workflows/curl.yml

@@ -1,50 +1,23 @@
 name: Curl Tests
 
-# START OF COMMON SECTION
+# OSP integration test for Curl Tests. Runs nightly via the
+# Nightly OSP Suite orchestrator (.github/workflows/nightly-osp.yml)
+# or manually via workflow_dispatch. NOT triggered on PR/push --
+# PR CI focuses on smoke + simple + cheap internal checks.
+
 on:
-  push:
-    branches: [ 'master', 'main', 'release/**' ]
-  pull_request:
-    branches: [ '*' ]
-    types: [opened, synchronize, reopened, ready_for_review]
-    paths-ignore:
-      - '**.md'
-      - 'docs/**'
-      - 'LICENSE*'
-      - '.github/ISSUE_TEMPLATE/**'
-      - '.github/dependabot.yml'
-      - '.gitignore'
-      - 'AUTHORS'
-      - 'COPYING'
-      - 'README*'
-      - 'CHANGELOG*'
+  workflow_call: {}
+  workflow_dispatch: {}
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
-# END OF COMMON SECTION
-
 jobs:
-  wait_for_smoke:
-    name: Wait for smoke
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
-    runs-on: ubuntu-22.04
-    timeout-minutes: 35
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          fetch-depth: 1
-      - uses: ./.github/actions/wait-for-smoke
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-
   discover_versions:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     uses: ./.github/workflows/_discover-versions.yml
 
   build_wolfprovider:
-    needs: [wait_for_smoke, discover_versions]
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
+    needs: discover_versions
     uses: ./.github/workflows/build-wolfprovider.yml
     with:
       wolfssl_ref: ${{ matrix.wolfssl_ref }}
@@ -60,7 +33,6 @@ jobs:
         replace_default: [ true ]
 
   test_curl:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     runs-on: ubuntu-22.04
     needs: [build_wolfprovider, discover_versions]
     container:
@@ -72,8 +44,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        # PR runs latest curl only. Older refs are exercised at release time.
-        curl_ref: [ 'curl-8_4_0' ]
+        curl_ref: [ 'curl-8_4_0', 'curl-7_88_1' ]
         wolfssl_ref: ${{ fromJson(needs.discover_versions.outputs.wolfssl_ref_array) }}
         openssl_ref: ${{ fromJson(needs.discover_versions.outputs.openssl_ref_array) }}
         fips_ref: [ 'FIPS', 'non-FIPS' ]

.github/workflows/debian-package.yml

@@ -1,50 +1,23 @@
 name: Debian Package Test
 
-# START OF COMMON SECTION
+# OSP integration test for Debian Package Test. Runs nightly via the
+# Nightly OSP Suite orchestrator (.github/workflows/nightly-osp.yml)
+# or manually via workflow_dispatch. NOT triggered on PR/push --
+# PR CI focuses on smoke + simple + cheap internal checks.
+
 on:
-  push:
-    branches: [ 'master', 'main', 'release/**' ]
-  pull_request:
-    branches: [ '*' ]
-    types: [opened, synchronize, reopened, ready_for_review]
-    paths-ignore:
-      - '**.md'
-      - 'docs/**'
-      - 'LICENSE*'
-      - '.github/ISSUE_TEMPLATE/**'
-      - '.github/dependabot.yml'
-      - '.gitignore'
-      - 'AUTHORS'
-      - 'COPYING'
-      - 'README*'
-      - 'CHANGELOG*'
+  workflow_call: {}
+  workflow_dispatch: {}
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
-# END OF COMMON SECTION
-
 jobs:
-  wait_for_smoke:
-    name: Wait for smoke
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
-    runs-on: ubuntu-22.04
-    timeout-minutes: 35
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          fetch-depth: 1
-      - uses: ./.github/actions/wait-for-smoke
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-
   discover_versions:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     uses: ./.github/workflows/_discover-versions.yml
 
   build_wolfprovider:
-    needs: [wait_for_smoke, discover_versions]
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
+    needs: discover_versions
     uses: ./.github/workflows/build-wolfprovider.yml
     with:
       wolfssl_ref: ${{ matrix.wolfssl_ref }}
@@ -60,7 +33,6 @@ jobs:
         replace_default: [ true, false ]
 
   libwolfprov-replace-default:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     name: libwolfprov ${{ matrix.replace_default && 'replace-default' || 'standalone' }} ${{ matrix.fips_ref }}
     runs-on: ubuntu-22.04
     needs: [build_wolfprovider, discover_versions]

.github/workflows/fips-ready.yml

@@ -25,7 +25,21 @@ concurrency:
 # END OF COMMON SECTION
 
 jobs:
+  wait_for_smoke:
+    name: Wait for smoke
+    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
+    runs-on: ubuntu-22.04
+    timeout-minutes: 35
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+      - uses: ./.github/actions/wait-for-smoke
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
   fips_ready_test:
+    needs: wait_for_smoke
     if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     name: FIPS Ready Bundle Test
     runs-on: ubuntu-22.04

.github/workflows/git-ssh-dr.yml

@@ -1,48 +1,23 @@
 name: Git SSH Default Replace Tests
 
+# OSP integration test for Git SSH Default Replace Tests. Runs nightly via the
+# Nightly OSP Suite orchestrator (.github/workflows/nightly-osp.yml)
+# or manually via workflow_dispatch. NOT triggered on PR/push --
+# PR CI focuses on smoke + simple + cheap internal checks.
+
 on:
-  push:
-    branches: [ 'master', 'main', 'release/**' ]
-  pull_request:
-    branches: [ '*' ]
-    types: [opened, synchronize, reopened, ready_for_review]
-    paths-ignore:
-      - '**.md'
-      - 'docs/**'
-      - 'LICENSE*'
-      - '.github/ISSUE_TEMPLATE/**'
-      - '.github/dependabot.yml'
-      - '.gitignore'
-      - 'AUTHORS'
-      - 'COPYING'
-      - 'README*'
-      - 'CHANGELOG*'
+  workflow_call: {}
+  workflow_dispatch: {}
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
-
 jobs:
-  wait_for_smoke:
-    name: Wait for smoke
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
-    runs-on: ubuntu-22.04
-    timeout-minutes: 35
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          fetch-depth: 1
-      - uses: ./.github/actions/wait-for-smoke
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-
   discover_versions:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     uses: ./.github/workflows/_discover-versions.yml
 
   build_wolfprovider:
-    needs: [wait_for_smoke, discover_versions]
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
+    needs: discover_versions
     uses: ./.github/workflows/build-wolfprovider.yml
     with:
       wolfssl_ref: ${{ matrix.wolfssl_ref }}
@@ -58,7 +33,6 @@ jobs:
         replace_default: [ true ]
 
   git-ssh-default-replace-test:
-    if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
     runs-on: ubuntu-22.04
     container:
       image: ghcr.io/${{ github.event.pull_request && github.event.pull_request.head.repo.owner.login || github.repository_owner }}/wolfprovider-test-deps:bookworm
@@ -74,10 +48,8 @@ jobs:
         openssl_ref: ${{ fromJson(needs.discover_versions.outputs.openssl_ref_array) }}
         fips_ref: [ 'FIPS', 'non-FIPS' ]
         replace_default: [ true ]
-        # PR matrix: 2 of 4 key types and 3 iterations.
-        # Other key_types and longer soak runs are exercised at release time.
-        key_type: [ 'rsa', 'ed25519' ]
-        iterations: [ 3 ]
+        key_type: [ 'rsa', 'ecdsa', 'ed25519', 'chacha20-poly1305' ]
+        iterations: [ 10 ] # Total of 80 runs
         # force_fail collapsed into sequential runs in the test step
     env:
       WOLFSSL_PACKAGES_PATH: /tmp/wolfssl-packages