feat(ci): run integration tests on Dependabot PRs#149
Conversation
| pull_request_target: | ||
| types: [opened, synchronize, reopened] |
There was a problem hiding this comment.
Using unsafe GitHub Actions trigger may allow privilege escalation via CI/CD - critical severity
Using pull_request_target or workflow_run as a trigger is not recommended, as it may allow an attacker to elevate its privileges via the CI/CD pipeline by exfiltrating secrets (e.g. by reading out the caches of the GitHub Actions pipeline or listing loaded secrets in the environment). If the affected repository is open source, the attacker doesn't have to be an insider but could be any GitHub user.
Show fix
Remediation: Avoid using pull_request_target or workflow_run as a trigger.
Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info
There was a problem hiding this comment.
@AikidoSec ignore: Deliberate, mitigated pull_request_target use. The credentialed job is gated to dependabot[bot] (authenticated author, unspoofable; workflow always runs from main so the gate can't be removed via a PR), so external users can't reach it. The cited exfil vectors are closed: no cache is used, and permissions: contents: read means no writable token. Residual risk (insider pushing to a dependabot branch; the bumped crate running with a test-scoped key) is inherent to live dependency coverage and accepted.
There was a problem hiding this comment.
✅ Based on your feedback, we ignored this issue because of the following reason:
Deliberate, mitigated pull_request_target use. The credentialed job is gated to
dependabot[bot](authenticated author, unspoofable; workflow always runs from main so the gate can't be removed via a PR), so external users can't reach it. The cited exfil vectors are closed: no cache is used, andpermissions: contents: readmeans no writable token. Residual risk (insider pushing to a dependabot branch; the bumped crate running with a test-scoped key) is inherent to live dependency coverage and accepted.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
| env: | ||
| HOTDATA_SDK_TEST_API_URL: ${{ vars.HOTDATA_SDK_TEST_API_URL }} | ||
| HOTDATA_SDK_TEST_API_KEY: ${{ secrets.HOTDATA_SDK_TEST_API_KEY }} | ||
| HOTDATA_SDK_TEST_WORKSPACE_ID: ${{ vars.HOTDATA_SDK_TEST_WORKSPACE_ID }} | ||
| HOTDATA_SDK_TEST_CONNECTION_ID: ${{ vars.HOTDATA_SDK_TEST_CONNECTION_ID }} |
There was a problem hiding this comment.
Blocking: This workflow will almost certainly run green while testing nothing.
The premise in the header comment — "pull_request_target executes in the base-repo context where the test credentials are available" — does not hold for Dependabot-authored events. GitHub treats any run where github.actor == 'dependabot[bot]' as untrusted regardless of trigger, including pull_request_target: the run gets a read-only GITHUB_TOKEN and no access to the Actions secrets/variables store. Dependabot-triggered runs can only read the separate Dependabot secrets/variables store (Settings → Secrets and variables → Dependabot).
So unless HOTDATA_SDK_TEST_API_KEY / HOTDATA_SDK_TEST_WORKSPACE_ID (and the URL/connection vars) are also configured as Dependabot secrets/variables, all four ${{ secrets.* }} / ${{ vars.* }} here resolve to empty strings. And the harness in tests/common/mod.rs treats empty as absent (non_empty() → has_creds() false) and skips every scenario, exiting green:
"GitHub Actions sets
env:keys even when the underlying secret/var is unset, producing empty strings rather than absent keys. We treat empty strings as absent."
Net result: the job passes, the PR looks covered, and zero credentialed scenarios actually ran — strictly worse than no workflow, because it manufactures false confidence.
Two things needed before merge:
- Confirm the four credentials are mirrored into the Dependabot secret/variable store (the Actions copies used by
integration-tests.ymlare not visible here). - Fail loudly on misconfiguration. The silent-skip behavior is correct for fork PRs in
integration-tests.yml, but it's wrong for a workflow whose sole purpose is credentialed runs. Add a guard so a missing/empty key fails the job instead of passing green, e.g.:
- name: Require credentials
env:
HOTDATA_SDK_TEST_API_KEY: ${{ secrets.HOTDATA_SDK_TEST_API_KEY }}
HOTDATA_SDK_TEST_WORKSPACE_ID: ${{ vars.HOTDATA_SDK_TEST_WORKSPACE_ID }}
run: |
test -n "$HOTDATA_SDK_TEST_API_KEY" || { echo "::error::HOTDATA_SDK_TEST_API_KEY is empty — is it set as a *Dependabot* secret?"; exit 1; }
test -n "$HOTDATA_SDK_TEST_WORKSPACE_ID" || { echo "::error::HOTDATA_SDK_TEST_WORKSPACE_ID is empty — is it set as a *Dependabot* variable?"; exit 1; }There was a problem hiding this comment.
You're right, and thanks for the catch — the premise was wrong. Verified against GitHub's docs that the Dependabot sandbox keys on pull_request.user.login == 'dependabot[bot]' and applies to pull_request_target too, so it would have skipped green exactly as you describe. Reworked: dropped pull_request_target entirely (also clears the Aikido finding), switched to a plain pull_request job that reads the four creds from the Dependabot secrets store, and added the fail-loud guard so an empty credential errors instead of passing. The four values must be set under Settings → Secrets and variables → Dependabot for the job to run.
| jobs: | ||
| integration: | ||
| if: github.event.pull_request.user.login == 'dependabot[bot]' | ||
| runs-on: ubuntu-latest |
There was a problem hiding this comment.
super nit: this job checks out and executes untrusted dependency code (build scripts, proc-macros) with a live API key present, as the header comment acknowledges. Consider adding a timeout-minutes: to the job as cheap defense-in-depth so a runaway/abusive build script can't sit on the credentialed runner indefinitely. (not blocking)
There was a problem hiding this comment.
Added timeout-minutes: 20 to the job — good defense-in-depth, thanks.
There was a problem hiding this comment.
Review
Blocking Issues
- The credentialed suite will silently no-op.
pull_request_targetruns triggered by Dependabot (github.actor == 'dependabot[bot]') are treated as untrusted by GitHub and do not receive Actions secrets/variables — only the separate Dependabot secrets/variables store is injected. Combined with the harness intests/common/mod.rs, which treats empty credentials as "skip cleanly and exit green," this workflow will pass while running zero credentialed scenarios unless the secrets are mirrored into the Dependabot store. That produces false coverage, which is worse than no workflow. See the inline comment on theenv:block.
Action Required
- Confirm
HOTDATA_SDK_TEST_API_KEY,HOTDATA_SDK_TEST_WORKSPACE_ID,HOTDATA_SDK_TEST_API_URL, andHOTDATA_SDK_TEST_CONNECTION_IDare configured as Dependabot secrets/variables (not just Actions secrets). - Add a guard step that fails the job when the required credentials are empty, so a misconfiguration surfaces loudly instead of as a green-but-untested run.
There was a problem hiding this comment.
Cycle 2: all prior blocking threads resolved in code.
- pull_request_target dropped — now plain
pull_request(clears the Aikido finding). - Switched to the Dependabot secrets store + fail-loud guard. The guard checks
HOTDATA_SDK_TEST_API_KEYandHOTDATA_SDK_TEST_WORKSPACE_ID, which are exactly the two credsTestEnv::has_creds()gates on, so the job can't pass green while silently skipping every scenario. timeout-minutes: 20added.
LGTM.
Dependabot's sandboxed
pull_requestruns get no secrets/vars, so the normal Integration Tests workflow skips every credentialed scenario — behavioral regressions in a bump slip through green. This adds apull_request_targetworkflow that runs the real suite in the base-repo context where credentials exist.Hardened: gated to
dependabot[bot](forks can't trigger it),contents: readonly (no writable token exposed to the build), and no shared cargo cache.