Block outbound calls from ci runners to unallowed endpoints #1175
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,10 +20,16 @@ jobs: | |
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: "Harden the runner (Block egress traffic: Only allow calls to allowed endpoints)" | ||
| uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 | ||
| with: | ||
| egress-policy: block | ||
| allowed-endpoints: >+ | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| github.com:443 | ||
| api.github.com:443 | ||
| release-assets.githubusercontent.com:443 | ||
| pypi.org:443 | ||
| files.pythonhosted.org:443 | ||
|
Comment on lines
+23
to
+32
Contributor
There was a problem hiding this comment. 🧩 Analysis chain🏁 Script executed: cat -n .github/workflows/python-publish.ymlRepository: dfetch-org/dfetch Length of output: 4899 🏁 Script executed: web_search
actions/upload-artifact endpoint requirements v7 what endpoints neededRepository: dfetch-org/dfetch Length of output: 182 🌐 Web query:
💡 Result: The actions/upload-artifact@v7 action, powered by the Citations:
🌐 Web query:
💡 Result: Yes, StepSecurity's Harden-Runner supports wildcard domain patterns in the allowed-endpoints configuration for egress policy blocking. This feature allows allowing egress traffic to all subdomains of a domain with a single rule using the * prefix, such as *.data.mcr.microsoft.com:443. This support was added in response to user feedback for handling dynamic endpoints like container registries (e.g., region-specific mcr.microsoft.com subdomains). It works in block mode (egress-policy: block). Example from official StepSecurity blog: steps: - name: Harden Runner uses: step-security/harden-runner@v2 with: egress-policy: block allowed-endpoints: > mcr.microsoft.com:443 *.data.mcr.microsoft.com:443 - run: docker pull mcr.microsoft.com/azure-cli Wildcard patterns can also be used in suppression rules for anomalous endpoint alerts and likely in Policy Store allowed endpoints. The format is domain:port with * for subdomains (e.g., *.example.com:443). It matches subdomains but not the root domain unless explicitly allowed. Citations:
🌐 Web query:
💡 Result: results-receiver.actions.githubusercontent.com is an internal GitHub Actions endpoint used by self-hosted runners to report progress, upload logs, step summaries, and job results during workflow execution. It is part of the "Results" service in the GitHub Actions runner infrastructure. It is not the public endpoint for workflow artifacts (which are managed via REST API at api.github.com/repos/{owner}/{repo}/actions/artifacts). Instead, it handles real-time runner-to-GitHub communication for job execution data, including step updates, logs, and summaries. Key usage from official sources: - Required network access for self-hosted runners: https://results-receiver.actions.githubusercontent.com/.../ for reporting progress and uploading logs (actions/runner docs). - Listed in GitHub Docs for GitHub-hosted runners firewall rules, alongside *.blob.core.windows.net for artifacts, logs, etc. - GitHub Changelog (2023): Required for job summaries and logs; self-hosted runners must allow access to this host. - Runner PRs confirm it receives step updates, powers logs, uploads step summaries. Health check: curl https://results-receiver.actions.githubusercontent.com/health For public artifact management, use the documented REST API endpoints, not this runner-specific service. Citations:
🌐 Web query:
💡 Result: The standard GitHub Actions upload-artifact action does not directly upload to Azure Blob Storage endpoints like *.blob.core.windows.net. Instead, it uploads files to GitHub's internal artifact service via the runner's connection to the GitHub server (using classes like FileContainerServer in the open-source runner code), which stores artifacts in GitHub's cloud storage (likely S3 or similar). Artifacts are accessible via GitHub REST API endpoints like /repos/{owner}/{repo}/actions/artifacts/{artifact_id}/zip, not direct blob URLs. However, blob.core.windows.net appears in two main contexts related to GitHub Actions and Azure: 1. GitHub Enterprise Server (GHES) configuration: For self-hosted GHES instances, admins can configure Azure Blob Storage as the backend for storing Actions artifacts, logs, and caches. The blob endpoint is https://{storage-account}.blob.core.windows.net, where {storage-account} is your Azure storage account name. Runners upload to this storage via GHES configuration (using OIDC or connection string). See official GHES docs for setup (e.g., provide storage account name and core.windows.net suffix). 2. Custom uploads from workflows: To upload build files/artifacts directly to your own Azure Blob Storage from GitHub Actions workflows (hosted or self-hosted runners), use Azure CLI or third-party actions. The endpoint is https://{storage-account}.blob.core.windows.net/{container}/{blob-path}. Official Microsoft-recommended example using Azure CLI (from learn.microsoft.com, updated 2025): - name: Upload to blob storage uses: azure/CLI@v1 with: inlineScript: | az storage blob upload-batch --account-name {STORAGE_ACCOUNT_NAME} --auth-mode key -d '$web' -s . Replace {STORAGE_ACCOUNT_NAME} with your account (e.g., mystorage.blob.core.windows.net). Other popular actions: - bacongobbler/azure-blob-storage-upload: Uses az storage blob upload-batch. - LanceMcCarthy/Action-AzureBlobUpload: Supports connection string or service principal. - fixpoint/azblob-upload-artifact: Mimics actions/upload-artifact syntax for easy migration. No public "upload endpoint" exists for direct PUT to blobs from runners without authentication (SAS token, key, etc.). Always use authenticated methods. For GHES backend storage, it's internal to the GHES setup, not a user-facing endpoint. Citations:
🌐 Web query:
💡 Result: For GitHub-hosted runners, no special firewall egress configuration is required as they have full outbound internet access. The query likely refers to self-hosted runners, where firewall egress policies must allow specific GitHub endpoints over HTTPS (port 443) for the actions/upload-artifact to function. Required outbound HTTPS endpoints/domains for self-hosted runners on github.com (allow recursively for CNAMEs): Essential operations: - github.com - api.github.com - *.actions.githubusercontent.com Downloading actions: - codeload.github.com Uploading/downloading artifacts, logs, caches (critical for upload-artifact): - results-receiver.actions.githubusercontent.com - *.blob.core.windows.net (Azure Blob Storage used by upload-artifact) Runner updates: - objects.githubusercontent.com - objects-origin.githubusercontent.com - github-releases.githubusercontent.com OIDC tokens: - *.actions.githubusercontent.com (already covered) Packages (if used): - *.pkg.github.com - ghcr.io Additional for workflows (e.g., LFS): - github-cloud.githubusercontent.com To test connectivity from the runner machine: curl -v https://api.github.com/zen curl -v https://codeload.github.com/_ping curl -v https://vstoken.actions.githubusercontent.com/_apis/health curl -v https://pipelines.actions.githubusercontent.com/_apis/health curl -v https://results-receiver.actions.githubusercontent.com/health Use the runner config script: ./config.sh --check --url https://github.com/OWNER/REPO --pat ghp_XXX GitHub does not recommend IP allowlisting due to frequent changes; use domain allowlisting instead. Retrieve current IPs via https://api.github.com/meta but monitor regularly. upload-artifact failures (e.g., ETIMEDOUT) explicitly reference ensuring access to these endpoints. Citations:
🌐 Web query:
💡 Result: The artifact upload failure in GitHub Actions when using step-security/harden-runner is caused by the security agent's egress policy blocking outbound connections to GitHub's artifact storage endpoints. Harden-Runner restricts network traffic to only explicitly allowed endpoints when configured with egress-policy: block. Key points: - Harden-Runner monitors and can block outbound network calls to prevent data exfiltration. - Artifact uploads require access to specific GitHub/Azure endpoints like *.blob.core.windows.net (for blob storage), results-receiver.actions.githubusercontent.com, and pipelines.actions.githubusercontent.com. - If these are missing from the allowed-endpoints list in the Harden-Runner step, uploads fail with errors like "missing endpoints," timeouts, ETIMEDOUT, or "Unable to make request." - Common error patterns match upload-artifact issues on self-hosted runners, but here it's due to the agent's blocking. Solution: 1. Run Harden-Runner in egress-policy: audit mode first to log all outbound calls and generate insights/recommended policy. 2. Review the security insights (linked in job summary) for network events during artifact upload steps. 3. Add the required endpoints to allowed-endpoints in block mode, e.g.: - uses: step-security/harden-runner@v2 with: egress-policy: block allowed-endpoints: > github.com:443 api.github.com:443 *.actions.githubusercontent.com:443 *.blob.core.windows.net:443 results-receiver.actions.githubusercontent.com:443 4. Harden-Runner auto-detects some GitHub endpoints like cache blobs, but artifact endpoints must be explicitly allowed or baselined. 5. Use the Policy Store feature for dynamic management without workflow changes. 6. Verify runner connectivity to GitHub endpoints using the official docs checklist. For self-hosted runners, ensure base network access; for GitHub-hosted, the agent enforces the policy strictly. Citations:
Add missing artifact storage endpoints to harden-runner allowlist. The Proposed fix allowed-endpoints: >+
github.com:443
api.github.com:443
release-assets.githubusercontent.com:443
+ results-receiver.actions.githubusercontent.com:443
+ *.blob.core.windows.net:443
pypi.org:443
files.pythonhosted.org:443🤖 Prompt for AI Agents |
||
|
|
||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| with: | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,10 +26,21 @@ jobs: | |
| id-token: write | ||
|
|
||
| steps: | ||
| - name: "Harden the runner (Block egress traffic: Only allow calls to allowed endpoints)" | ||
| uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 | ||
| with: | ||
| egress-policy: block | ||
| allowed-endpoints: >+ | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| github.com:443 | ||
| api.github.com:443 | ||
| raw.githubusercontent.com:443 | ||
| codeload.github.com:443 | ||
| uploads.github.com:443 | ||
| api.osv.dev:443 | ||
| www.bestpractices.dev:443 | ||
| api.securityscorecards.dev:443 | ||
| rekor.sigstore.dev:443 | ||
| fulcio.sigstore.dev:443 | ||
|
Comment on lines
+29
to
+43
Contributor
There was a problem hiding this comment. 🧩 Analysis chain🏁 Script executed: cat -n .github/workflows/scorecard.yml | head -80Repository: dfetch-org/dfetch Length of output: 2970 🏁 Script executed: grep -A 10 "upload-artifact" .github/workflows/scorecard.ymlRepository: dfetch-org/dfetch Length of output: 438 🌐 Web query:
💡 Result: The query appears to seek information on using GitHub Actions' actions/upload-artifact with Azure Blob Storage, specifically network endpoints and requirements for uploads. However, actions/upload-artifact uploads to GitHub's artifact storage service, not directly to Azure Blob Storage (sources 9-12,28-30). For Azure Blob Storage uploads from GitHub Actions, use third-party actions like azure-blob-storage-upload or azblob-upload-artifact, or Azure CLI commands (sources 1-5). Azure Blob Storage upload uses REST API endpoints primarily via PUT requests: - Primary endpoint: https://{account}.blob.core.windows.net/{container}/{blob} (HTTPS recommended) (sources 19,20,24). - Key operations: - Put Blob: Creates/updates block, page, or append blobs. Requires x-ms-blob-type header (BlockBlob, PageBlob, AppendBlob), Content-Length, x-ms-version (e.g., 2023-08-04 or later). For block blobs up to ~5 GiB (preview expansion); larger use Put Block + Put Block List (sources 19,21,24,26). - Put Block: https://{account}.blob.core.windows.net/{container}/{blob}?comp=block&blockid={base64id}. Block size <=4 MiB (sources 26). - Put Block List: Commits blocks. Network requirements for Azure Blob Storage (from GitHub Actions runners): - Public access: Outbound HTTPS (port 443) to *.blob.core.windows.net. GitHub-hosted runners have internet access by default. - Private endpoints: Use privatelink.blob.core.windows.net. Requires VNet configuration, DNS resolution to private IP. GitHub runners cannot directly access private endpoints without public IP allowlisting or Azure service endpoints (sources 14,15,17). - Firewall rules: Allow GitHub runner IPs or disable public network access and use authorized networks. IP rules don't block same-region Azure traffic (source 15). - Scalability: Default 20,000 req/s, ingress/egress up to 25-50 Gbps per account (source 16). - Auth: Shared Key, SAS, Microsoft Entra ID (RBAC roles like Storage Blob Data Contributor), connection string in secrets (sources 1-5,19,27). For GitHub Actions artifacts (not Azure): Uses internal GitHub endpoints like /repos/{owner}/{repo}/actions/artifacts/{id}/zip (sources 9,29). Self-hosted runners may need firewall rules for uploads (source 11). Examples: - Third-party action: uses: bacongobbler/azure-blob-storage-upload@main with connection_string secret (source 1). - CLI: az storage blob upload-batch --account-name --auth-mode key -d $web -s . (source 5). Citations:
🌐 Web query:
💡 Result: actions/upload-artifact v7 uses the Citations:
🌐 Web query:
💡 Result: results-receiver.actions.githubusercontent.com is an internal GitHub endpoint used by GitHub Actions runners and the Citations:
🌐 Web query:
💡 Result: Yes, StepSecurity's Harden-Runner supports wildcard endpoints, including for blob.core.windows.net, since version 2.4.0. This feature allows using patterns like ".data.mcr.microsoft.com:443" or ".blob.core.windows.net:443" in the allowed-endpoints list when egress-policy is set to 'block'. For GitHub Actions cache/artifacts specifically, which use varying *.blob.core.windows.net subdomains, Harden-Runner auto-detects and allows the specific cache endpoint (since v1.5.0), eliminating the need for manual whitelisting or wildcards in most cases. However, wildcards can still be used explicitly if preferred, as users have requested and implemented in some workflows (e.g., Azure/karpenter-provider-azure PR). No security concerns are raised against using the wildcard for GitHub cache endpoints, though broad whitelisting is generally discouraged for non-essential traffic. Citations:
Missing artifact upload endpoints in harden-runner allowlist will block artifact uploads. This job uses 🐛 Proposed fix allowed-endpoints: >+
github.com:443
api.github.com:443
raw.githubusercontent.com:443
codeload.github.com:443
uploads.github.com:443
+ results-receiver.actions.githubusercontent.com:443
+ *.blob.core.windows.net:443
api.osv.dev:443
www.bestpractices.dev:443
api.securityscorecards.dev:443🤖 Prompt for AI Agents |
||
|
|
||
| - name: "Checkout code" | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.