diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 4a32aa271df1..f6b1a440e175 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -545,3 +545,4 @@ PostCommit Jobs run in a schedule against master branch and generally do not get | [ Infrastructure Policy Enforcer ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml) | N/A | [![.github/workflows/beam_Infrastructure_PolicyEnforcer.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml?query=event%3Aschedule) | | [ Modify the GCP User Roles according to the infra/users.yml file ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml) | N/A | [![.github/workflows/beam_Infrastructure_UsersPermissions.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml?query=event%3Aschedule) | | [ Service Account Keys Management ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml) | N/A | [![.github/workflows/beam_Infrastructure_ServiceAccountKeys.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml?query=event%3Aschedule) | +| [ Unmanaged Service Accounts Keys Audit ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml) | N/A | [![.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml?query=event%3Aschedule) | diff --git a/.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml b/.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml new file mode 100644 index 000000000000..6d34bd1d156f --- /dev/null +++ b/.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml @@ -0,0 +1,69 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# This workflow works with the GCP security log analyzer to +# generate weekly security reports and initialize log sinks + +name: Unmanaged Service Accounts Keys Audit + +on: + workflow_dispatch: + schedule: + # Every day at 00:00 UTC + - cron: '0 0 * * *' + +concurrency: + group: ${{ github.workflow }} + cancel-in-progress: false + +permissions: + contents: read + issues: write + id-token: write + +jobs: + beam_UnmanagedKeysAudit: + name: Audit Unmanaged Service Account Keys + runs-on: [self-hosted, ubuntu-24.04, main] + timeout-minutes: 30 + steps: + - uses: actions/checkout@v6 + + - name: Setup gcloud + uses: google-github-actions/setup-gcloud@aa5489c8933f4cc7a4f7d45035b3b1440c9c10db + + - name: Setup Python + uses: actions/setup-python@v4 + with: + python-version: '3.13' + + - name: Install Python dependencies + working-directory: ./infra/enforcement + run: | + python -m pip install --upgrade pip + pip install -r requirements.txt + + - name: Run Unmanaged Service Account Keys Audit + working-directory: ./infra/enforcement + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GITHUB_REPOSITORY: ${{ github.repository }} + run: python account_keys.py --action announce + + + + diff --git a/infra/enforcement/README.md b/infra/enforcement/README.md index 8136081fed75..6d883f7e6806 100644 --- a/infra/enforcement/README.md +++ b/infra/enforcement/README.md @@ -134,8 +134,9 @@ The enforcement tools are integrated with GitHub Actions to provide automated co ### Workflow Configuration -The GitHub Actions workflow (`.github/workflows/beam_Infrastructure_PolicyEnforcer.yml`) runs: -- **Schedule**: Weekly on Mondays at 9:00 AM UTC +The repository includes workflows for different security domains: +- **IAM Policy Enforcer** (`.github/workflows/beam_Infrastructure_PolicyEnforcer.yml`): Runs weekly on Mondays at 9:00 AM UTC. +- **Unmanaged Keys Audit** (`.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml`): Runs daily at 00:00 UTC. It manages the continuous execution of the `account_keys.py` script to swiftly detect rogue service account keys generated outside the official rotation system. - **Manual trigger**: Can be triggered manually via `workflow_dispatch` - **Actions**: Runs both IAM and Account Keys enforcement with the `announce` action @@ -170,7 +171,9 @@ python account_keys.py --action generate ### Actions - **check**: Validates service account keys and their permissions against defined policies and reports any differences (default behavior) -- **announce**: Creates or updates a GitHub issue and sends an email notification when service account keys policies differ from the defined ones. If no open issue exists, creates a new one; if an open issue exists, updates the issue body with current violations +- **announce**: Creates or updates a GitHub issue and sends an email notification when service account keys policies differ from the defined ones. + - For general configuration errors, it updates the main compliance issue. + - **For unmanaged/rogue keys (Security Alerts)**, it consolidates alerts into a dedicated `[SECURITY]` issue acting as a live dashboard. It updates the issue by placing the newest audit report at the top and moving the previous reports into a collapsed `
` history section. If the keys are revoked and the infrastructure becomes healthy, the system automatically resolves and closes the issue. - **print**: Prints announcement details for testing purposes without creating actual GitHub issues or sending emails - **generate**: Updates the compliance file to match the current GCP service account keys and Secret Manager permissions @@ -183,6 +186,9 @@ The Account Keys enforcement tool provides the following capabilities: - **Permission Validation**: Ensures that Secret Manager permissions match the declared authorized users - **Compliance Reporting**: Identifies missing service accounts, undeclared managed secrets, and permission mismatches - **Automatic Remediation**: Can automatically update the compliance file to match current infrastructure state +- **Unmanaged Key Detection**: Identifies physical keys in IAM that were created outside the automated rotation system by comparing them against the legal, managed keys registered in Google Cloud Secret Manager. +- **Stateful Security Alerts**: Consolidates security violations into a single, dynamically updated GitHub issue, keeping a collapsed `
` history of past scans to prevent alert fatigue. +- **Self-Healing Resolution**: Automatically closes the security GitHub issue when all unmanaged keys have been successfully revoked. ### Configuration