including GitHub Actions to schedule daily audits and Add documentati… (#39116)

* including GitHub Actions to schedule daily audits and Add documentation for the changes made to manage keys created outside of the Google Cloud Secret Manager key rotation system

* correcting a typo in the file name

* Fix PreCommit GHA and PreCommit Whitespace, and preparing the test of the action from the pull request

* Testing the GitHub action from the PR

* Remove the pull request configuration from the github Action

Author: HansMarcus01

.github/workflows/README.md

@@ -545,3 +545,4 @@ PostCommit Jobs run in a schedule against master branch and generally do not get
 | [ Infrastructure Policy Enforcer ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml) | N/A | [![.github/workflows/beam_Infrastructure_PolicyEnforcer.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml?query=event%3Aschedule) |
 | [ Modify the GCP User Roles according to the infra/users.yml file ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml) | N/A | [![.github/workflows/beam_Infrastructure_UsersPermissions.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml?query=event%3Aschedule) |
 | [ Service Account Keys Management ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml) | N/A | [![.github/workflows/beam_Infrastructure_ServiceAccountKeys.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml?query=event%3Aschedule) |
+| [ Unmanaged Service Accounts Keys Audit ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml) | N/A | [![.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml?query=event%3Aschedule) |

.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml

@@ -0,0 +1,69 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This workflow works with the GCP security log analyzer to
+# generate weekly security reports and initialize log sinks
+
+name: Unmanaged Service Accounts Keys Audit
+
+on:
+    workflow_dispatch:
+    schedule:
+      # Every day at 00:00 UTC
+      - cron: '0 0 * * *'
+
+concurrency:
+  group: ${{ github.workflow }}
+  cancel-in-progress: false
+
+permissions:
+  contents: read
+  issues: write
+  id-token: write
+
+jobs:
+  beam_UnmanagedKeysAudit:
+    name: Audit Unmanaged Service Account Keys
+    runs-on: [self-hosted, ubuntu-24.04, main]
+    timeout-minutes: 30
+    steps:
+    - uses: actions/checkout@v6
+
+    - name: Setup gcloud
+      uses: google-github-actions/setup-gcloud@aa5489c8933f4cc7a4f7d45035b3b1440c9c10db
+
+    - name: Setup Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.13'
+
+    - name: Install Python dependencies
+      working-directory: ./infra/enforcement
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+
+    - name: Run Unmanaged Service Account Keys Audit
+      working-directory: ./infra/enforcement
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        GITHUB_REPOSITORY: ${{ github.repository }}
+      run: python account_keys.py --action announce
+
+
+
+

infra/enforcement/README.md

@@ -134,8 +134,9 @@ The enforcement tools are integrated with GitHub Actions to provide automated co
 
 ### Workflow Configuration
 
-The GitHub Actions workflow (`.github/workflows/beam_Infrastructure_PolicyEnforcer.yml`) runs:
-- **Schedule**: Weekly on Mondays at 9:00 AM UTC
+The repository includes workflows for different security domains:
+- **IAM Policy Enforcer** (`.github/workflows/beam_Infrastructure_PolicyEnforcer.yml`): Runs weekly on Mondays at 9:00 AM UTC.
+- **Unmanaged Keys Audit** (`.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml`): Runs daily at 00:00 UTC. It manages the continuous execution of the `account_keys.py` script to swiftly detect rogue service account keys generated outside the official rotation system.
 - **Manual trigger**: Can be triggered manually via `workflow_dispatch`
 - **Actions**: Runs both IAM and Account Keys enforcement with the `announce` action
 
@@ -170,7 +171,9 @@ python account_keys.py --action generate
 ### Actions
 
 - **check**: Validates service account keys and their permissions against defined policies and reports any differences (default behavior)
-- **announce**: Creates or updates a GitHub issue and sends an email notification when service account keys policies differ from the defined ones. If no open issue exists, creates a new one; if an open issue exists, updates the issue body with current violations
+- **announce**: Creates or updates a GitHub issue and sends an email notification when service account keys policies differ from the defined ones.
+  - For general configuration errors, it updates the main compliance issue.
+  - **For unmanaged/rogue keys (Security Alerts)**, it consolidates alerts into a dedicated `[SECURITY]` issue acting as a live dashboard. It updates the issue by placing the newest audit report at the top and moving the previous reports into a collapsed `<details>` history section. If the keys are revoked and the infrastructure becomes healthy, the system automatically resolves and closes the issue.
 - **print**: Prints announcement details for testing purposes without creating actual GitHub issues or sending emails
 - **generate**: Updates the compliance file to match the current GCP service account keys and Secret Manager permissions
 
@@ -183,6 +186,9 @@ The Account Keys enforcement tool provides the following capabilities:
 - **Permission Validation**: Ensures that Secret Manager permissions match the declared authorized users
 - **Compliance Reporting**: Identifies missing service accounts, undeclared managed secrets, and permission mismatches
 - **Automatic Remediation**: Can automatically update the compliance file to match current infrastructure state
+- **Unmanaged Key Detection**: Identifies physical keys in IAM that were created outside the automated rotation system by comparing them against the legal, managed keys registered in Google Cloud Secret Manager.
+- **Stateful Security Alerts**: Consolidates security violations into a single, dynamically updated GitHub issue, keeping a collapsed `<details>` history of past scans to prevent alert fatigue.
+- **Self-Healing Resolution**: Automatically closes the security GitHub issue when all unmanaged keys have been successfully revoked.
 
 ### Configuration