-
Notifications
You must be signed in to change notification settings - Fork 33
Create GitHub Action script to prevent API v1 deployment if v1 and v2 model versions don't align #2534
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create GitHub Action script to prevent API v1 deployment if v1 and v2 model versions don't align #2534
Changes from all commits
2aaf068
ca80156
3b7519d
3c46841
81ba34e
3464a7f
1136b66
58c8328
f38bf21
d523215
157b425
ba5c9ce
f8235b9
6f8804e
6c5d164
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| import os | ||
| import sys | ||
| from policyengine_api.constants import COUNTRY_PACKAGE_VERSIONS | ||
|
|
||
|
|
||
| def find_api_model_versions_and_output_to_github(): | ||
| """ | ||
| Find the API model versions and output them to a file for GitHub. | ||
| """ | ||
| # Try to get package versions for US and UK | ||
| us_version = COUNTRY_PACKAGE_VERSIONS.get("us") | ||
| uk_version = COUNTRY_PACKAGE_VERSIONS.get("uk") | ||
|
|
||
| if not us_version: | ||
| print("Error: US package version not found.", file=sys.stderr) | ||
| sys.exit(1) | ||
|
|
||
| if not uk_version: | ||
| print("Error: UK package version not found.", file=sys.stderr) | ||
| sys.exit(1) | ||
|
|
||
| # Write to GitHub Actions environment | ||
| with open(os.environ["GITHUB_ENV"], "a") as f: | ||
| f.write(f"US_VERSION={us_version}\n") | ||
| f.write(f"UK_VERSION={uk_version}\n") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| find_api_model_versions_and_output_to_github() | ||
| print("API model versions found and written to GitHub environment.") | ||
| sys.exit(0) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| #! /usr/bin/env bash | ||
|
|
||
| set -e | ||
|
|
||
| apt-get update | ||
| apt-get install -y jq |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,10 +3,10 @@ | |
| set -e | ||
|
|
||
| # Google Cloud Workflow execution script | ||
| # Usage: ./wait_for_country_versions.sh -b <bucket_name> -us <us_version> -uk <uk_version> [-t timeout] [-i interval] | ||
| # Usage: ./request-simulation-model-versions.sh -b <bucket_name> -us <us_version> -uk <uk_version> [-t timeout] [-i interval] | ||
|
|
||
| usage() { | ||
| echo "Usage: $0 -b <bucket_name> -us <us_version> -uk <uk_version> [-t timeout] [-i interval]" | ||
| echo "Usage: $0 -us <us_version> -uk <uk_version> [-t timeout] [-i interval]" | ||
| echo "" | ||
| echo "Required flags:" | ||
| echo " -b bucket_name - GCS bucket name" | ||
|
|
@@ -19,8 +19,8 @@ usage() { | |
| echo " -h help - Show this help message" | ||
| echo "" | ||
| echo "Example:" | ||
| echo " $0 -b my-bucket -us v1.2.3 -uk v1.2.4" | ||
| echo " $0 -b my-bucket -us v1.2.3 -uk v1.2.4 -t 600 -i 15" | ||
| echo " $0 -us v1.2.3 -uk v1.2.4" | ||
| echo " $0 -us v1.2.3 -uk v1.2.4 -t 600 -i 15" | ||
| exit 1 | ||
| } | ||
|
|
||
|
|
@@ -34,6 +34,9 @@ CHECK_INTERVAL="10" | |
| # Parse command line arguments | ||
| while [ $# -gt 0 ]; do | ||
| case "$1" in | ||
| -h|--help) | ||
| usage | ||
| ;; | ||
| -b) | ||
| if [ -z "$2" ]; then | ||
| echo "Error: -b requires a bucket name" | ||
|
|
@@ -85,7 +88,7 @@ while [ $# -gt 0 ]; do | |
| done | ||
|
|
||
| # Validate required arguments | ||
| if [ -z "$BUCKET_NAME" ] || [ -z "$US_VERSION" ] || [ -z "$UK_VERSION" ]; then | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggesiton, please don't block - this is my fault, but we should configure the workflow with the bucket as an environment variable instead of an input and then set that env variable in terraform as part of deploying it. Making a v1 deployment script hardcodes an internal detail like the bucket we store our metadata in (I.e. now it knows there is a bucket and which one) is just a whole lot of coupling that doesn't need to exist.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This makes a lot of sense. I think you may have been reviewing this at an earlier point, as I modified the code to not even ask for a bucket name, which I suppose is even further away from what we want. Being that we don't use Terraform to deploy API v1 at the moment, I'd prefer to punt this issue, but noted - in API v2-related scripts like this, I will set via Terraform as part of deploy. |
||
| if [ -z "$BUCKET_NAME" ] || -z "$US_VERSION" ] || [ -z "$UK_VERSION" ]; then | ||
| echo "Error: Missing required arguments" | ||
| echo "bucket_name (-b), us_version (-us), and uk_version (-uk) are required" | ||
| usage | ||
|
|
@@ -98,15 +101,10 @@ if ! [[ "$TIMEOUT_SECONDS" =~ ^[0-9]+$ ]] || ! [[ "$CHECK_INTERVAL" =~ ^[0-9]+$ | |
| fi | ||
|
|
||
| # Configuration | ||
| PROJECT_ID="${GOOGLE_CLOUD_PROJECT:-$(gcloud config get-value project 2>/dev/null)}" | ||
| WORKFLOW_LOCATION="${WORKFLOW_LOCATION:-us-central1}" | ||
| PROJECT_ID="prod-api-v2-c4d5" | ||
| WORKFLOW_LOCATION="us-central1" | ||
| WORKFLOW_NAME="wait-for-country-packages" | ||
|
|
||
| if [ -z "$PROJECT_ID" ]; then | ||
| echo "Error: Could not determine project ID. Set GOOGLE_CLOUD_PROJECT environment variable." | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Starting workflow execution..." | ||
| echo "Project: $PROJECT_ID" | ||
| echo "Location: $WORKFLOW_LOCATION" | ||
|
|
@@ -134,6 +132,7 @@ echo "Input: $INPUT_JSON" | |
| # Execute workflow | ||
| echo "Executing workflow..." | ||
| EXECUTION_RESULT=$(gcloud workflows execute "$WORKFLOW_NAME" \ | ||
| --project="$PROJECT_ID" \ | ||
| --location="$WORKFLOW_LOCATION" \ | ||
| --data="$INPUT_JSON" \ | ||
| --format="json") | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| - bump: minor | ||
| changes: | ||
| added: | ||
| - GitHub Actions checks that full API and simulation API model versions are in sync |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
question This works because we install everything for v1 globally in the build host? (i.e. so when we run it as an action it's pulling the same version of policyengine_api.constants that is in the docker container)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. This is done via the install step added before
find-api-model-versions.pyis invoked.