Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/find-api-model-versions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import os
import sys
from policyengine_api.constants import COUNTRY_PACKAGE_VERSIONS


def find_api_model_versions_and_output_to_github():

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question This works because we install everything for v1 globally in the build host? (i.e. so when we run it as an action it's pulling the same version of policyengine_api.constants that is in the docker container)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. This is done via the install step added before find-api-model-versions.py is invoked.

"""
Find the API model versions and output them to a file for GitHub.
"""
# Try to get package versions for US and UK
us_version = COUNTRY_PACKAGE_VERSIONS.get("us")
uk_version = COUNTRY_PACKAGE_VERSIONS.get("uk")

if not us_version:
print("Error: US package version not found.", file=sys.stderr)
sys.exit(1)

if not uk_version:
print("Error: UK package version not found.", file=sys.stderr)
sys.exit(1)

# Write to GitHub Actions environment
with open(os.environ["GITHUB_ENV"], "a") as f:
f.write(f"US_VERSION={us_version}\n")
f.write(f"UK_VERSION={uk_version}\n")


if __name__ == "__main__":
find_api_model_versions_and_output_to_github()
print("API model versions found and written to GitHub environment.")
sys.exit(0)
6 changes: 6 additions & 0 deletions .github/install-jq.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#! /usr/bin/env bash

set -e

apt-get update
apt-get install -y jq
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
set -e

# Google Cloud Workflow execution script
# Usage: ./wait_for_country_versions.sh -b <bucket_name> -us <us_version> -uk <uk_version> [-t timeout] [-i interval]
# Usage: ./request-simulation-model-versions.sh -b <bucket_name> -us <us_version> -uk <uk_version> [-t timeout] [-i interval]

usage() {
echo "Usage: $0 -b <bucket_name> -us <us_version> -uk <uk_version> [-t timeout] [-i interval]"
echo "Usage: $0 -us <us_version> -uk <uk_version> [-t timeout] [-i interval]"
echo ""
echo "Required flags:"
echo " -b bucket_name - GCS bucket name"
Expand All @@ -19,8 +19,8 @@ usage() {
echo " -h help - Show this help message"
echo ""
echo "Example:"
echo " $0 -b my-bucket -us v1.2.3 -uk v1.2.4"
echo " $0 -b my-bucket -us v1.2.3 -uk v1.2.4 -t 600 -i 15"
echo " $0 -us v1.2.3 -uk v1.2.4"
echo " $0 -us v1.2.3 -uk v1.2.4 -t 600 -i 15"
exit 1
}

Expand All @@ -34,6 +34,9 @@ CHECK_INTERVAL="10"
# Parse command line arguments
while [ $# -gt 0 ]; do
case "$1" in
-h|--help)
usage
;;
-b)
if [ -z "$2" ]; then
echo "Error: -b requires a bucket name"
Expand Down Expand Up @@ -85,7 +88,7 @@ while [ $# -gt 0 ]; do
done

# Validate required arguments
if [ -z "$BUCKET_NAME" ] || [ -z "$US_VERSION" ] || [ -z "$UK_VERSION" ]; then

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggesiton, please don't block - this is my fault, but we should configure the workflow with the bucket as an environment variable instead of an input and then set that env variable in terraform as part of deploying it.

Making a v1 deployment script hardcodes an internal detail like the bucket we store our metadata in (I.e. now it knows there is a bucket and which one) is just a whole lot of coupling that doesn't need to exist.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes a lot of sense. I think you may have been reviewing this at an earlier point, as I modified the code to not even ask for a bucket name, which I suppose is even further away from what we want. Being that we don't use Terraform to deploy API v1 at the moment, I'd prefer to punt this issue, but noted - in API v2-related scripts like this, I will set via Terraform as part of deploy.

if [ -z "$BUCKET_NAME" ] || -z "$US_VERSION" ] || [ -z "$UK_VERSION" ]; then
echo "Error: Missing required arguments"
echo "bucket_name (-b), us_version (-us), and uk_version (-uk) are required"
usage
Expand All @@ -98,15 +101,10 @@ if ! [[ "$TIMEOUT_SECONDS" =~ ^[0-9]+$ ]] || ! [[ "$CHECK_INTERVAL" =~ ^[0-9]+$
fi

# Configuration
PROJECT_ID="${GOOGLE_CLOUD_PROJECT:-$(gcloud config get-value project 2>/dev/null)}"
WORKFLOW_LOCATION="${WORKFLOW_LOCATION:-us-central1}"
PROJECT_ID="prod-api-v2-c4d5"
WORKFLOW_LOCATION="us-central1"
WORKFLOW_NAME="wait-for-country-packages"

if [ -z "$PROJECT_ID" ]; then
echo "Error: Could not determine project ID. Set GOOGLE_CLOUD_PROJECT environment variable."
exit 1
fi

echo "Starting workflow execution..."
echo "Project: $PROJECT_ID"
echo "Location: $WORKFLOW_LOCATION"
Expand Down Expand Up @@ -134,6 +132,7 @@ echo "Input: $INPUT_JSON"
# Execute workflow
echo "Executing workflow..."
EXECUTION_RESULT=$(gcloud workflows execute "$WORKFLOW_NAME" \
--project="$PROJECT_ID" \
--location="$WORKFLOW_LOCATION" \
--data="$INPUT_JSON" \
--format="json")
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,14 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies (required for finding API model versions)
run: make install
Comment thread
anth-volk marked this conversation as resolved.
- name: Install jq (required only for GitHub Actions)
run: ".github/install-jq.sh"
- name: Find API model versions and write to environment variable
run: python3 .github/find-api-model-versions.py
- name: Ensure full API and simulation API model versions are in sync
run: ".github/request-simulation-model-versions.sh -b prod-api-v2-c4d5-metadata -us ${{ env.US_VERSION }} -uk ${{ env.UK_VERSION }}"
- name: GCP authentication
uses: "google-github-actions/auth@v2"
with:
Expand Down
4 changes: 4 additions & 0 deletions changelog_entry.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
- bump: minor
changes:
added:
- GitHub Actions checks that full API and simulation API model versions are in sync
Loading