Skip to content

Add fix for #151 - Google hourly / annual data mixup#156

Open
mrchrisadams wants to merge 1 commit into
Green-Software-Foundation:mainfrom
thegreenwebfoundation:ca-issue-151-check-gcp-annual-hourly-data
Open

Add fix for #151 - Google hourly / annual data mixup#156
mrchrisadams wants to merge 1 commit into
Green-Software-Foundation:mainfrom
thegreenwebfoundation:ca-issue-151-check-gcp-annual-hourly-data

Conversation

@mrchrisadams

@mrchrisadams mrchrisadams commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

The other week, one thing we identified an issue with the data relating to annual and hourly carbon intensity figures from Google. These were lower that I think the hourly data suggests, and there were also new cloud regions added to the published google csv file in December that were not in this dataset before.

This PR proposes a fix for the numbers that were likely wrong, and also includes the new data.

But before it's merged in, it likely makes sense to ask @VincentPoncet about what the annual claim that Google makes is, as I have tried to capture the detail. I'd want a sense check from someone closer to the data, and he's joined the WG meetings before.

If there is not a per-region annual matching claim like this, it might make more sense to leave the data blank, so I've set a constant GOOGLE_ANNUAL_MATCHING_CLAIM that we can set to an empty / None value if need be.

A machine generated summary of the changes is below.


This pull request updates the GCP data update script, improves documentation, and refreshes the GCP region data for 2024 in Cloud_Region_Metadata.csv. The main focus is to clarify and correctly apply Google's annual renewable energy matching claim, distinguish between hourly and annual CFE values, and improve testability and documentation.

Key changes:

GCP Data Handling and Script Updates

  • Renamed the GCP data update script from gcp-data-update.py to gcp_data_update.py throughout the codebase and documentation for consistency and Python module naming conventions. [1] [2] [3] [4]
  • Updated the script logic to:
    • Use Google's global annual renewable energy matching claim (provider-cfe-annual = 1.0) for all GCP regions, reflecting Google's public sustainability statements. [1] [2]
    • Clearly separate and update both provider-cfe-hourly (region-specific) and provider-cfe-annual (fleet-wide claim) in both data normalization and metadata update logic. [1] [2] [3] [4] [5] [6] [7]

Data and CSV Updates

  • Refreshed all 2024 GCP region entries in Cloud_Region_Metadata.csv:
    • Set provider-cfe-annual to 1.0 for all regions.
    • Updated provider-cfe-hourly values as per the latest data.
    • Added new regions and updated carbon intensity and location fields where applicable.

Documentation and Testing Improvements

  • Expanded the README.md with:
    • Updated script names and usage examples to match the renamed script. [1] [2] [3]
    • Detailed sections on two approaches for testing: legacy shell script and new pytest-based tests for the GCP update script, including setup and example output. [1] [2]

These changes ensure the project reflects the latest GCP sustainability data and claims, improves maintainability and clarity, and enhances test coverage for future updates.

…ta mixup

Add distinguish between the kinds of CFE in the gcp import script, and add tests
@adrianco

Copy link
Copy Markdown
Contributor

Verified the data fix against Google's source (region-carbon-info 2024.csv): provider-cfe-hourly now matches the authoritative Google CFE column for all 44 regions (0 mismatches), europe-west10 lands at hourly 0.68 / annual 1.0 exactly as the issue describes, and the new Dec-2024 regions (me-central1, us-central2, us-east5) are picked up. The hourly/annual mixup is correctly resolved. ✅

One change to request before merge, on the provider-cfe-annual = 1.0 part: the WG position is that the global 100% annual-matching claim shouldn't populate the per-region CFE column — it nets renewable purchases across electrically-unconnected grids (AWS makes the same claim), so it isn't useful as per-region data. We want only the per-region physical figure (provider-cfe-hourly).

Concretely: set your GOOGLE_ANNUAL_MATCHING_CLAIM constant to None so provider-cfe-annual is left blank, keeping the per-region hourly values untouched. That's exactly the escape hatch you built in for this, so no structural change needed. (No per-region annual figure is published by Google, so blank is the right call rather than a per-region number.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants