Skip to content

fix: add missing csv / json files to build#903

Merged
benoit-cty merged 9 commits into
masterfrom
patch/add_cpu_power_csv_to_build
Jul 15, 2025
Merged

fix: add missing csv / json files to build#903
benoit-cty merged 9 commits into
masterfrom
patch/add_cpu_power_csv_to_build

Conversation

@SaboniAmine

@SaboniAmine SaboniAmine commented Jul 13, 2025

Copy link
Copy Markdown
Member

Description

This PR adds a section to the pyproject.toml file to add missing csv files to the build.

Related Issue

Fixes #902

Motivation and Context

Why is this change required? What problem does it solve?

How Has This Been Tested?

The produced wheel has the csv files present in it

Screenshots (if appropriate):

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

Go over all the following points, and put an x in all the boxes that apply.

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING.md document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@SaboniAmine SaboniAmine requested review from benoit-cty and inimaz July 13, 2025 14:13
@benoit-cty

Copy link
Copy Markdown
Contributor

Changes Made

1. Updated Package Data Configuration (pyproject.toml)

  • Added cpu_power.csv to ensure CPU power data is included
  • Added data/**/*.geojson to include geographic data files, for carbonboard.
  • This ensures all critical data files identified in the DataSource class are packaged

2. Created Package Integrity Test (test_package_integrity.py)

  • Tests that all critical data files are accessible from the installed package
  • Validates that data can actually be read (not just that files exist)
  • Tests basic functionality like EmissionsTracker instantiation
  • Tests CLI import capabilities
  • Tests core module imports

3. Added New Workflow Job (test-package-from-wheel)

  • Runs without source code: Uses sparse checkout to only get test files, then installs from wheel
  • Clean environment: Creates isolated virtual environment for each Python version
  • Package-only testing: Installs built wheel without access to source code
  • Comprehensive validation: Runs integrity tests and CLI functionality tests
  • Multi-version testing: Tests on Python 3.9, 3.12, and 3.13

4. Added Task Configuration

  • Added test-package-integrity task for running just the integrity tests
  • This can be used locally with uv run task test-package-integrity

5. Renamed Existing Job

  • Renamed python-test to python-test-with-source for clarity
  • This clearly distinguishes between source-based testing and package-based testing

Key Benefits

  1. True Package Validation: The test-package-from-wheel job installs only the built wheel, ensuring tests run against what users will actually install
  2. Data File Verification: Explicit tests verify that all critical data files are accessible and readable
  3. No Source Code Access: Uses sparse checkout and clean environments to prevent accidental reliance on source code
  4. Comprehensive Coverage: Tests both functionality and data accessibility
  5. Early Detection: Will catch missing data files before they reach users

How It Works

  1. Build Phase: Creates the wheel package with all necessary data files
  2. Package Test Phase:
    • Downloads only the built wheel and minimal test files
    • Creates clean virtual environment
    • Installs wheel without source code
    • Runs comprehensive integrity tests
    • Validates CLI functionality
  3. Source Test Phase: Traditional testing with full source code access for development

This ensures that if any critical data file is missing from the package configuration, the CI will fail before the package is released, preventing issues for end users who install via pip.

@inimaz inimaz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good thanks!

@benoit-cty benoit-cty force-pushed the patch/add_cpu_power_csv_to_build branch from f3be7df to 343f611 Compare July 15, 2025 09:02
benoit-cty and others added 8 commits July 15, 2025 11:05
wip : packaging
fux indent

fix indent

Remove matrix

Fix artifact version

fix venv

fix tests files

Add missing files

Change master pipeline
cleaning
@benoit-cty benoit-cty force-pushed the patch/add_cpu_power_csv_to_build branch from 343f611 to 63958e4 Compare July 15, 2025 09:05
@benoit-cty benoit-cty merged commit a30fe4d into master Jul 15, 2025
11 checks passed
@benoit-cty benoit-cty deleted the patch/add_cpu_power_csv_to_build branch July 15, 2025 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FileNotFoundError initializing EmissionsTracker on Colab (missing cpu_power.csv)

3 participants