Add function for gas tariff mapping#146
Conversation
jpvelez
left a comment
jpvelez
left a comment
There was a problem hiding this comment.
Mostly just the same fixes as in the electric tariff mapper
| @@ -0,0 +1,11 @@ | |||
| # Resolve project root as absolute path from this Justfile's location (works for any clone) | |||
| project_root := absolute_path(justfile_directory() / ".." / ".." / ".." / ".." / "..") | |||
There was a problem hiding this comment.
Same fix as in the other PR
| @@ -0,0 +1,11 @@ | |||
| # Resolve project root as absolute path from this Justfile's location (works for any clone) | |||
| project_root := absolute_path(justfile_directory() / ".." / ".." / ".." / ".." / "..") | |||
| data_base := "s3://data.sb/nrel/resstock/res_2024_amy2018_2/metadata" | |||
There was a problem hiding this comment.
Same fix as in the other PR
| from utils.types import electric_utility | ||
|
|
||
| # Project root (rate-design-platform); independent of cwd or caller | ||
| _PROJECT_ROOT = Path(__file__).resolve().parent.parent |
There was a problem hiding this comment.
Same fix as in the other PR
|
|
||
|
|
||
| def map_gas_tariff( | ||
| SB_metadata_path: S3Path, |
There was a problem hiding this comment.
Same fix as in the other PR
| .filter(pl.col("sb.electric_utility") == electric_utility_name) | ||
| .collect() | ||
| ) | ||
| if utility_metadata_df.is_empty(): |
There was a problem hiding this comment.
Same fix as in the other PR
| # For now, we will manually add the electric utility name column. Later on, the metadata parquet will be updated to include this column. | ||
| # Assign first ~1/3 to Coned, next ~1/3 to National Grid, last ~1/3 to NYSEG. | ||
| metadata_path = S3Path(args.metadata_path) | ||
| metadata_df = pl.read_parquet(io.BytesIO(metadata_path.read_bytes())) |
There was a problem hiding this comment.
Same fix as in the other PR
| n = len(metadata_df) | ||
| # Create electricity utility name column | ||
| metadata_df = ( | ||
| metadata_df.with_row_index("_row_idx") |
There was a problem hiding this comment.
Same fix as in the other PR
| raise FileNotFoundError(f"SB metadata path {SB_metadata_path} does not exist") | ||
|
|
||
| utility_metadata_df = ( | ||
| pl.scan_parquet(io.BytesIO(SB_metadata_path.read_bytes())) |
There was a problem hiding this comment.
Same fix as in the other PR
| utility_metadata_df.select(pl.col("bldg_id", "sb.gas_utility")) | ||
| .with_columns( | ||
| pl.when(pl.col("sb.gas_utility") == "National Grid") | ||
| .then(pl.lit("National Grid")) |
There was a problem hiding this comment.
Obviously, this will need to be generalized, but it's fine for now.
However, the since the tariff_keys will get spliced into filenames, they need to be lowercase_with_underscore.
| ) | ||
| if not output_path.parent.exists(): | ||
| output_path.parent.mkdir(parents=True) | ||
| gas_tariff_mapping_df.write_csv(output_path) |
There was a problem hiding this comment.
Same fix as in the other PR
| --state NY \ | ||
| --upgrade_id "{{upgrade_id}}" \ | ||
| --electric_utility "{{electric_utility}}" \ | ||
| {{ if output_dir != "" { "--output_dir \"" + output_dir + "\"" } else { "" } }} |
There was a problem hiding this comment.
I was allowing output_dir to be an optional input. Changed in the latest commit to make mandatory.
|
|
||
| # Some useful enumerations: | ||
|
|
||
| map-coned-default: |
There was a problem hiding this comment.
there's just 1 utility in rhode island—Rhode Island Energy (RIE)
There was a problem hiding this comment.
These values correspond to the electrical utility, not the gas utility. We take electrical_utility_name as input here, filter out the bldg_id's that correspond to the provided electrical utility, then assign gas tariff_key's based on the gas_utility column value in the metadata-sb.parquet
| just map-electric-tariff Coned class_specific_seasonal 1 00 | ||
|
|
||
| map-national-grid-default: | ||
| just map-gas-tariff National Grid 00 |
There was a problem hiding this comment.
This is going to cause a syntax error... should be "National Grid" if it has whitespace
There was a problem hiding this comment.
Addressed in the latest commit
| _PROJECT_ROOT = Path(__file__).resolve().parent.parent | ||
| RATE_DESIGN_DIR = _PROJECT_ROOT / "rate_design" | ||
|
|
||
| AWS_REGION = "us-west-2" |
There was a problem hiding this comment.
This really shouldn't be hardcoded... the aws tools should be able to pick this up from the environment.
There was a problem hiding this comment.
Addressed in the latest commit
| raise FileNotFoundError(f"Metadata path {metadata_path} does not exist") | ||
| SB_metadata_lazy_df = pl.scan_parquet(str(metadata_path)) | ||
|
|
||
| SB_metadata_lazy_df_with_utilities = SB_metadata_lazy_df.with_columns( |
There was a problem hiding this comment.
no need for "_lazy_df" type stuff in the variable names... we have type annotations and type inference in IDEs.
There was a problem hiding this comment.
Addressed in the latest commit
| try: | ||
| out_base = S3Path(args.output_dir) | ||
| output_path = out_base / output_filename | ||
| if not output_path.parent.exists(): |
There was a problem hiding this comment.
No need for this, and therefore for S3Path. It can just fail if the directory is missing and then the user knows to make it.
There was a problem hiding this comment.
Addressed in the latest commit
| gas_tariff_mapping_df.sink_csv(str(output_path)) | ||
| else: | ||
| output_path = ( | ||
| RATE_DESIGN_DIR |
There was a problem hiding this comment.
Get rid of this (and of RATE_DESIGN_DIR) and fail if output path isn't provided
There was a problem hiding this comment.
Addressed in the latest commit
* Add skeleton code for gas tariff mapper * Finish python script for gas tariff key mapping * Clean up python code for gas tariff key mapping * Add Justfile command for gas tariff_key mapping * Address comments and reformat scripts * Address comments and reformat scripts * Add some useful enumerations for just commands * Fix formatting in justfiles * Remove relational path defining and fix justfile enumerations * Address comments and reformat scripts
2 participants
Summary
This PR adds python script and function that generates gas tariff mapping for bldg_id's based on electrical_utility and gas_utility
Closes #138