-
Notifications
You must be signed in to change notification settings - Fork 38
Feature: EIA Natural Gas Price Retrieval for Feedstocks #719
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
johnjasa
merged 91 commits into
NatLabRockies:develop
from
RHammond2:feature/ng-industrial-price
May 21, 2026
Merged
Changes from all commits
Commits
Show all changes
91 commits
Select commit
Hold shift + click to select a range
b6de770
add unintegrated, working EIA API data querier
RHammond2 f0898f5
probably working integration into H2I
RHammond2 194f592
ensure data are always in monthly format
RHammond2 80987d2
add openmdao
RHammond2 aaa333a
move feedstocks to its own folder
RHammond2 f8fd1be
move natural gas pricing to feedstocks and convert config for compliance
RHammond2 6e978f6
update long comment and start building out NG cost model
RHammond2 52a5bcb
update compute docstring and include feedstocks folder in api docs
RHammond2 17b62dd
update inline comment usage
RHammond2 febfa21
convert price to hourly over full year and update comments/docstrings
RHammond2 6cf2eb7
Merge branch 'develop' into feature/ng-industrial-price
RHammond2 fcf54c5
remove placeholder for lint ignore
RHammond2 7675156
ensure price timeseries matches n_timesteps
RHammond2 829835d
add leap year handling for clean 8760s
RHammond2 d5baed6
update changelog
RHammond2 adbb8d8
fix typos, make api_key_file consistent, and enable environment varia…
RHammond2 2cbcfdc
update docstrings and spacing
RHammond2 ddba1d1
add the EIA model to the docs page
RHammond2 d714c6d
add requests instead of relying on other packages requiring it
RHammond2 786bd0e
add to supported models
RHammond2 789a417
update class hierarchy
RHammond2 5464b49
fix import
RHammond2 78e18e9
fix typo
RHammond2 f5c068f
better column handling for price
RHammond2 aa0f68c
add compliant performance model to accompany the cost model
RHammond2 75bab15
update changelog
RHammond2 db24117
update supported models
RHammond2 db00870
remove lingering resource reference
RHammond2 319c1f5
update column filtering from previous modification
RHammond2 2f48312
remove deprecated arg
RHammond2 4a5f199
fix miscellaneous data handling issues
RHammond2 6da03da
ensure saved data is compatible with reload
RHammond2 df2f837
fix final data handling issue
RHammond2 8f39425
update changelog for pr number
RHammond2 605c1d0
fix conflict and merge develop
RHammond2 25676f9
Merge branch 'develop' into feature/ng-industrial-price
RHammond2 cb56bb6
add latitude and longitude, and handle no api key file input
RHammond2 05a4866
add reverse geocoding
RHammond2 3794a6e
update docs for reverse geocoding
RHammond2 8e37cd9
add back correct annual reindex logic
RHammond2 1cad6b1
add test for all but feedstock cost model and fix minor issues
RHammond2 67c2fb9
add missing docstring
RHammond2 68070f3
add test for configuration class and fix class issues
RHammond2 c0074f2
Merge branch 'develop' into feature/ng-industrial-price
RHammond2 e1e9e45
Merge branch 'develop' into feature/ng-industrial-price
RHammond2 5379114
merge develop and fix conflicts
RHammond2 34b1e94
Merge branch 'develop' into feature/ng-industrial-price
RHammond2 989cc8e
move the base functionality to separate preprocessing modules
RHammond2 6f57b50
reinstate more descriptive todo comment
RHammond2 9c4d10d
remove custom performance model
RHammond2 766506d
remove deprecated model
RHammond2 27637a2
add ability for setup to be used by subclass even if it uses its own …
RHammond2 539a2b4
continue to split out functionality to preprocessing
RHammond2 7270904
split out most of data gathering
RHammond2 22e611d
Merge branch 'develop' into feature/ng-industrial-price
RHammond2 b7ff323
fix control flow and extra comma
RHammond2 76e6c8a
add argument to function
RHammond2 75a1038
make monthly conversion robust enough for multi-index and add state c…
RHammond2 b0c9e7e
check for state being kept
RHammond2 54e9b43
move monthly to 8760 conversion to eia preprocessing
RHammond2 4878186
remove duplicitous code
RHammond2 2d980fa
change file name and update single site compatibility
RHammond2 4b14c14
update module name in docs
RHammond2 aa46381
update state retrieval
RHammond2 7c0fc67
add docstring
RHammond2 b2bfd7c
move geospatial tests to preprocessing
RHammond2 7176e41
move eia api testing to preprocessing and add more tests
RHammond2 819b84f
update default file handling and state/category sort order
RHammond2 85f03f3
remove pre-return
RHammond2 fa2dcff
add data download test
RHammond2 8b0886e
update feedstock config test and fix reverse geocode issue in config
RHammond2 8c80f60
add hourly conversion test and fix issues in function
RHammond2 37bc358
Merge branch 'develop' into feature/ng-industrial-price
RHammond2 af803d2
update feedstock cost model checks
RHammond2 e055518
udpate docs
RHammond2 55c8735
update changelog
RHammond2 12af9ec
fix str-None comparison
RHammond2 cd7a5d6
add the none default to get data function
RHammond2 5031d23
fix ci test issues
RHammond2 669e2df
remove extraneous note
RHammond2 38250e1
add site_name and handling
RHammond2 2577afd
add additional site handling
RHammond2 cadf021
add user warning for modified price
RHammond2 9dbd7d5
add feedstock model tests
RHammond2 aacc818
be specific about us-states when applicable
RHammond2 39de377
add handling for when no geocoder available
RHammond2 e421d20
reinstate original file handler
RHammond2 1079c10
Merging
johnjasa 1188758
update feedstock import pathway and use subpackage level import, not …
RHammond2 aeacc9a
minor warning change
johnjasa 1c2c699
Merge branch 'feature/ng-industrial-price' of https://github.com/RHam…
johnjasa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,6 +8,7 @@ | |
| :recursive: | ||
|
|
||
| core | ||
| feedstocks | ||
| converters | ||
| control | ||
| finances | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| from h2integrate.feedstocks.feedstocks import FeedstockPerformanceModel, FeedstockCostModel | ||
| from h2integrate.feedstocks.eia_ng_price import EIANaturalGasFeedstockCostModel |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,174 @@ | ||
| import warnings | ||
| from pathlib import Path | ||
| from datetime import datetime | ||
|
|
||
| import attrs | ||
| import numpy as np | ||
| import pandas as pd | ||
| from attrs import field, define | ||
|
|
||
| from h2integrate.preprocess import eia, geospatial | ||
| from h2integrate.core.utilities import merge_shared_inputs | ||
| from h2integrate.core.file_utils import get_path | ||
| from h2integrate.core.validators import range_val | ||
| from h2integrate.feedstocks.feedstocks import FeedstockCostModel | ||
| from h2integrate.core.model_baseclasses import BaseConfig | ||
|
|
||
|
|
||
| HOURS_PER_YEAR = 8760 | ||
| SECONDS_PER_HOUR = 3600 | ||
| CURRENT_YEAR = datetime.now().year | ||
|
|
||
| default_price = pd.DataFrame( | ||
| np.zeros(8760, dtype=float).reshape(-1, 1), | ||
| columns=["price"], | ||
| index=pd.date_range("2001-01-01", "2001-12-31 23:00:00", freq="h"), | ||
| ) | ||
|
|
||
|
|
||
| @define | ||
| class EIANaturalGasFeedstockConfig(BaseConfig): | ||
| """EIA Industrial Natural Gas Pricing API configuration and downloader for the US and all 50 US | ||
| states, in $/MCF, converted to $/MMBtu. Please see | ||
| https://www.eia.gov/opendata/browser/natural-gas/pri/sum for further details about data | ||
| availability. | ||
|
|
||
| Args: | ||
| state (str): Full name of the state or two-letter state abbreviation, such as | ||
| "United States" or "US". Only the "US" or all 50 states will produce valid results. | ||
| resource_year (int): The YYYY-format year whose data should be retrieved. Must be between | ||
| 2001 and the current year, inclusive of endpoints. | ||
| cost_year (int): dollar-year for costs. Defaults to the current year. | ||
| monthly (Path): True, if monthly data is desired, False if annual data is desired. | ||
| price_category (str): One of "wellhead", "imports", "citygate", "residential", "commercial", | ||
| "industrial", "electrical_power", or "exports". Note that not all categories will return | ||
| state-level data. | ||
| api_key_file (Path, optional): Full file name of the file where the API key is located. If | ||
| no file name is provided, then the environment variables ``EIA_API_KEY`` is used. | ||
| latitude (float | None): WGS-84 y-coordinate of the site. Will not be used if | ||
| :py:attr:`state` has been provided. | ||
| longitude (float | None): WGS-84 x-coordinate of the site. Will not be used if | ||
| :py:attr:`state` has been provided. | ||
| site_name (str, optional): Name of the site from a multi-site private configuration. When | ||
| provided, the :py:class:`EIANaturalGasFeedstockCostModel` will populate the | ||
| :py:attr:`latitude` and :py:attr:`longitude` from the plant site configuration. | ||
| filename (str, optional): Filename for where to save the data or where the data may | ||
| already be located. If the file exists, the columns "period", "state", and "price" must | ||
| exist, otherwise the file will not be used. "period" should be of the form YYYY or | ||
| YYYY-MM, and state should be either the full state name or the two-letter abbreviation. | ||
| annual_cost (float, optional): fixed cost associated with the feedstock in USD/year. | ||
| Defaults to 0.0. | ||
| start_up_cost (float, optional): one-time capital cost associated with the feedstock in USD. | ||
| Defaults to 0.0. | ||
| """ | ||
|
|
||
| resource_year: int = field(validator=attrs.validators.in_(range(2001, CURRENT_YEAR + 1))) | ||
| monthly: bool = field(validator=attrs.validators.instance_of(bool)) | ||
| price_category: str = field( | ||
| converter=str.lower, validator=attrs.validators.in_(eia.EIA_NG_FACET) | ||
| ) | ||
| api_key_file: str | None = field(default=None, converter=attrs.converters.optional(get_path)) | ||
| state: str = field( | ||
| default=None, | ||
| converter=attrs.converters.optional( | ||
| attrs.converters.pipe(geospatial.convert_state_value, geospatial.convert_state_to_code) | ||
| ), | ||
| validator=attrs.validators.optional( | ||
| attrs.validators.in_([*geospatial.US_STATE_MAP, *geospatial.US_STATE_MAP.values()]) | ||
| ), | ||
| ) | ||
| latitude: float | None = field( | ||
| default=None, validator=attrs.validators.optional(range_val(-90.0, 90.0)) | ||
| ) | ||
| longitude: float | None = field( | ||
| default=None, validator=attrs.validators.optional(range_val(-180.0, 180.0)) | ||
| ) | ||
| site_name: str = field( | ||
| default=None, validator=attrs.validators.optional(attrs.validators.instance_of(str)) | ||
| ) | ||
| cost_year: int = field(default=CURRENT_YEAR) | ||
| annual_cost: float = field(default=0.0, converter=float) | ||
| start_up_cost: float = field(default=0.0, converter=float) | ||
| filename: str = field(default=None) | ||
|
|
||
| commodity: str = field(default="natural_gas", init=False) | ||
| commodity_rate_units: str = field(default="MMBtu/h", init=False) | ||
| commodity_amount_units: str = field(default="MMBtu", init=False) | ||
| price: pd.DataFrame = field( | ||
| default=default_price, init=False, validator=attrs.validators.instance_of(pd.DataFrame) | ||
| ) | ||
|
|
||
| def __attrs_post_init__(self): | ||
| """Creates the EIA natural gas facet series code based on validated user inputs, sets the | ||
| :py:attr:`commodity_amount_units` if not given a value, and fetches the EIA natural gas | ||
| price. | ||
| """ | ||
| if self.filename is not None: | ||
| try: | ||
| self.filename = get_path(self.filename) | ||
| except FileNotFoundError: | ||
| self.filename = Path(self.filename).resolve() | ||
|
|
||
| if self.state is None: | ||
| if self.latitude is None or self.longitude is None: | ||
| msg = ( | ||
| "The EIA natural gas feedstock model require one of `state` or" | ||
| " `latitude` and `longitude`." | ||
| ) | ||
| raise ValueError(msg) | ||
|
|
||
| self.state = geospatial.get_state_from_coords( | ||
| latitude=self.latitude, longitude=self.longitude | ||
| ) | ||
|
|
||
|
|
||
| class EIANaturalGasFeedstockCostModel(FeedstockCostModel): | ||
| """Feedstock cost model based on the EIA natural gas price API results that uses | ||
| annual or monthly data to model an hourly time step for a single year to model the | ||
| price of natural gas used in the model. | ||
|
|
||
| The model will pull the single site data if it is made available to pass the ``latitude`` and | ||
| ``longitude`` from the site configuration, otherwise the ``state`` will need to be provided | ||
| in the configuration. | ||
| """ | ||
|
|
||
| def setup(self): | ||
| """Defines the inputs and outputs of the model and converts the | ||
| :py:attr:`EIANaturalGasFeedstockConfig.price` to an hourly timeseries for the | ||
| ``plant_life``. | ||
|
|
||
| Populates the site latitude and longitude with either the configuration's provided site | ||
| name or the first site in plant configuration's ``sites`` dictionary when a ``state`` is | ||
| not directly provided. | ||
| """ | ||
| site_config = {} | ||
| sites = self.options["plant_config"].get("sites", {}) | ||
| cost_config = merge_shared_inputs(self.options["tech_config"]["model_inputs"], "cost") | ||
|
|
||
| if cost_config.get("state") is None: | ||
| if (site_name := cost_config.get("site_name")) is not None and sites: | ||
| site_config = sites.get(site_name, {}) | ||
| if site_name is None and sites: | ||
| site_config = sites[[*sites][0]] | ||
|
|
||
| self.config = EIANaturalGasFeedstockConfig.from_dict( | ||
| cost_config | site_config, additional_cls_name=self.__class__.__name__, strict=False | ||
| ) | ||
|
|
||
| price = eia.get_eia_ng_data( | ||
| api_key_file=self.config.api_key_file, | ||
| resource_year=self.config.resource_year, | ||
| price_category=self.config.price_category, | ||
| state=self.config.state, | ||
| monthly=self.config.monthly, | ||
| filename=self.config.filename, | ||
| ) | ||
| price = eia.convert_to_hourly(price) | ||
| self.config.price = price | ||
| super().setup() | ||
|
|
||
| def compute(self, inputs, outputs): | ||
| if not np.isclose(inputs["price"], self.config.price, rtol=1e-6): | ||
| warn_msg = "The NG price has changed from EIA price. This may be intended." | ||
| warnings.warn(warn_msg, UserWarning) | ||
| super().compute(inputs, outputs) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.