Skip to content

Commit 4250dd2

Browse files
RHammond2johnjasa
andauthored
Feature: EIA Natural Gas Price Retrieval for Feedstocks (#719)
* add unintegrated, working EIA API data querier * probably working integration into H2I * ensure data are always in monthly format * add openmdao * move feedstocks to its own folder * move natural gas pricing to feedstocks and convert config for compliance * update long comment and start building out NG cost model * update compute docstring and include feedstocks folder in api docs * update inline comment usage * convert price to hourly over full year and update comments/docstrings * remove placeholder for lint ignore * ensure price timeseries matches n_timesteps * add leap year handling for clean 8760s * update changelog * fix typos, make api_key_file consistent, and enable environment variables * update docstrings and spacing * add the EIA model to the docs page * add requests instead of relying on other packages requiring it * add to supported models * update class hierarchy * fix import * fix typo * better column handling for price * add compliant performance model to accompany the cost model * update changelog * update supported models * remove lingering resource reference * update column filtering from previous modification * remove deprecated arg * fix miscellaneous data handling issues * ensure saved data is compatible with reload * fix final data handling issue * update changelog for pr number * add latitude and longitude, and handle no api key file input * add reverse geocoding * update docs for reverse geocoding * add back correct annual reindex logic * add test for all but feedstock cost model and fix minor issues * add missing docstring * add test for configuration class and fix class issues * move the base functionality to separate preprocessing modules * reinstate more descriptive todo comment * remove custom performance model * remove deprecated model * add ability for setup to be used by subclass even if it uses its own config * continue to split out functionality to preprocessing * split out most of data gathering * fix control flow and extra comma * add argument to function * make monthly conversion robust enough for multi-index and add state correction * check for state being kept * move monthly to 8760 conversion to eia preprocessing * remove duplicitous code * change file name and update single site compatibility * update module name in docs * update state retrieval * add docstring * move geospatial tests to preprocessing * move eia api testing to preprocessing and add more tests * update default file handling and state/category sort order * remove pre-return * add data download test * update feedstock config test and fix reverse geocode issue in config * add hourly conversion test and fix issues in function * update feedstock cost model checks * udpate docs * update changelog * fix str-None comparison * add the none default to get data function * fix ci test issues * remove extraneous note * add site_name and handling * add additional site handling * add user warning for modified price * add feedstock model tests * be specific about us-states when applicable * add handling for when no geocoder available * reinstate original file handler * update feedstock import pathway and use subpackage level import, not module * minor warning change --------- Co-authored-by: John Jasa <johnjasa11@gmail.com>
1 parent 6334faa commit 4250dd2

18 files changed

Lines changed: 1494 additions & 29 deletions

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Changelog
22

33
## Unreleased
4+
45
- Change commodity in DRI and EAF model from pig iron to sponge iron based on likely carbon content [PR 670](https://github.com/NatLabRockies/H2Integrate/pull/670)
56
- Bugfix for round-trip efficiency handling when calling `check_inputs` around `StoragePerformanceModel` [PR 684](https://github.com/NatLabRockies/H2Integrate/pull/684)
67
- Bugfix. Include nuclear in electricity producing tech list and improve error message for zero-length electricity producing techs in model when electricity is specified as the commodity. [PR 685](https://github.com/NatLabRockies/H2Integrate/pull/685)
@@ -17,6 +18,16 @@
1718
- Add `{commodity}_set_point` as an input to hydrogen fuel cell model [PR 709](https://github.com/NatLabRockies/H2Integrate/pull/709)
1819
- Rename `n_control_window` to `n_control_window_hours` for unit clarity [PR 712](https://github.com/NatLabRockies/H2Integrate/pull/712)
1920
- Update N2 diagram for demand openloop control from static and outdated to dynamic and interactive [PR 714](https://github.com/NatLabRockies/H2Integrate/pull/714)
21+
- `feedstocks.py` has moved from `h2integrate/core/` to `h2_integrate/feedstocks`
22+
[PR 719](https://github.com/NatLabRockies/H2Integrate/pull/719)
23+
- Creates the `EIANaturalGasFeedstockConfig` and `EIANaturalGasFeedstockCostModel` to load EIA
24+
natural gas prices from file or to retrieve them from the EIA API. The model is able to retrieve
25+
the US or any of the 50 states' annual or monthly values, which will be converted into an hourly
26+
timeseries. [PR 719](https://github.com/NatLabRockies/H2Integrate/pull/719)
27+
- Creates a series of preprocessing tools for basic state name and abbreviation handling, and for
28+
reverse geocoding coordinates to state data. [PR 719](https://github.com/NatLabRockies/H2Integrate/pull/719)
29+
- Creates a series of preprocessing tools for downloading EIA natural gas prices to monthly and
30+
hourly timeseries that could be extended to other EIA API data in the future. [PR 719](https://github.com/NatLabRockies/H2Integrate/pull/719)
2031
- Added basic check of 4-length connections in `technology_interconnections` [PR 720](https://github.com/NatLabRockies/H2Integrate/pull/720)
2132
- Adds `H2IntegrateModel`, `load_yaml`, `write_yaml`, and `write_readable_yaml` as package-level imports [PR 728](https://github.com/NatLabRockies/H2Integrate/pull/728).
2233
- Update N2 diagram for Pyomo heuristic control from static image to dynamic and interactive embedded diagram [PR 726](https://github.com/NatLabRockies/H2Integrate/pull/726)
@@ -26,6 +37,7 @@
2637
- Add `constant` pricing mode for Grid cost models, allowing an explicit scalar price configuration alongside `per_timestep` and `per_year` modes. [PR 764](https://github.com/NatLabRockies/H2Integrate/pull/764)
2738

2839
## 0.8 [April 15, 2026]
40+
2941
- Updated README and docs intro page with expanded H2I description, reorganized sections, and streamlined installation instructions [PR 677](https://github.com/NatLabRockies/H2Integrate/pull/677)
3042
- Update energy conversion ratio in H2 SMR model [PR 606](https://github.com/NatLabRockies/H2Integrate/pull/606)
3143
- Update iron models and examples [PR 601](https://github.com/NatLabRockies/H2Integrate/pull/601)

docs/_static/class_hierarchy.html

Lines changed: 2 additions & 2 deletions
Large diffs are not rendered by default.

docs/api.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
:recursive:
99
1010
core
11+
feedstocks
1112
converters
1213
control
1314
finances

docs/technology_models/feedstocks.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ technology_interconnections: [
3232
```
3333

3434
Where:
35+
3536
- `name_of_feedstock_source`: Name of your feedstock source
3637
- `consuming_technology`: Technology that uses the feedstock
3738
- `commodity`: Type identifier (e.g., "natural_gas", "water", "electricity")
@@ -85,9 +86,73 @@ The `price` parameter is flexible - you can specify constant pricing with a sing
8586
```
8687

8788
### Consumed Feedstock Outputs
89+
8890
The feedstock model outputs cost and performance information about the consumed feedstock. The most notable outputs are:
8991
- `VarOpEx`: cost of the feedstock consumed (in `USD/yr`)
9092
- `total_{commodity}_consumed`: total feedstock consumed over simulation (in `commodity_amount_units`)
9193
- `annual_{commodity}_consumed`: annual feedstock consumed (in `commodity_amount_units/yr`)
9294
- `rated_{commodity}_production`: this is equal to the the `rated_capacity` of the feedstock model (in `commodity_rate_units`)
9395
- `capacity_factor`: ratio of the feedstock consumed to the maximum feedstock available
96+
97+
## EIA Natural Gas Pricing
98+
99+
A special case of the feedstock cost model `EIANaturalGasFeedstockCostModel` exists to enable users
100+
to download data from the EIA API's natural gas price
101+
portal for a single site (see
102+
[the relevant API docs](https://h2integrate.readthedocs.io/en/latest/_autosummary/h2integrate.feedstocks.eia_ng_price.html)
103+
for complete details). Access to the wellhead, import, citygate, residential, commercial,
104+
industrial, electrical power, and exports price facets are supported for the US
105+
as a whole, though it is best to see which data are
106+
[available online in the EIA API documentation](https://www.eia.gov/opendata/browser/natural-gas/pri/sum)
107+
prior to using in an analysis.
108+
109+
Users are expected to get an EIA API key from the
110+
[EIA Open Data portal](https://www.eia.gov/opendata/), or to download the data as as CSV file for
111+
the model to load.
112+
113+
At present, the EIA natural gas cost model uses only a single year of price data (annual or monthly)
114+
and extrapolates it to an hourly timeseries automatically. For users that wish to download a large
115+
batch of data from the EIA, please see the EIA preprocessing tools in
116+
[`h2integrate/preprocess/eia.py`](https://github.com/NatLabRockies/H2Integrate/h2integrate/preprocess/eia.py).
117+
In particular, use the `get_eia_ng_data` function once an API has been created and saved to your
118+
environment variables or to a file.
119+
120+
### Configuring the EIA Cost Model
121+
122+
Similar to the standard feedstock model, the following variables are able to be set by the user
123+
124+
- `cost_year` (int): Dollar year for cost inputs
125+
- `annual_cost` (float, optional): Fixed cost per year in USD/year. Defaults to 0.0
126+
- `start_up_cost` (float, optional): One-time capital cost in USD. Defaults to 0.0
127+
128+
Additionally, there are a few other settings that users will need to provide for the model to work
129+
that differ from the standard cost model.
130+
131+
:::{important}
132+
If relying on site coordinate data (`latitude` and `longitude`), then the `reverse_geocoder` package
133+
is required, which can be pip installed directly or through the `gis` library extras.
134+
:::
135+
136+
- `resource_year` (int): Which year to obtain the data from in the range [2001, 2026].
137+
- `monthly` (bool): If True, the monthly data are retrieved, otherwise the annual price is used.
138+
- location data options 1) use `state` or 2) use `latitude` and `longitude`:
139+
- `state` (str): Full name of the state or two-letter state abbreviation, such as "United States" or
140+
"US". Only the "US" or all 50 states will produce valid results. When `state` is provided, the
141+
site coordinate data will be ignored.
142+
- `latitude` (float): Latitude of the natural gas plant site. Only used when `state` is not
143+
provided, and will be filled from the plant configuration's site data if not provided.
144+
- `longitude` (float): Longitude of the natural gas plant site. Only used when `state` is not
145+
provided, and will be filled from the plant configuration's site data if not provided.
146+
147+
- `price_category` (str): One of "wellhead", "imports", "citygate", "residential", "commercial",
148+
"industrial", "electrical_power", or "exports". Note that not all categories will return
149+
state-level data.
150+
- `api_key_file` (str, optional): The file where the user's API key is stored. If storing in a file,
151+
define it on its own line using the convention "EIA_API_KEY: xxxx", or have the API key defined as
152+
an environment variable set as "EIA_API_KEY".
153+
- `filename` (str, optional): Filename where the data should be loaded from if it exists or saved
154+
to once it is downloaded.
155+
156+
- `commodity`: Set to "natural_gas" internally.
157+
- `commodity_rate_units`: Set to "MMBtu/h" internally.
158+
- `commodity_amount_units`: Set to "MMBtu" internally.

h2integrate/core/h2integrate_model.py

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -624,9 +624,9 @@ def create_technology_models(self):
624624
self.finance_models.append(finance_object)
625625

626626
for tech_name, individual_tech_config in self.technology_config["technologies"].items():
627-
cost_model = individual_tech_config.get("cost_model", {}).get("model")
627+
cost_model = individual_tech_config.get("cost_model", {}).get("model", "")
628628

629-
if cost_model == "FeedstockCostModel":
629+
if "FeedstockCostModel" in cost_model:
630630
comp = self.supported_models[cost_model](
631631
driver_config=self.driver_config,
632632
plant_config=self.plant_config,
@@ -1146,11 +1146,11 @@ def connect_technologies(self):
11461146
# Get the performance model of the source_tech
11471147
source_tech_config = self.technology_config["technologies"].get(source_tech, {})
11481148
perf_model_name = source_tech_config.get("performance_model", {}).get("model")
1149-
cost_model_name = source_tech_config.get("cost_model", {}).get("model")
1149+
cost_model_name = source_tech_config.get("cost_model", {}).get("model", "")
11501150

11511151
# If the source is a feedstock, make sure to connect the amount of
11521152
# feedstock consumed from the technology back to the feedstock cost model
1153-
if cost_model_name == "FeedstockCostModel":
1153+
if "FeedstockCostModel" in cost_model_name:
11541154
self.plant.connect(
11551155
f"{dest_tech}.{transport_item}_consumed",
11561156
f"{source_tech}.{transport_item}_consumed",
@@ -1745,7 +1745,7 @@ def _check_tech_connections(self):
17451745
group = getattr(self.prob.model.plant, f"{tech_name}_source")
17461746
else:
17471747
group = getattr(self.prob.model.plant, tech_name)
1748-
if model_name != "FeedstockCostModel":
1748+
if "FeedstockCostModel" not in model_name:
17491749
group = getattr(group, model_name, None)
17501750
if group is None:
17511751
continue

h2integrate/core/supported_models.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -165,8 +165,9 @@ def copy(self):
165165
"PyomoRuleStorageMinOperatingCosts": "control.control_rules.storage.pyomo_storage_rule_min_operating_cost:PyomoRuleStorageMinOperatingCosts",
166166
"PyomoDispatchGenericConverterMinOperatingCosts": "control.control_rules.converters.generic_converter_min_operating_cost:PyomoDispatchGenericConverterMinOperatingCosts",
167167
# Feedstock
168-
"FeedstockPerformanceModel": "core.feedstocks:FeedstockPerformanceModel",
169-
"FeedstockCostModel": "core.feedstocks:FeedstockCostModel",
168+
"FeedstockPerformanceModel": "feedstocks:FeedstockPerformanceModel",
169+
"FeedstockCostModel": "feedstocks:FeedstockCostModel",
170+
"EIANaturalGasFeedstockCostModel": "feedstocks:EIANaturalGasFeedstockCostModel",
170171
# Grid
171172
"GridPerformanceModel": "converters.grid.grid:GridPerformanceModel",
172173
"GridCostModel": "converters.grid.grid:GridCostModel",

h2integrate/feedstocks/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
from h2integrate.feedstocks.feedstocks import FeedstockPerformanceModel, FeedstockCostModel
2+
from h2integrate.feedstocks.eia_ng_price import EIANaturalGasFeedstockCostModel
Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
import warnings
2+
from pathlib import Path
3+
from datetime import datetime
4+
5+
import attrs
6+
import numpy as np
7+
import pandas as pd
8+
from attrs import field, define
9+
10+
from h2integrate.preprocess import eia, geospatial
11+
from h2integrate.core.utilities import merge_shared_inputs
12+
from h2integrate.core.file_utils import get_path
13+
from h2integrate.core.validators import range_val
14+
from h2integrate.feedstocks.feedstocks import FeedstockCostModel
15+
from h2integrate.core.model_baseclasses import BaseConfig
16+
17+
18+
HOURS_PER_YEAR = 8760
19+
SECONDS_PER_HOUR = 3600
20+
CURRENT_YEAR = datetime.now().year
21+
22+
default_price = pd.DataFrame(
23+
np.zeros(8760, dtype=float).reshape(-1, 1),
24+
columns=["price"],
25+
index=pd.date_range("2001-01-01", "2001-12-31 23:00:00", freq="h"),
26+
)
27+
28+
29+
@define
30+
class EIANaturalGasFeedstockConfig(BaseConfig):
31+
"""EIA Industrial Natural Gas Pricing API configuration and downloader for the US and all 50 US
32+
states, in $/MCF, converted to $/MMBtu. Please see
33+
https://www.eia.gov/opendata/browser/natural-gas/pri/sum for further details about data
34+
availability.
35+
36+
Args:
37+
state (str): Full name of the state or two-letter state abbreviation, such as
38+
"United States" or "US". Only the "US" or all 50 states will produce valid results.
39+
resource_year (int): The YYYY-format year whose data should be retrieved. Must be between
40+
2001 and the current year, inclusive of endpoints.
41+
cost_year (int): dollar-year for costs. Defaults to the current year.
42+
monthly (Path): True, if monthly data is desired, False if annual data is desired.
43+
price_category (str): One of "wellhead", "imports", "citygate", "residential", "commercial",
44+
"industrial", "electrical_power", or "exports". Note that not all categories will return
45+
state-level data.
46+
api_key_file (Path, optional): Full file name of the file where the API key is located. If
47+
no file name is provided, then the environment variables ``EIA_API_KEY`` is used.
48+
latitude (float | None): WGS-84 y-coordinate of the site. Will not be used if
49+
:py:attr:`state` has been provided.
50+
longitude (float | None): WGS-84 x-coordinate of the site. Will not be used if
51+
:py:attr:`state` has been provided.
52+
site_name (str, optional): Name of the site from a multi-site private configuration. When
53+
provided, the :py:class:`EIANaturalGasFeedstockCostModel` will populate the
54+
:py:attr:`latitude` and :py:attr:`longitude` from the plant site configuration.
55+
filename (str, optional): Filename for where to save the data or where the data may
56+
already be located. If the file exists, the columns "period", "state", and "price" must
57+
exist, otherwise the file will not be used. "period" should be of the form YYYY or
58+
YYYY-MM, and state should be either the full state name or the two-letter abbreviation.
59+
annual_cost (float, optional): fixed cost associated with the feedstock in USD/year.
60+
Defaults to 0.0.
61+
start_up_cost (float, optional): one-time capital cost associated with the feedstock in USD.
62+
Defaults to 0.0.
63+
"""
64+
65+
resource_year: int = field(validator=attrs.validators.in_(range(2001, CURRENT_YEAR + 1)))
66+
monthly: bool = field(validator=attrs.validators.instance_of(bool))
67+
price_category: str = field(
68+
converter=str.lower, validator=attrs.validators.in_(eia.EIA_NG_FACET)
69+
)
70+
api_key_file: str | None = field(default=None, converter=attrs.converters.optional(get_path))
71+
state: str = field(
72+
default=None,
73+
converter=attrs.converters.optional(
74+
attrs.converters.pipe(geospatial.convert_state_value, geospatial.convert_state_to_code)
75+
),
76+
validator=attrs.validators.optional(
77+
attrs.validators.in_([*geospatial.US_STATE_MAP, *geospatial.US_STATE_MAP.values()])
78+
),
79+
)
80+
latitude: float | None = field(
81+
default=None, validator=attrs.validators.optional(range_val(-90.0, 90.0))
82+
)
83+
longitude: float | None = field(
84+
default=None, validator=attrs.validators.optional(range_val(-180.0, 180.0))
85+
)
86+
site_name: str = field(
87+
default=None, validator=attrs.validators.optional(attrs.validators.instance_of(str))
88+
)
89+
cost_year: int = field(default=CURRENT_YEAR)
90+
annual_cost: float = field(default=0.0, converter=float)
91+
start_up_cost: float = field(default=0.0, converter=float)
92+
filename: str = field(default=None)
93+
94+
commodity: str = field(default="natural_gas", init=False)
95+
commodity_rate_units: str = field(default="MMBtu/h", init=False)
96+
commodity_amount_units: str = field(default="MMBtu", init=False)
97+
price: pd.DataFrame = field(
98+
default=default_price, init=False, validator=attrs.validators.instance_of(pd.DataFrame)
99+
)
100+
101+
def __attrs_post_init__(self):
102+
"""Creates the EIA natural gas facet series code based on validated user inputs, sets the
103+
:py:attr:`commodity_amount_units` if not given a value, and fetches the EIA natural gas
104+
price.
105+
"""
106+
if self.filename is not None:
107+
try:
108+
self.filename = get_path(self.filename)
109+
except FileNotFoundError:
110+
self.filename = Path(self.filename).resolve()
111+
112+
if self.state is None:
113+
if self.latitude is None or self.longitude is None:
114+
msg = (
115+
"The EIA natural gas feedstock model require one of `state` or"
116+
" `latitude` and `longitude`."
117+
)
118+
raise ValueError(msg)
119+
120+
self.state = geospatial.get_state_from_coords(
121+
latitude=self.latitude, longitude=self.longitude
122+
)
123+
124+
125+
class EIANaturalGasFeedstockCostModel(FeedstockCostModel):
126+
"""Feedstock cost model based on the EIA natural gas price API results that uses
127+
annual or monthly data to model an hourly time step for a single year to model the
128+
price of natural gas used in the model.
129+
130+
The model will pull the single site data if it is made available to pass the ``latitude`` and
131+
``longitude`` from the site configuration, otherwise the ``state`` will need to be provided
132+
in the configuration.
133+
"""
134+
135+
def setup(self):
136+
"""Defines the inputs and outputs of the model and converts the
137+
:py:attr:`EIANaturalGasFeedstockConfig.price` to an hourly timeseries for the
138+
``plant_life``.
139+
140+
Populates the site latitude and longitude with either the configuration's provided site
141+
name or the first site in plant configuration's ``sites`` dictionary when a ``state`` is
142+
not directly provided.
143+
"""
144+
site_config = {}
145+
sites = self.options["plant_config"].get("sites", {})
146+
cost_config = merge_shared_inputs(self.options["tech_config"]["model_inputs"], "cost")
147+
148+
if cost_config.get("state") is None:
149+
if (site_name := cost_config.get("site_name")) is not None and sites:
150+
site_config = sites.get(site_name, {})
151+
if site_name is None and sites:
152+
site_config = sites[[*sites][0]]
153+
154+
self.config = EIANaturalGasFeedstockConfig.from_dict(
155+
cost_config | site_config, additional_cls_name=self.__class__.__name__, strict=False
156+
)
157+
158+
price = eia.get_eia_ng_data(
159+
api_key_file=self.config.api_key_file,
160+
resource_year=self.config.resource_year,
161+
price_category=self.config.price_category,
162+
state=self.config.state,
163+
monthly=self.config.monthly,
164+
filename=self.config.filename,
165+
)
166+
price = eia.convert_to_hourly(price)
167+
self.config.price = price
168+
super().setup()
169+
170+
def compute(self, inputs, outputs):
171+
if not np.isclose(inputs["price"], self.config.price, rtol=1e-6):
172+
warn_msg = "The NG price has changed from EIA price. This may be intended."
173+
warnings.warn(warn_msg, UserWarning)
174+
super().compute(inputs, outputs)

0 commit comments

Comments
 (0)