Skip to content

Commit 8b65e31

Browse files
jmartin4ujohnjasa
andauthored
Switching lots of tech config values from a .csv (#242)
* Switching HB cases * Docstring updates * Running while checking data types * Test getting there * Test working * Docs * Updated docs and tests * pre-commit fix --------- Co-authored-by: John Jasa <johnjasa11@gmail.com>
1 parent fa20cf6 commit 8b65e31

9 files changed

Lines changed: 287 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Changelog
22

3+
## 0.4.x [TBD]
4+
- Added `tools/run_cases.py` with tools to run different `tech_config` cases from a spreadsheet, with new docs page to describe: docs/user_guide/how_to_run_several_cases_in_sequence.md
5+
36
## 0.4.0 [October 1, 2025]
47

58
This release introduces significant new technology models and framework capabilities for system design and optimization, alongside major refactoring and user experience improvements.

docs/_toc.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ parts:
1717
- file: user_guide/design_optimization_in_h2i
1818
- file: user_guide/postprocessing_results
1919
- file: user_guide/how_to_interface_with_user_defined_model
20+
- file: user_guide/how_to_run_several_cases_in_sequence
2021
- file: user_guide/specifying_finance_parameters
2122
sections:
2223
- file: user_guide/cost_years
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Running several cases in sequence
2+
3+
## Overview
4+
5+
If you have several different cases you want to run with different input parameters, these cases can be set up in an input spreadsheet rather than directly modifying the `tech_config`.
6+
This is done using the functions in `h2integrate/tools/run_cases.py`
7+
8+
## Setting up a variation of parameters in a .csv
9+
10+
To set up the inputs for each case, the input .csv should be set up like so:
11+
| Index 0 | Index 1 |...| Index N | Type | <Case #1 Name> | <Case #2 Name> |...| <Case #N Name> |
12+
|--------------|----------------|---|----------------|-------|-----------------|-----------------|---|-----------------|
13+
| technologies | <param_1_tech> |...| <param_1_name> | float | <Case #1 value> | <Case #2 value> |...| <Case #N value> |
14+
| technologies | <param_2_tech> |...| <param_2_name> | str | <Case #1 value> | <Case #2 value> |...| <Case #N value> |
15+
| ... | ... |...| ... | ... | ... | ... |...| ... |
16+
| technologies | <param_N_tech> |...| <param_N_name> | bool | <Case #1 value> | <Case #2 value> |...| <Case #N value> |
17+
18+
19+
## Example: Two different sizes of Haber Bosch ammonia plant
20+
21+
To demonstrate this capability, we include a short example that modifies the size and hydrogen storage type for a Haber Bosch ammonia plant in `examples/12_ammonia_synloop`.
22+
The example spreadsheet `hb_inputs.csv` shows the format:
23+
24+
|Index 0|Index 1|Index 2|Index 3|Index 4|Index 5|Type|Haber Bosch Big|Haber Bosch Small
25+
|--- |--- |--- |--- |--- |--- |--- |--- |--- |
26+
technologies|ammonia|model_inputs|shared_parameters|production_capacity||float|100000|10000
27+
technologies|h2_storage|model_inputs|performance_parameters|type||str|salt_cavern|lined_rock_cavern
28+
technologies|electrolyzer|model_inputs|performance_parameters|n_clusters||int|16|2
29+
technologies|electrolyzer|model_inputs|performance_parameters|include_degradation_penalty||bool|TRUE|FALSE
30+
technologies|electrolyzer|model_inputs|financial_parameters|capital_items|replacement_cost_percent|float|0.15|0.25
31+
32+
33+
### Things to note about this format
34+
35+
- The nested depth of the parameters in the `tech_config` can vary based on the parameter you're setting. If some parameters do not use as many levels, leave the unused levels blank (like the Index 5 level for most parameters in the above example)
36+
- The currently available data types for each parameter are `float`, `str`, `int`, and `bool`. Be sure to specify the correct data type for each parameter.
37+
- For parameters declared as `bool`, you can enter "0", "false", or "no" for `False`, and "1", "true", or "yes" for `True` (case insensitive). When making .csvs in Excel, the default "TRUE" and "FALSE" formatting will work.
38+
- For all other parameters not included in the spreadsheet, their values will be kept the same as originally defined in the `tech_config.yaml`.
39+
40+
The variation of parameters can be run by first creating an H2I model (with a `tech_config.yaml`), then modifying only the `tech_config` values that need to change.
41+
First, the spreadsheet with each case is loaded into a Pandas DataFrame using `load_tech_config_cases`.
42+
Then, in a loop, individual cases are selected and the model is modified to use these parameters using `modify_tech_config`.
43+
An example is shown in `run_ammonia_synloop.py`:
44+
45+
```
46+
from pathlib import Path
47+
48+
from h2integrate.tools.run_cases import modify_tech_config, load_tech_config_cases
49+
from h2integrate.core.h2integrate_model import H2IntegrateModel
50+
51+
52+
# Create a H2Integrate model
53+
model = H2IntegrateModel("12_ammonia_synloop.yaml")
54+
55+
# Load cases
56+
case_file = Path("hb_inputs.csv")
57+
cases = load_tech_config_cases(case_file)
58+
59+
# Modify and run the model for different cases
60+
caselist = [
61+
"Haber Bosch Big",
62+
"Haber Bosch Small",
63+
]
64+
for casename in caselist:
65+
case = cases[casename]
66+
model = modify_tech_config(model, case)
67+
model.run()
68+
model.post_process()
69+
```
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Index 0,Index 1,Index 2,Index 3,Index 4,Index 5,Type,Haber Bosch Big,Haber Bosch Small
2+
technologies,ammonia,model_inputs,shared_parameters,production_capacity,,float,100000,10000
3+
technologies,h2_storage,model_inputs,performance_parameters,type,,str,salt_cavern,lined_rock_cavern
4+
technologies,electrolyzer,model_inputs,performance_parameters,n_clusters,,int,16,2
5+
technologies,electrolyzer,model_inputs,performance_parameters,include_degradation_penalty,,bool,TRUE,FALSE
6+
technologies,electrolyzer,model_inputs,financial_parameters,capital_items,replacement_cost_percent,float,0.15,0.25
Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,23 @@
1+
from pathlib import Path
2+
3+
from h2integrate.tools.run_cases import modify_tech_config, load_tech_config_cases
14
from h2integrate.core.h2integrate_model import H2IntegrateModel
25

36

47
# Create a H2Integrate model
58
model = H2IntegrateModel("12_ammonia_synloop.yaml")
69

7-
# Run the model
8-
model.run()
10+
# Load cases
11+
case_file = Path("hb_inputs.csv")
12+
cases = load_tech_config_cases(case_file)
913

10-
model.post_process()
14+
# Modify and run the model for different cases
15+
caselist = [
16+
"Haber Bosch Big",
17+
"Haber Bosch Small",
18+
]
19+
for casename in caselist:
20+
case = cases[casename]
21+
model = modify_tech_config(model, case)
22+
model.run()
23+
model.post_process()

h2integrate/tools/inflation/cepci.csv

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,3 +27,4 @@ Year,CEPCI
2727
2021,628
2828
2022,794
2929
2023,801
30+
2024,800

h2integrate/tools/run_cases.py

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
import operator
2+
from functools import reduce
3+
4+
import pandas as pd
5+
6+
7+
def cast_by_name(type_name, value):
8+
"""Cast a string read in from an input file as a data type also given as a string.
9+
Currently allowed data types: int, float, bool, str
10+
11+
Args:
12+
type_name (str): The data type to cast into, as a string.
13+
value (str): The value, as a string.
14+
15+
Returns:
16+
The value in the specified data type
17+
"""
18+
19+
bool_map = {"true": True, "false": False, "yes": True, "no": False, "1": True, "0": False}
20+
21+
trusted_types = ["int", "float", "bool", "str"] ## others as needed
22+
if type_name in trusted_types:
23+
if type_name == "bool":
24+
return bool_map.get(value.lower())
25+
else:
26+
return __builtins__[type_name](value)
27+
else:
28+
msg = f"Specified data type {type_name} invalid, must be one of {trusted_types}"
29+
raise TypeError(msg)
30+
31+
32+
def get_from_dict(dataDict, mapList):
33+
"""Get value from nested dictionary using a list of keys.
34+
35+
Allows for programmatic calling of items in a nested dict using a variable-length list.
36+
Instead of dataDict[item1][item2][item3][item4][item5], you can use
37+
get_from_dict(dataDict, [item1, item2, item3, item4, item5]).
38+
39+
Args:
40+
dataDict (dict): The nested dictionary to access.
41+
mapList (list): List of keys to traverse the nested dictionary.
42+
43+
Returns:
44+
The value at the specified nested location in the dictionary.
45+
46+
Example:
47+
>>> data = {"a": {"b": {"c": 42}}}
48+
>>> get_from_dict(data, ["a", "b", "c"])
49+
42
50+
"""
51+
return reduce(operator.getitem, mapList, dataDict)
52+
53+
54+
def set_in_dict(dataDict, mapList, value):
55+
"""Set value in nested dictionary using a list of keys.
56+
57+
Allows for programmatic setting of items in a nested dict using a variable-length list.
58+
Instead of dataDict[item1][item2][item3][item4][item5] = value, you can use
59+
set_in_dict(dataDict, [item1, item2, item3, item4, item5], value).
60+
61+
Args:
62+
dataDict (dict): The nested dictionary to modify.
63+
mapList (list): List of keys to traverse the nested dictionary.
64+
value: The value to set at the specified nested location.
65+
66+
Example:
67+
>>> data = {"a": {"b": {}}}
68+
>>> set_in_dict(data, ["a", "b", "c"], 42)
69+
>>> data["a"]["b"]["c"]
70+
42
71+
"""
72+
get_from_dict(dataDict, mapList[:-1])[mapList[-1]] = value
73+
74+
75+
def load_tech_config_cases(case_file):
76+
"""Load extensive lists of values from a spreadsheet to run many different cases.
77+
78+
Loads tech_config values from a CSV file to run multiple cases with different
79+
technology configuration values.
80+
81+
Args:
82+
case_file (Path): Path to the .csv file where the different tech_config values
83+
are listed. The CSV must be formatted with "Index 1", "Index 2", etc.
84+
columns followed by case name columns. Each row should have "technologies"
85+
as the first index value, followed by tech_name and parameter names.
86+
87+
Returns:
88+
pd.DataFrame: DataFrame with the indexes of the tech_config as a MultiIndex
89+
and the different case names as the column names.
90+
91+
Note:
92+
The CSV format should be:
93+
| "Index 1" |...| "Index <N>" | "Type" | <Case 1 Name> |...| <Case N Name> |
94+
| "technologies" |...| <param_1_name> | "float" | <Case 1 value> |...| <Case N value> |
95+
| "technologies" |...| <param_2_name> | "str" | <Case 1 value> |...| <Case N value> |
96+
97+
If some parameters are nested deeper than others, make as many Index columns for the deepest-
98+
nested parameters and leave any unused Indexes blank.
99+
100+
See example .csv in h2integrate/tools/test/test_inputs.csv
101+
"""
102+
tech_config_cases = pd.read_csv(case_file)
103+
column_names = tech_config_cases.columns.values
104+
index_names = list(filter(lambda x: "Index" in x, column_names))
105+
index_names.append("Type")
106+
tech_config_cases = tech_config_cases.set_index(index_names)
107+
108+
return tech_config_cases
109+
110+
111+
def modify_tech_config(h2i_model, tech_config_case):
112+
"""Modify particular tech_config values on an existing H2I model before it is run.
113+
114+
Args:
115+
h2i_model: H2IntegrateModel that has been set up but not run.
116+
tech_config_case (pd.Series): Series that was indexed from tech_config_cases
117+
DataFrame containing the parameter values to modify.
118+
119+
Returns:
120+
H2IntegrateModel: The H2IntegrateModel with modified tech_config values.
121+
"""
122+
for index_tup, value in tech_config_case.items():
123+
index_list = list(index_tup)
124+
data_type = index_list[-1]
125+
index_list = index_list[:-1]
126+
# Remove nans from blank index fields
127+
while type(index_list[-1]) is not str:
128+
index_list = index_list[:-1]
129+
set_in_dict(h2i_model.technology_config, index_list, cast_by_name(data_type, value))
130+
131+
return h2i_model
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Index 0,Index 1,Index 2,Index 3,Index 4,Index 5,Type,Float Test,Int Test,Bool Test,Str Test
2+
technologies,solar,model_inputs,performance_parameters,pv_capacity_kWdc,,float,100000.,150000.,150000.,150000.
3+
technologies,electrolyzer,model_inputs,performance_parameters,n_clusters,,int,18,20,18,18
4+
technologies,electrolyzer,model_inputs,performance_parameters,include_degradation_penalty,,bool,TRUE,TRUE,FALSE,TRUE
5+
technologies,electrolyzer,model_inputs,performance_parameters,sizing,size_for,str,BOL,BOL,BOL,EOL
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
import os
2+
from pathlib import Path
3+
4+
import pytest
5+
6+
from h2integrate import EXAMPLE_DIR
7+
from h2integrate.tools.run_cases import modify_tech_config, load_tech_config_cases
8+
from h2integrate.core.h2integrate_model import H2IntegrateModel
9+
10+
11+
def test_tech_config_modifier(subtests):
12+
"""Test cases for modifying and running tech_config from csv.
13+
Using 15 example as test case
14+
"""
15+
16+
# Make an H2I model from the 15 example
17+
os.chdir(EXAMPLE_DIR / "15_wind_solar_electrolyzer")
18+
example_yaml = "15_wind_solar_electrolyzer.yaml"
19+
model = H2IntegrateModel(example_yaml)
20+
21+
# Modify using csv
22+
case_file = Path(__file__).resolve().parent / "test_inputs.csv"
23+
cases = load_tech_config_cases(case_file)
24+
with subtests.test("float"):
25+
case = cases["Float Test"]
26+
model = modify_tech_config(model, case)
27+
model.run()
28+
assert (
29+
pytest.approx(model.prob.get_val("finance_subgroup_hydrogen.LCOH")[0], rel=1e-3)
30+
== 5.327792370180044
31+
)
32+
with subtests.test("bool"):
33+
case = cases["Bool Test"]
34+
model = modify_tech_config(model, case)
35+
model.run()
36+
assert (
37+
pytest.approx(model.prob.get_val("finance_subgroup_hydrogen.LCOH")[0], rel=1e-3)
38+
== 5.226443205147294
39+
)
40+
with subtests.test("int"):
41+
case = cases["Int Test"]
42+
model = modify_tech_config(model, case)
43+
model.run()
44+
assert (
45+
pytest.approx(model.prob.get_val("finance_subgroup_hydrogen.LCOH")[0], rel=1e-3)
46+
== 5.4601971211592115
47+
)
48+
with subtests.test("str"):
49+
case = cases["Str Test"]
50+
model = modify_tech_config(model, case)
51+
model.run()
52+
assert (
53+
pytest.approx(model.prob.get_val("finance_subgroup_hydrogen.LCOH")[0], rel=1e-3)
54+
== 5.22644320514729
55+
)

0 commit comments

Comments
 (0)