Switching lots of tech config values from a .csv

CHANGELOG.md

@@ -1,5 +1,8 @@
 # Changelog
 
+## 0.4.x [TBD]
+- Added `tools/run_cases.py` with tools to run different `tech_config` cases from a spreadsheet, with new docs page to describe: docs/user_guide/how_to_run_several_cases_in_sequence.md
+
 ## 0.4.0 [October 1, 2025]
 
 This release introduces significant new technology models and framework capabilities for system design and optimization, alongside major refactoring and user experience improvements.

docs/_toc.yml

@@ -17,6 +17,7 @@ parts:
   - file: user_guide/design_optimization_in_h2i
   - file: user_guide/postprocessing_results
   - file: user_guide/how_to_interface_with_user_defined_model
+  - file: user_guide/how_to_run_several_cases_in_sequence
   - file: user_guide/specifying_finance_parameters
     sections:
     - file: user_guide/cost_years

docs/user_guide/how_to_run_several_cases_in_sequence.md

@@ -0,0 +1,69 @@
+# Running several cases in sequence
+
+## Overview
+
+If you have several different cases you want to run with different input parameters, these cases can be set up in an input spreadsheet rather than directly modifying the `tech_config`.
+This is done using the functions in `h2integrate/tools/run_cases.py`
+
+## Setting up a variation of parameters in a .csv
+
+To set up the inputs for each case, the input .csv should be set up like so:
+|   Index 0    |    Index 1     |...|     Index N    | Type  | <Case #1 Name>  | <Case #2 Name>  |...| <Case #N Name>  |
+|--------------|----------------|---|----------------|-------|-----------------|-----------------|---|-----------------|
+| technologies | <param_1_tech> |...| <param_1_name> | float | <Case #1 value> | <Case #2 value> |...| <Case #N value> |
+| technologies | <param_2_tech> |...| <param_2_name> | str   | <Case #1 value> | <Case #2 value> |...| <Case #N value> |
+| ...          | ...            |...| ...            | ...   | ...             | ...             |...| ...             |
+| technologies | <param_N_tech> |...| <param_N_name> | bool  | <Case #1 value> | <Case #2 value> |...| <Case #N value> |
+
+
+## Example: Two different sizes of Haber Bosch ammonia plant
+
+To demonstrate this capability, we include a short example that modifies the size and hydrogen storage type for a Haber Bosch ammonia plant in `examples/12_ammonia_synloop`.
+The example spreadsheet `hb_inputs.csv` shows the format:
+
+|Index 0|Index 1|Index 2|Index 3|Index 4|Index 5|Type|Haber Bosch Big|Haber Bosch Small
+|---    |---    |---    |---    |---    |---    |---    |---    |---    |
+technologies|ammonia|model_inputs|shared_parameters|production_capacity||float|100000|10000
+technologies|h2_storage|model_inputs|performance_parameters|type||str|salt_cavern|lined_rock_cavern
+technologies|electrolyzer|model_inputs|performance_parameters|n_clusters||int|16|2
+technologies|electrolyzer|model_inputs|performance_parameters|include_degradation_penalty||bool|TRUE|FALSE
+technologies|electrolyzer|model_inputs|financial_parameters|capital_items|replacement_cost_percent|float|0.15|0.25
+
+
+### Things to note about this format
+
+- The nested depth of the parameters in the `tech_config` can vary based on the parameter you're setting. If some parameters do not use as many levels, leave the unused levels blank (like the Index 5 level for most parameters in the above example)
+- The currently available data types for each parameter are `float`, `str`, `int`, and `bool`. Be sure to specify the correct data type for each parameter.
+- For parameters declared as `bool`, you can enter "0", "false", or "no" for `False`, and "1", "true", or "yes" for `True` (case insensitive). When making .csvs in Excel, the default "TRUE" and "FALSE" formatting will work.
+- For all other parameters not included in the spreadsheet, their values will be kept the same as originally defined in the `tech_config.yaml`.
+
+The variation of parameters can be run by first creating an H2I model (with a `tech_config.yaml`), then modifying only the `tech_config` values that need to change.
+First, the spreadsheet with each case is loaded into a Pandas DataFrame using `load_tech_config_cases`.
+Then, in a loop, individual cases are selected and the model is modified to use these parameters using `modify_tech_config`.
+An example is shown in `run_ammonia_synloop.py`:
+
+```
+from pathlib import Path
+
+from h2integrate.tools.run_cases import modify_tech_config, load_tech_config_cases
+from h2integrate.core.h2integrate_model import H2IntegrateModel
+
+
+# Create a H2Integrate model
+model = H2IntegrateModel("12_ammonia_synloop.yaml")
+
+# Load cases
+case_file = Path("hb_inputs.csv")
+cases = load_tech_config_cases(case_file)
+
+# Modify and run the model for different cases
+caselist = [
+    "Haber Bosch Big",
+    "Haber Bosch Small",
+]
+for casename in caselist:
+    case = cases[casename]
+    model = modify_tech_config(model, case)
+    model.run()
+    model.post_process()
+```

examples/12_ammonia_synloop/hb_inputs.csv

@@ -0,0 +1,6 @@
+Index 0,Index 1,Index 2,Index 3,Index 4,Index 5,Type,Haber Bosch Big,Haber Bosch Small
+technologies,ammonia,model_inputs,shared_parameters,production_capacity,,float,100000,10000
+technologies,h2_storage,model_inputs,performance_parameters,type,,str,salt_cavern,lined_rock_cavern
+technologies,electrolyzer,model_inputs,performance_parameters,n_clusters,,int,16,2
+technologies,electrolyzer,model_inputs,performance_parameters,include_degradation_penalty,,bool,TRUE,FALSE
+technologies,electrolyzer,model_inputs,financial_parameters,capital_items,replacement_cost_percent,float,0.15,0.25

examples/12_ammonia_synloop/run_ammonia_synloop.py

@@ -1,10 +1,23 @@
+from pathlib import Path
+
+from h2integrate.tools.run_cases import modify_tech_config, load_tech_config_cases
 from h2integrate.core.h2integrate_model import H2IntegrateModel
 
 
 # Create a H2Integrate model
 model = H2IntegrateModel("12_ammonia_synloop.yaml")
 
-# Run the model
-model.run()
+# Load cases
+case_file = Path("hb_inputs.csv")
+cases = load_tech_config_cases(case_file)
 
-model.post_process()
+# Modify and run the model for different cases
+caselist = [
+    "Haber Bosch Big",
+    "Haber Bosch Small",
+]
+for casename in caselist:
+    case = cases[casename]
+    model = modify_tech_config(model, case)
+    model.run()
+    model.post_process()

h2integrate/tools/inflation/cepci.csv

@@ -27,3 +27,4 @@ Year,CEPCI
 2021,628
 2022,794
 2023,801
+2024,800

h2integrate/tools/run_cases.py

@@ -0,0 +1,131 @@
+import operator
+from functools import reduce
+
+import pandas as pd
+
+
+def cast_by_name(type_name, value):
+    """Cast a string read in from an input file as a data type also given as a string.
+    Currently allowed data types: int, float, bool, str
+
+    Args:
+        type_name (str): The data type to cast into, as a string.
+        value (str): The value, as a string.
+
+    Returns:
+        The value in the specified data type
+    """
+
+    bool_map = {"true": True, "false": False, "yes": True, "no": False, "1": True, "0": False}
+
+    trusted_types = ["int", "float", "bool", "str"]  ## others as needed
+    if type_name in trusted_types:
+        if type_name == "bool":
+            return bool_map.get(value.lower())
+        else:
+            return __builtins__[type_name](value)
+    else:
+        msg = f"Specified data type {type_name} invalid, must be one of {trusted_types}"
+        raise TypeError(msg)
+
+
+def get_from_dict(dataDict, mapList):
+    """Get value from nested dictionary using a list of keys.
+
+    Allows for programmatic calling of items in a nested dict using a variable-length list.
+    Instead of dataDict[item1][item2][item3][item4][item5], you can use
+    get_from_dict(dataDict, [item1, item2, item3, item4, item5]).
+
+    Args:
+        dataDict (dict): The nested dictionary to access.
+        mapList (list): List of keys to traverse the nested dictionary.
+
+    Returns:
+        The value at the specified nested location in the dictionary.
+
+    Example:
+        >>> data = {"a": {"b": {"c": 42}}}
+        >>> get_from_dict(data, ["a", "b", "c"])
+        42
+    """
+    return reduce(operator.getitem, mapList, dataDict)
+
+
+def set_in_dict(dataDict, mapList, value):
+    """Set value in nested dictionary using a list of keys.
+
+    Allows for programmatic setting of items in a nested dict using a variable-length list.
+    Instead of dataDict[item1][item2][item3][item4][item5] = value, you can use
+    set_in_dict(dataDict, [item1, item2, item3, item4, item5], value).
+
+    Args:
+        dataDict (dict): The nested dictionary to modify.
+        mapList (list): List of keys to traverse the nested dictionary.
+        value: The value to set at the specified nested location.
+
+    Example:
+        >>> data = {"a": {"b": {}}}
+        >>> set_in_dict(data, ["a", "b", "c"], 42)
+        >>> data["a"]["b"]["c"]
+        42
+    """
+    get_from_dict(dataDict, mapList[:-1])[mapList[-1]] = value
+
+
+def load_tech_config_cases(case_file):
+    """Load extensive lists of values from a spreadsheet to run many different cases.
+
+    Loads tech_config values from a CSV file to run multiple cases with different
+    technology configuration values.
+
+    Args:
+        case_file (Path): Path to the .csv file where the different tech_config values
+            are listed. The CSV must be formatted with "Index 1", "Index 2", etc.
+            columns followed by case name columns. Each row should have "technologies"
+            as the first index value, followed by tech_name and parameter names.
+
+    Returns:
+        pd.DataFrame: DataFrame with the indexes of the tech_config as a MultiIndex
+            and the different case names as the column names.
+
+    Note:
+        The CSV format should be:
+        |   "Index 1"    |...|  "Index <N>"   | "Type"  | <Case 1 Name>  |...| <Case N Name>  |
+        | "technologies" |...| <param_1_name> | "float" | <Case 1 value> |...| <Case N value> |
+        | "technologies" |...| <param_2_name> | "str"   | <Case 1 value> |...| <Case N value> |
+
+    If some parameters are nested deeper than others, make as many Index columns for the deepest-
+    nested parameters and leave any unused Indexes blank.
+
+    See example .csv in h2integrate/tools/test/test_inputs.csv
+    """
+    tech_config_cases = pd.read_csv(case_file)
+    column_names = tech_config_cases.columns.values
+    index_names = list(filter(lambda x: "Index" in x, column_names))
+    index_names.append("Type")
+    tech_config_cases = tech_config_cases.set_index(index_names)
+
+    return tech_config_cases
+
+
+def modify_tech_config(h2i_model, tech_config_case):
+    """Modify particular tech_config values on an existing H2I model before it is run.
+
+    Args:
+        h2i_model: H2IntegrateModel that has been set up but not run.
+        tech_config_case (pd.Series): Series that was indexed from tech_config_cases
+            DataFrame containing the parameter values to modify.
+
+    Returns:
+        H2IntegrateModel: The H2IntegrateModel with modified tech_config values.
+    """
+    for index_tup, value in tech_config_case.items():
+        index_list = list(index_tup)
+        data_type = index_list[-1]
+        index_list = index_list[:-1]
+        # Remove nans from blank index fields
+        while type(index_list[-1]) is not str:
+            index_list = index_list[:-1]
+        set_in_dict(h2i_model.technology_config, index_list, cast_by_name(data_type, value))
+
+    return h2i_model

h2integrate/tools/test/test_inputs.csv

@@ -0,0 +1,5 @@
+Index 0,Index 1,Index 2,Index 3,Index 4,Index 5,Type,Float Test,Int Test,Bool Test,Str Test
+technologies,solar,model_inputs,performance_parameters,pv_capacity_kWdc,,float,100000.,150000.,150000.,150000.
+technologies,electrolyzer,model_inputs,performance_parameters,n_clusters,,int,18,20,18,18
+technologies,electrolyzer,model_inputs,performance_parameters,include_degradation_penalty,,bool,TRUE,TRUE,FALSE,TRUE
+technologies,electrolyzer,model_inputs,performance_parameters,sizing,size_for,str,BOL,BOL,BOL,EOL

h2integrate/tools/test/test_tools.py

@@ -0,0 +1,55 @@
+import os
+from pathlib import Path
+
+import pytest
+
+from h2integrate import EXAMPLE_DIR
+from h2integrate.tools.run_cases import modify_tech_config, load_tech_config_cases
+from h2integrate.core.h2integrate_model import H2IntegrateModel
+
+
+def test_tech_config_modifier(subtests):
+    """Test cases for modifying and running tech_config from csv.
+    Using 15 example as test case
+    """
+
+    # Make an H2I model from the 15 example
+    os.chdir(EXAMPLE_DIR / "15_wind_solar_electrolyzer")
+    example_yaml = "15_wind_solar_electrolyzer.yaml"
+    model = H2IntegrateModel(example_yaml)
+
+    # Modify using csv
+    case_file = Path(__file__).resolve().parent / "test_inputs.csv"
+    cases = load_tech_config_cases(case_file)
+    with subtests.test("float"):
+        case = cases["Float Test"]
+        model = modify_tech_config(model, case)
+        model.run()
+        assert (
+            pytest.approx(model.prob.get_val("finance_subgroup_hydrogen.LCOH")[0], rel=1e-3)
+            == 5.327792370180044
+        )
+    with subtests.test("bool"):
+        case = cases["Bool Test"]
+        model = modify_tech_config(model, case)
+        model.run()
+        assert (
+            pytest.approx(model.prob.get_val("finance_subgroup_hydrogen.LCOH")[0], rel=1e-3)
+            == 5.226443205147294
+        )
+    with subtests.test("int"):
+        case = cases["Int Test"]
+        model = modify_tech_config(model, case)
+        model.run()
+        assert (
+            pytest.approx(model.prob.get_val("finance_subgroup_hydrogen.LCOH")[0], rel=1e-3)
+            == 5.4601971211592115
+        )
+    with subtests.test("str"):
+        case = cases["Str Test"]
+        model = modify_tech_config(model, case)
+        model.run()
+        assert (
+            pytest.approx(model.prob.get_val("finance_subgroup_hydrogen.LCOH")[0], rel=1e-3)
+            == 5.22644320514729
+        )