Call graphs from eesyplan methods by paulapreuss · Pull Request #479 · open-plan-tool/gui

paulapreuss · 2026-05-27T15:15:29Z

Goal

Save copy of datapackage to database to create enerysystem without having to build datapackage from scratch
Call graphs directly from eesyplan methods instead of openplan code

Notes:

Added datapackage field to Scenario instead of Simulation to not have to worry about if a simulation object already exists
Added argument store_database_record to to_jsonified_datapackage method instead of calling again with n_timestamps=0 to only build the datapackage once and strip the timeseries instead of building twice (once for simulation and once for db)
EnergySystem can be rebuilt from db datapackage after repopulating with Timeseries values
WIP state: Sankey diagram now works like this (renders) but does not display the correct values yet (probably something is different in the logic of rebuilding the flows?)

Fetches timeseries data from database based on foreignkeys and rebuilds the original datapackage structure

Bachibouzouk

Thanks for tackling this @paulapreuss - I think it looks nice. However I would wait that the PR oemof/oemof-datapackage#4 is merged (or check that with this version of oemof-datapackage it still works as expected)

Bachibouzouk · 2026-06-03T09:02:03Z

+    # TODO this sankey implementation does not allow for single timesteps
+    # if ts is not None:
+    #     ts = int(ts)
+    rebuilt_dp = add_timeseries_to_database_datapackage(scenario)


I am wondering whether we should add this as a method of the Scenario model. However this might add confusion if there are too many methods to get a datapackage or a jsonified datapackage from the Scenario. Because then one has to be sure where the datapackages comes from, from the DB or generated on the fly. And what if the user changed some components without resimulating and would like to export this as a datapackage.

If we decide to go for the method I would

rebuilt_dp = scenario.rebuilt_datapackage() and this method should return an error message if the datapackage attribute wasn't present (or it could generate it on the fly)

You're right, I think it makes a lot of sense to add this as a Scenario method instead.

Bachibouzouk · 2026-06-03T09:07:57Z

+        qs = FancyResults.objects.filter(simulation=scenario.simulation)
+
+        if qs.exists():
+            flows = {
+                (asset, bus): json.loads(flow_data)
+                for asset, bus, flow_data in qs.values_list("asset", "bus", "flow_data")
+            }
+            flows_df = pd.DataFrame(flows)
+            flows_df.index = scenario.get_timestamps()[: len(flows_df)]
+        fig, links_df = eesyplan_graphs.sankey(flows=flows_df, es=es)


In the future we should then get all this from the results datapackage I guess

Bachibouzouk · 2026-06-03T09:39:31Z

I updated the EZP simulation server with pip install "oemof-datapackage @ git+https://github.com/oemof/oemof-datapackage.git@fix/multilevel-columns-resources"

paulapreuss · 2026-06-04T08:46:33Z

Not sure about why, but now with these changes I get an CastError, seemingly about the order of the timeseries names

Traceback (most recent call last):
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\oemof\datapackage\datapackage\reading.py", line 185, in deserialize_energy_system       
    r.read()
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\datapackage\resource.py", line 383, in read
    return self.__get_table().read(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\tableschema\table.py", line 353, in read
    for count, row in enumerate(rows, start=1):
                      ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\tableschema\table.py", line 231, in iter
    exc_handler(
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\tableschema\helpers.py", line 90, in default_exc_handler
    raise exc
datapackage.exceptions.CastError: Table headers (['timeindex', 'PV - Halle 18_migration', 'Wind - Karholz_migration', 'Wind - Winnberg 03_migration', 'Wind - Winnberg 04_migration'
, 'Industriestandort_migration', 'PV - Erweiterung (Wasser)_migration']) don't match schema field names (['timeindex', 'Industriestandort_migration', 'PV - Halle 18_migration', 'PV - Erweiterung (Wasser)_migration', 'Wind - Karholz_migration', 'Wind - Winnberg 04_migration', 'Wind - Winnberg 03_migration']) in row 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\django\core\handlers\exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\django\core\handlers\base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\django\contrib\auth\decorators.py", line 59, in _view_wrapper
    return view_func(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\django\views\decorators\http.py", line 64, in inner
    return func(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\paula.preuss\PycharmProjects\gui\app\projects\views.py", line 1211, in scenario_review
    parse_ezp_results(simulation, simulation.results)
  File "C:\Users\paula.preuss\PycharmProjects\gui\app\projects\requests.py", line 170, in parse_ezp_results
    es = create_energy_system_from_dp(dp_path)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\oemof\eesyplan\datapackage\energy_system.py", line 16, in create_energy_system_from_dp  
    return EnergySystem.from_datapackage(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\paula.preuss\AppData\Local\miniforge3\envs\open-plan-5.2\Lib\site-packages\oemof\datapackage\datapackage\reading.py", line 187, in deserialize_energy_system       
    raise dp.exceptions.CastError(
datapackage.exceptions.CastError:
Metadata structure of resource `profiles` does not match data structure. Check the column names, types and their order.

Bachibouzouk · 2026-06-08T13:17:03Z

+            if qs.exists():
+                flows = {
+                    (asset, bus): json.loads(flow_data)
+                    for asset, bus, flow_data in qs.values_list(


This will always list asset on the first line and bus on the second line of the multiIndex column. I think now it is better to save the results as a jsonified datapackage and use

from oemof.eesyplan.datapackage.energy_system import ( create_energy_system_from_dp, ) from oemof.eesyplan import import_results from oemof.eesyplan.postprocessing.graphs import sankey energy_system = create_energy_system_from_dp(es_dp_path) results = import_results(destination_path, energy_system) fig, links_df = sankey(results["flow"], title="Test Sankey")

So you would add a datapackage_results and store it? I guess here it is mostly a question of how much slower (or not slower at all?) doing import_results for each graph might be compared to fetching ònly the relevant columns from the more structured database object.

This is why I might also want to store the graphs as dict/json within the datapackage under the folder graph

paulapreuss added 3 commits May 28, 2026 09:27

Allow to save datapackage copy to database

1200fcf

Add migration file

9ed9e77

Exclude datapackage field from scenario forms

9a9c1ee

paulapreuss force-pushed the feature/eesyplan-graphs branch from 8b96118 to db15e10 Compare May 28, 2026 08:44

Fix component type lookup

1154f2c

paulapreuss force-pushed the feature/eesyplan-graphs branch from db15e10 to e2e4a81 Compare May 28, 2026 11:50

paulapreuss added 4 commits May 28, 2026 19:23

Rehydrate db datapackage instead of rebuilding from scratch

72e47a2

Fetches timeseries data from database based on foreignkeys and rebuilds the original datapackage structure

Remove datapackage handling from simulation request

df3041c

Wrap rehydrate of dp in function

4cf40be

Display sankey from eesyplan method instead of openplan

21a33ef

paulapreuss force-pushed the feature/eesyplan-graphs branch from e2e4a81 to 21a33ef Compare May 28, 2026 17:25

paulapreuss requested a review from Bachibouzouk June 1, 2026 11:49

Bachibouzouk reviewed Jun 3, 2026

View reviewed changes

Bachibouzouk added 2 commits June 3, 2026 23:58

Adhere to the lighter json format for profiles

be4dc1b

Allow the display of both eesyplan and mvs results

84e825d

paulapreuss and others added 3 commits June 4, 2026 15:37

Fix CastError caused by timeseries order

b87e86e

Move rebuild datapackage method to Scenario class

9a069b9

Remove mention of deleted function

1c9646b

Bachibouzouk reviewed Jun 8, 2026

View reviewed changes

Bachibouzouk added 4 commits June 9, 2026 15:19

Add new field for datapackage results

4d6a1a7

Add migration file

ba4f00d

Save the ezp results in the corresponding Simulation's field

74dd9e2

Generate sankey from eezyplan method

42dad3f

Bachibouzouk mentioned this pull request Jun 9, 2026

The graphs linked to a simulation should be saved into the result datapackage structure #481

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Call graphs from eesyplan methods#479

Call graphs from eesyplan methods#479
paulapreuss wants to merge 17 commits into
mainfrom
feature/eesyplan-graphs

paulapreuss commented May 27, 2026 •

edited

Loading

Uh oh!

Bachibouzouk left a comment

Uh oh!

Bachibouzouk Jun 3, 2026

Uh oh!

paulapreuss Jun 3, 2026

Uh oh!

Bachibouzouk Jun 3, 2026

Uh oh!

Bachibouzouk commented Jun 3, 2026

Uh oh!

paulapreuss commented Jun 4, 2026

Uh oh!

Bachibouzouk Jun 8, 2026 •

edited

Loading

Uh oh!

paulapreuss Jun 8, 2026

Uh oh!

Bachibouzouk Jun 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

paulapreuss commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Bachibouzouk left a comment

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

paulapreuss Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk commented Jun 3, 2026

Uh oh!

paulapreuss commented Jun 4, 2026

Uh oh!

Bachibouzouk Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulapreuss Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

Bachibouzouk Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

paulapreuss commented May 27, 2026 •

edited

Loading

Bachibouzouk Jun 8, 2026 •

edited

Loading