Skip to content

Commit 58ce446

Browse files
committed
Add accessors for plotting, statistics and more (#506)
* Add planning doc * Finalize planning * Add plotting acessor * Add plotting acessor * Add tests * Improve * Improve * Update docs * Updated the plotting API so that .data always returns xarray DataArray or Dataset instead of pandas DataFrame. * All .data now returns xr.Dataset consistently. * Fixed Inconsistencies and Unused Parameters * New Plot Accessors results.plot.variable(pattern) Plots the same variable type across multiple elements for easy comparison. # All binary operation states across all components results.plot.variable('on') # All flow_rates, filtered to Boiler-related elements results.plot.variable('flow_rate', include='Boiler') # All storage charge states results.plot.variable('charge_state') # With aggregation results.plot.variable('flow_rate', aggregate='sum') Key features: - Searches all elements for variables matching the pattern - Filter with include/exclude on element names - Supports aggregation, faceting, and animation - Returns Dataset with element names as variables results.plot.duration_curve(variables) Plots load duration curves (sorted time series) showing utilization patterns. # Single variable results.plot.duration_curve('Boiler(Q_th)|flow_rate') # Multiple variables results.plot.duration_curve(['CHP|on', 'Boiler|on']) # Normalized x-axis (0-100%) results.plot.duration_curve('demand', normalize=True) Key features: - Sorts values from highest to lowest - Shows how often each power level is reached - normalize=True shows percentage of time on x-axis - Returns Dataset with duration_hours or duration_pct dimension * Fix duration curve * Fix duration curve * Fix duration curve * Fix duration curve * xr.apply_ufunc to sort along the time axis while preserving all other dimensions automatically * ⏺ The example runs successfully. Now let me summarize the fixes: Summary of Fixes I addressed the actionable code review comments from CodeRabbitAI: 1. Documentation Issue - reshape parameter name ✓ (No fix needed) The CodeRabbitAI comment was incorrect. The public API parameter in PlotAccessor.heatmap() is correctly named reshape (line 335). The reshape_time parameter exists in the lower-level heatmap_with_plotly function, but the documentation correctly shows the public API parameter. 2. Fixed simple_example.py (lines 129-130) Problem: The example called balance() and duration_curve() without required arguments, which would cause TypeError at runtime. Fix: Added the required arguments: - optimization.results.plot.balance('Fernwärme') - specifying the bus to plot - optimization.results.plot.duration_curve('Boiler(Q_th)|flow_rate') - specifying the variable to plot 3. Fixed variable collision in plot_accessors.py (lines 985-988) Problem: When building the Dataset in the variable() method, using element names as keys could cause collisions if multiple variables from the same element matched the pattern (e.g., 'Boiler|flow_rate' and 'Boiler|flow_rate_max' would both map to 'Boiler', with the latter silently overwriting the former). Fix: Changed to use the full variable names as keys instead of just element names: ds = xr.Dataset({var_name: self._results.solution[var_name] for var_name in filtered_vars}) All tests pass (40 passed, 1 skipped) and the example runs successfully. * make variable names public in results * Fix sankey * Fix effects() * Fix effects * Remove bargaps * made faceting consistent across all plot methods: | Method | facet_col | facet_row | |------------------|-------------------------------------------|-----------------------------| | balance() | 'scenario' | 'period' | | heatmap() | 'scenario' | 'period' | | storage() | 'scenario' | 'period' | | flows() | 'scenario' | 'period' | | effects() | 'scenario' | 'period' | | variable() | 'scenario' | 'period' | | duration_curve() | 'scenario' | 'period' (already had this) | | compare() | N/A (uses its own mode='overlay'/'facet') | | | sankey() | N/A (aggregates to single diagram) | | Removed animate_by parameter from all methods * Update storage method * Remove mode parameter for simpli | Method | Default mode | |------------------|---------------------------------------------------| | balance() | stacked_bar | | storage() | stacked_bar (flows) + line (charge state overlay) | | flows() | line | | variable() | line | | duration_curve() | line | | effects() | bar | * Make plotting_accessors.py more self contained * Use faster to_long() * Add 0-dim case * sankey diagram now properly handles scenarios and periods: Changes made: 1. Period aggregation with weights: Uses period_weights from flow_system to properly weight periods by their duration 2. Scenario aggregation with weights: Uses normalized scenario_weights to compute a weighted average across scenarios 3. Selection support: Users can filter specific scenarios/periods via select parameter before aggregation Weighting logic: - Periods (for aggregate='sum'): (da * period_weights).sum(dim='period') - this gives the total energy across all periods weighted by their duration - Periods (for aggregate='mean'): (da * period_weights).sum(dim='period') / period_weights.sum() - weighted mean - Scenarios: Always uses normalized weights (sum to 1) for weighted averaging, since scenarios represent probability-weighted alternatives * Add colors to sankey * Add sizes * Add size filtering * Include storage sizes * Remove storage sizes * Add charge state and status accessor * Summary of Changes 1. Added new methods to PlotAccessor (plot_accessors.py) charge_states() (line 658): - Returns a Dataset with each storage's charge state as a variable - Supports filtering with include/exclude parameters - Default plot: line chart on_states() (line 753): - Returns a Dataset with each component's |status variable - Supports filtering with include/exclude parameters - Default plot: heatmap (good for binary data visualization) 2. Added data building helper functions (plot_accessors.py) build_flow_rates(results) (line 315): - Builds a DataArray containing flow rates for all flows - Used internally by PlotAccessor methods build_flow_hours(results) (line 333): - Builds a DataArray containing flow hours for all flows build_sizes(results) (line 347): - Builds a DataArray containing sizes for all flows _filter_dataarray_by_coord(da, **kwargs) (line 284): - Helper for filtering DataArrays by coordinate values 3. Deprecated old Results methods (results.py) The following methods now emit DeprecationWarning: - results.flow_rates() → Use results.plot.flows(plot=False).data - results.flow_hours() → Use results.plot.flows(unit='flow_hours', plot=False).data - results.sizes() → Use results.plot.sizes(plot=False).data 4. Updated PlotAccessor methods to use new helpers - flows() now uses build_flow_rates() / build_flow_hours() directly - sizes() now uses build_sizes() directly - sankey() now uses build_flow_hours() directly This ensures the deprecation warnings only fire when users directly call the old methods, not when using the plot accessor * 1. New methods added to PlotAccessor - charge_states(): Returns Dataset with all storage charge states - on_states(): Returns Dataset with all component status variables (heatmap display) 2. Data building helper functions (plot_accessors.py) - build_flow_rates(results): Builds DataArray of flow rates - build_flow_hours(results): Builds DataArray of flow hours - build_sizes(results): Builds DataArray of sizes - _filter_dataarray_by_coord(da, **kwargs): Filter helper - _assign_flow_coords(da, results): Add flow coordinates 3. Caching in PlotAccessor Added lazy-cached properties for expensive computations: - _all_flow_rates - cached DataArray of all flow rates - _all_flow_hours - cached DataArray of all flow hours - _all_sizes - cached DataArray of all sizes - _all_charge_states - cached Dataset of all storage charge states - _all_status_vars - cached Dataset of all status variables 4. Deprecated methods in Results class Added deprecation warnings to: - results.flow_rates() → Use results.plot.flows(plot=False).data - results.flow_hours() → Use results.plot.flows(unit='flow_hours', plot=False).data - results.sizes() → Use results.plot.sizes(plot=False).data 5. Updated PlotAccessor methods to use cached properties - flows() uses _all_flow_rates / _all_flow_hours - sankey() uses _all_flow_hours - sizes() uses _all_sizes - charge_states() uses _all_charge_states - on_states() uses _all_status_vars * Move deprectated functionality into results.py instead of porting to the new module * Revert to simply deprectae old methods without forwarding to new code * Remove planning file * Update plotting methods for new datasets * 1. Renamed data properties in PlotAccessor to use all_ prefix: - all_flow_rates - All flow rates as Dataset - all_flow_hours - All flow hours as Dataset - all_sizes - All flow sizes as Dataset - all_charge_states - All storage charge states as Dataset - all_on_states - All component on/off status as Dataset 2. Updated internal references - All usages in flows(), sankey(), sizes(), charge_states(), and on_states() methods now use the new names. 3. Updated deprecation messages in results.py to point to the new API: - results.flow_rates() → results.plot.all_flow_rates - results.flow_hours() → results.plot.all_flow_hours - results.sizes() → results.plot.all_sizes 4. Updated docstring examples in PlotAccessor to use the new all_* names. * Update deprecations messages * Update deprecations messages * Thsi seems much better. * Updaet docstrings and variable name generation in plotting acessor * Change __ to _ in private dataset caching * Revert breaking io changes * New solution storing interface * Add new focused statistics and plot accessors * Renamed all properties: - all_flow_rates → flow_rates - all_flow_hours → flow_hours - all_sizes → sizes - all_charge_states → charge_states * Cache Statistics * Invalidate caches * Add effect related statistics * Simplify statistics accessor to rely on flow_system directly instead of solution attrs * Fix heatma fallback for 1D Data * Add topology accessor * All deprecation warnings in the codebase now consistently use the format will be removed in v{DEPRECATION_REMOVAL_VERSION}. * Update tests * created comprehensive documentation for all FlowSystem accessors * Update results documentation * Update results documentation * Update effect statistics * Update effect statistics * Update effect statistics * Add mkdocs plotly plugin * Add section about custom constraints * documentation updates: docs/user-guide/results/index.md: - Updated table to replace effects_per_component with temporal_effects, periodic_effects, total_effects, and effect_share_factors - Fixed flow_hours['Boiler(Q_th)|flow_rate'] → flow_hours['Boiler(Q_th)'] - Fixed sizes['Boiler(Q_th)|size'] → sizes['Boiler(Q_th)'] - Replaced effects_per_component example with new effect properties and groupby examples - Updated complete example to use total_effects docs/user-guide/results-plotting.md: - Fixed colors example from 'Boiler(Q_th)|flow_rate' → 'Boiler(Q_th)' - Fixed duration_curve examples to use clean labels docs/user-guide/migration-guide-v6.md: - Added new "Statistics Accessor" section explaining the clean labels and new effect properties * implemented the effects() method in StatisticsPlotAccessor at flixopt/statistics_accessor.py:1132-1258. Summary of what was done: 1. Implemented effects() method in StatisticsPlotAccessor class that was missing but documented - Takes aspect parameter: 'total', 'temporal', or 'periodic' - Takes effect parameter to filter to a specific effect (e.g., 'costs', 'CO2') - Takes by parameter: 'component' or 'time' for grouping - Supports all standard plotting parameters: select, colors, facet_col, facet_row, show - Returns PlotResult with both data and figure 2. Verified the implementation works with all parameter combinations: - Default call: flow_system.statistics.plot.effects() - Specific effect: flow_system.statistics.plot.effects(effect='costs') - Temporal aspect: flow_system.statistics.plot.effects(aspect='temporal') - Temporal by time: flow_system.statistics.plot.effects(aspect='temporal', by='time') - Periodic aspect: flow_system.statistics.plot.effects(aspect='periodic') * Remove intermediate plot accessor * 1. pyproject.toml: Removed duplicate mkdocs-plotly-plugin>=0.1.3 entry (kept the exact pin ==0.1.3) 2. flixopt/plotting.py: Fixed dimension name consistency by using squeezed_data.name instead of data.name in the fallback heatmap logic 3. flixopt/statistics_accessor.py: - Fixed _dataset_to_long_df() to only use coordinates that are actually present as columns after reset_index() - Fixed the nested loop inefficiency with include_flows by pre-computing the flows list outside the loop - (Previously fixed) Fixed asymmetric NaN handling in validation check * _create_effects_dataset method in statistics_accessor.py was simplified: 1. Detect contributors from solution data variables instead of assuming they're only flows - Uses regex pattern to find {contributor}->{effect}(temporal|periodic) variables - Contributors can be flows OR components (e.g., components with effects_per_active_hour) 2. Exclude effect-to-effect shares - Filters out contributors whose base name matches any effect label - For example, costs(temporal) is excluded because costs is an effect label - These intermediate shares are already included in the computation 3. Removed the unused _compute_effect_total method - The new simplified implementation directly looks up shares from the solution - Uses effect_share_factors for conversion between effects 4. Key insight from user: The solution already contains properly computed share values including all effect-to-effect conversions. The computation uses conversion factors because derived effects (like Effect1 which shares 0.5 from costs) don't have direct {flow}->Effect1(temporal) variables - only the source effect shares exist ({flow}->costs(temporal)). * Update docs * Improve to_netcdf method * Update examples * Fix IIS computaion flag * Fix examples * Fix faceting in heatmap and use period as facet col everywhere * Inline plotting methods to deprecate plotting.py (#508) * Inline plotting methods to deprecate plotting.py * Fix test * Simplify Color Management * ColorType is now defined in color_processing.py and imported into statistics_accessor.py. * Fix ColorType typing * statistics_accessor.py - Heatmap colors type safety (lines 121-148, 820-853) - Changed _heatmap_figure() parameter type from colors: ColorType = None to colors: str | list[str] | None = None - Changed heatmap() method parameter type similarly - Updated docstrings to clarify that dicts are not supported for heatmaps since px.imshow's color_continuous_scale only accepts colorscale names or lists 2. statistics_accessor.py - Use configured qualitative colorscale (lines 284, 315) - Updated _create_stacked_bar() to use CONFIG.Plotting.default_qualitative_colorscale as the default colorscale - Updated _create_line() similarly - This ensures user-configured CONFIG.Plotting.default_qualitative_colorscale affects all bar/line plots consistently 3. topology_accessor.py - Path type alignment (lines 219-222) - Added normalization of path=False to None before calling _plot_network() - This resolves the type mismatch where TopologyAccessor.plot() accepts bool | str | Path but _plot_network() only accepts str | Path | None * fix usage if index name in aggregation plot
1 parent 2cb7a4f commit 58ce446

26 files changed

Lines changed: 3193 additions & 384 deletions

docs/home/quick-start.md

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -88,21 +88,31 @@ battery = fx.Storage(
8888
flow_system.add_elements(solar, demand, battery, electricity_bus)
8989
```
9090

91-
### 5. Run Optimization
91+
### 5. Visualize and Run Optimization
9292

9393
```python
94-
# Run optimization directly on the flow system
94+
# Optional: visualize your system structure
95+
flow_system.topology.plot(path='system.html')
96+
97+
# Run optimization
9598
flow_system.optimize(fx.solvers.HighsSolver())
9699
```
97100

98-
### 6. Access Results
101+
### 6. Access and Visualize Results
99102

100103
```python
101-
# Access results directly from the flow system
104+
# Access raw solution data
102105
print(flow_system.solution)
103106

104-
# Or access component-specific results
107+
# Use statistics for aggregated data
108+
print(flow_system.statistics.flow_hours)
109+
110+
# Access component-specific results
105111
print(flow_system.components['battery'].solution)
112+
113+
# Visualize results
114+
flow_system.statistics.plot.balance('electricity')
115+
flow_system.statistics.plot.storage('battery')
106116
```
107117

108118
### 7. Save Results (Optional)
@@ -132,8 +142,10 @@ Most flixOpt projects follow this pattern:
132142
2. **Create flow system** - Initialize with time series and effects
133143
3. **Add buses** - Define connection points
134144
4. **Add components** - Create generators, storage, converters, loads
135-
5. **Run optimization** - Call `flow_system.optimize(solver)`
136-
6. **Access Results** - Via `flow_system.solution` or component `.solution` attributes
145+
5. **Verify structure** - Use `flow_system.topology.plot()` to visualize
146+
6. **Run optimization** - Call `flow_system.optimize(solver)`
147+
7. **Analyze results** - Via `flow_system.statistics` and `.solution`
148+
8. **Visualize** - Use `flow_system.statistics.plot.*` methods
137149

138150
## Tips
139151

docs/user-guide/core-concepts.md

Lines changed: 21 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -127,23 +127,29 @@ Define your system structure, parameters, and time series data.
127127

128128
### 2. Run the Optimization
129129

130-
Create an [`Optimization`][flixopt.optimization.Optimization] and solve it:
130+
Optimize your FlowSystem with a solver:
131131

132132
```python
133-
optimization = fx.Optimization('my_model', flow_system)
134-
results = optimization.solve(fx.solvers.HighsSolver())
133+
flow_system.optimize(fx.solvers.HighsSolver())
135134
```
136135

137136
### 3. Analyze Results
138137

139-
The [`Results`][flixopt.results.Results] object contains all solution data:
138+
Access solution data directly from the FlowSystem:
140139

141140
```python
142-
# Access component results
143-
boiler_output = results['Boiler'].node_balance()
141+
# Access component solutions
142+
boiler = flow_system.components['Boiler']
143+
print(boiler.solution)
144144

145145
# Get total costs
146-
total_costs = results.solution['Costs']
146+
total_costs = flow_system.solution['costs|total']
147+
148+
# Use statistics for aggregated data
149+
print(flow_system.statistics.flow_hours)
150+
151+
# Plot results
152+
flow_system.statistics.plot.balance('HeatBus')
147153
```
148154

149155
<figure markdown>
@@ -185,12 +191,17 @@ While our example used a heating system, flixOpt works for any flow-based optimi
185191
flixOpt is built on [linopy](https://github.com/PyPSA/linopy). You can access and extend the underlying optimization model for custom constraints:
186192

187193
```python
188-
# Access the linopy model after building
189-
optimization.do_modeling()
190-
model = optimization.model
194+
# Build the model (without solving)
195+
flow_system.build_model()
196+
197+
# Access the linopy model
198+
model = flow_system.model
191199

192200
# Add custom constraints using linopy API
193201
model.add_constraints(...)
202+
203+
# Then solve
204+
flow_system.solve(fx.solvers.HighsSolver())
194205
```
195206

196207
This allows advanced users to add domain-specific constraints while keeping flixOpt's convenience for standard modeling.

docs/user-guide/migration-guide-v6.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,31 @@ The new API also applies to advanced optimization modes:
296296

297297
---
298298

299+
## Statistics Accessor
300+
301+
The new `statistics` accessor provides convenient aggregated data:
302+
303+
```python
304+
stats = flow_system.statistics
305+
306+
# Flow data (clean labels, no |flow_rate suffix)
307+
stats.flow_rates['Boiler(Q_th)'] # Not 'Boiler(Q_th)|flow_rate'
308+
stats.flow_hours['Boiler(Q_th)']
309+
stats.sizes['Boiler(Q_th)']
310+
stats.charge_states['Battery']
311+
312+
# Effect breakdown by contributor (replaces effects_per_component)
313+
stats.temporal_effects['costs'] # Per timestep, per contributor
314+
stats.periodic_effects['costs'] # Investment costs per contributor
315+
stats.total_effects['costs'] # Total per contributor
316+
317+
# Group by component or component type
318+
stats.total_effects['costs'].groupby('component').sum()
319+
stats.total_effects['costs'].groupby('component_type').sum()
320+
```
321+
322+
---
323+
299324
## 🔧 Quick Reference
300325

301326
### Common Conversions

docs/user-guide/optimization/index.md

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,21 @@
22

33
This section covers how to run optimizations in flixOpt, including different optimization modes and solver configuration.
44

5+
## Verifying Your Model
6+
7+
Before running an optimization, it's helpful to visualize your system structure:
8+
9+
```python
10+
# Generate an interactive network diagram
11+
flow_system.topology.plot(path='my_system.html')
12+
13+
# Or get structure info programmatically
14+
nodes, edges = flow_system.topology.infos()
15+
print(f"Components: {[n for n, d in nodes.items() if d['class'] == 'Component']}")
16+
print(f"Buses: {[n for n, d in nodes.items() if d['class'] == 'Bus']}")
17+
print(f"Flows: {list(edges.keys())}")
18+
```
19+
520
## Standard Optimization
621

722
The recommended way to run an optimization is directly on the `FlowSystem`:
@@ -78,6 +93,107 @@ print(clustered_fs.solution)
7893
| Standard | Small-Medium | Slow | Optimal |
7994
| Clustered | Very Large | Fast | Approximate |
8095

96+
## Custom Constraints
97+
98+
flixOpt is built on [linopy](https://github.com/PyPSA/linopy), allowing you to add custom constraints beyond what's available through the standard API.
99+
100+
### Adding Custom Constraints
101+
102+
To add custom constraints, build the model first, then access the underlying linopy model:
103+
104+
```python
105+
# Build the model (without solving)
106+
flow_system.build_model()
107+
108+
# Access the linopy model
109+
model = flow_system.model
110+
111+
# Access variables from the solution namespace
112+
# Variables are named: "ElementLabel|variable_name"
113+
boiler_flow = model.variables['Boiler(Q_th)|flow_rate']
114+
chp_flow = model.variables['CHP(Q_th)|flow_rate']
115+
116+
# Add a custom constraint: Boiler must produce at least as much as CHP
117+
model.add_constraints(
118+
boiler_flow >= chp_flow,
119+
name='boiler_min_chp'
120+
)
121+
122+
# Solve with the custom constraint
123+
flow_system.solve(fx.solvers.HighsSolver())
124+
```
125+
126+
### Common Use Cases
127+
128+
**Minimum runtime constraint:**
129+
```python
130+
# Require component to run at least 100 hours total
131+
on_var = model.variables['CHP|on'] # Binary on/off variable
132+
hours = flow_system.hours_per_timestep
133+
model.add_constraints(
134+
(on_var * hours).sum() >= 100,
135+
name='chp_min_runtime'
136+
)
137+
```
138+
139+
**Linking flows across components:**
140+
```python
141+
# Heat pump and boiler combined must meet minimum base load
142+
hp_flow = model.variables['HeatPump(Q_th)|flow_rate']
143+
boiler_flow = model.variables['Boiler(Q_th)|flow_rate']
144+
model.add_constraints(
145+
hp_flow + boiler_flow >= 50, # At least 50 kW combined
146+
name='min_heat_supply'
147+
)
148+
```
149+
150+
**Seasonal constraints:**
151+
```python
152+
import pandas as pd
153+
154+
# Different constraints for summer vs winter
155+
summer_mask = flow_system.timesteps.month.isin([6, 7, 8])
156+
winter_mask = flow_system.timesteps.month.isin([12, 1, 2])
157+
158+
flow_var = model.variables['Boiler(Q_th)|flow_rate']
159+
160+
# Lower capacity in summer
161+
model.add_constraints(
162+
flow_var.sel(time=flow_system.timesteps[summer_mask]) <= 100,
163+
name='summer_limit'
164+
)
165+
```
166+
167+
### Inspecting the Model
168+
169+
Before adding constraints, inspect available variables and existing constraints:
170+
171+
```python
172+
flow_system.build_model()
173+
model = flow_system.model
174+
175+
# List all variables
176+
print(model.variables)
177+
178+
# List all constraints
179+
print(model.constraints)
180+
181+
# Get details about a specific variable
182+
print(model.variables['Boiler(Q_th)|flow_rate'])
183+
```
184+
185+
### Variable Naming Convention
186+
187+
Variables follow this naming pattern:
188+
189+
| Element Type | Pattern | Example |
190+
|--------------|---------|---------|
191+
| Flow rate | `Component(FlowLabel)\|flow_rate` | `Boiler(Q_th)\|flow_rate` |
192+
| Flow size | `Component(FlowLabel)\|size` | `Boiler(Q_th)\|size` |
193+
| On/off status | `Component\|on` | `CHP\|on` |
194+
| Charge state | `Storage\|charge_state` | `Battery\|charge_state` |
195+
| Effect totals | `effect_name\|total` | `costs\|total` |
196+
81197
## Solver Configuration
82198

83199
### Available Solvers

0 commit comments

Comments
 (0)