Skip to content

Commit dbdb801

Browse files
ribalbaribalba
authored
Adds the carbon provider documentation (#136)
* Adds the carbon provider documentation
1 parent 34a2b8a commit dbdb801

2 files changed

Lines changed: 303 additions & 0 deletions

File tree

‎content/en/docs/measuring/metric-providers/carbon-intensity-electricitymaps-machine.md‎

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
---
2+
title: "Carbon Intensity - Electricity Maps - Machine"
3+
description: "Documentation for CarbonIntensityElectricitymapsMachineProvider of the Green Metrics Tool"
4+
date: 2026-05-11T08:49:15+00:00
5+
weight: 220
6+
---
7+
8+
### What it does
9+
10+
This provider fetches the carbon intensity (gCO2e/kWh) of the electricity grid for a configured
11+
region from the [Electricity Maps API](https://www.electricitymaps.com/) and aligns the returned
12+
time series with the measurement window of the current run.
13+
14+
After every run the provider queries the *past-range* endpoint of the Electricity Maps API for the
15+
zone configured in the `config.yml` and uses the carbon intensity data points that fall between
16+
the `start_profiling` and `stop_profiling` timestamps. The values are then sampled into the
17+
sampling rate granularity of the Green Metrics Tool and stored as a regular metric of the run.
18+
19+
If the run is too short to produce data points on the *past-range* endpoint (the API typically lags
20+
by 5 minutes up to one hour, depending on the zone), the provider transparently falls back to the
21+
*forecast* endpoint and uses the value closest to the measurement window. This means that even
22+
short-running benchmarks have a carbon intensity value attached to them, although for very short
23+
runs this value will be a forecast rather than a measured grid value.
24+
25+
### Classname
26+
27+
- `CarbonIntensityElectricitymapsMachineProvider`
28+
29+
### Metric Name
30+
31+
- `carbon_intensity_electricity_maps_machine`
32+
33+
### Unit
34+
35+
- `gCO2e/kWh`
36+
37+
The provider stores integer values. The API returns floating point numbers which are rounded to
38+
the nearest integer before being persisted.
39+
40+
### Prerequisites & Installation
41+
42+
The provider is a pure Python provider and does not need any binary to be compiled. It does however
43+
require network access from the measurement machine to `api.electricitymaps.com` and a valid
44+
Electricity Maps API token.
45+
46+
You can obtain a free token (subject to fair-use limits) at
47+
[https://api-portal.electricitymaps.com/](https://api-portal.electricitymaps.com/).
48+
49+
Please note: The free token is for non-commercial use only. If you want to use Electricity Maps
50+
data in a commercial context you need to acquire a commercial license from Electricity Maps.
51+
52+
The provider must be configured in the `config.yml`:
53+
54+
```yml
55+
measurement:
56+
metric-providers:
57+
common:
58+
carbon_intensity_electricitymaps_machine:
59+
region: 'DE'
60+
token: 'XXXX'
61+
sampling_rate: 99 # Remove if you don't want value padding
62+
```
63+
64+
Please see [Configuration →]({{< relref "/docs/measuring/configuration" >}}) for further info.
65+
66+
### Input Parameters
67+
68+
- `region` — The Electricity Maps zone identifier (e.g. `DE`, `FR`, `US-CAL-CISO`). A list of
69+
available zones can be found in the [Electricity Maps API documentation](https://portal.electricitymaps.com/docs/getting-started#geographical-coverage).
70+
- `token` — A valid Electricity Maps API token. Without a token the provider will refuse to start.
71+
- `sampling_rate` — The sampling rate in milliseconds. This value is not used to call the API more
72+
frequently (the API is only called once per run) but to *pad* the returned values to the same
73+
resolution as the other metric providers so that the time series can be joined cleanly.
74+
75+
### Output
76+
77+
The provider does not produce a continuous Stdout stream like the C-based providers. Instead, the
78+
data is fetched once per run from the Electricity Maps API and inserted directly into the database
79+
as a metric of the run.
80+
81+
Each row consists of:
82+
83+
- `time`: The timestamp of the data point in microseconds (UNIX epoch).
84+
- `value`: The carbon intensity at that point in time in `gCO2e/kWh` as an integer.
85+
- `detail_name`: Always `electricity_maps` to indicate the data source.
86+
87+
Any errors that occur while talking to the API are appended to the provider's stderr buffer and can
88+
be inspected in the run details in the frontend.
89+
90+
### How it works
91+
92+
On `start_profiling` and `stop_profiling` the provider records UTC timestamps. When the metrics are
93+
read at the end of the run it issues a single HTTP GET request to
94+
95+
```
96+
https://api.electricitymaps.com/v4/carbon-intensity/past-range
97+
```
98+
99+
with the configured zone, the start/end times of the run, and a temporal granularity of 5 minutes.
100+
The response is a list of `{datetime, carbonIntensity}` entries which are filtered to the
101+
measurement window, sorted ascending in time, rounded to integer values and finally expanded to the
102+
configured `sampling_rate` so that the values align with the other providers' time series.
103+
104+
If the response is empty (typically because the run finished before the grid data caught up) the
105+
provider falls back to the forecast endpoint
106+
107+
```
108+
https://api.electricitymaps.com/v4/carbon-intensity/forecast
109+
```
110+
111+
and picks the forecast value closest to the measurement window.
112+
113+
### Health check
114+
115+
When the provider starts, it issues a small `past-range` request for the last hour to confirm that
116+
the configured zone is valid and that the API token is accepted. If the API returns `401` or `403`
117+
the provider raises a `MetricProviderConfigurationError` with a hint that the token is incorrect.
118+
If the API endpoint cannot be reached at all (DNS/network issues) the same error is raised with the
119+
underlying exception.
120+
121+
### Caveats
122+
123+
- The provider requires network access during the run. If the measurement machine is air-gapped
124+
this provider will not work. Use the [Elephant carbon intensity provider]({{< relref "carbon-intensity-elephant-machine" >}})
125+
in that case, which can be hosted locally.
126+
- The free Electricity Maps token is rate-limited. If you run a large number of measurements you
127+
may hit the limit and the provider will record an error and skip data for that run.
128+
- The smallest temporal granularity supported by this provider is 5 minutes. Short-running
129+
benchmarks (under a few minutes) will usually have only a single carbon intensity value
130+
associated with them.
131+
- The carbon intensity returned by Electricity Maps is a grid average, not a marginal value. If
132+
you need marginal values you have to compute them yourself based on the Electricity Maps
133+
*power-breakdown* endpoint, which is not supported by this provider.
134+
135+
### Troubleshooting
136+
137+
- *"Electricity Maps token was rejected"* — Check that the `token`
138+
field on the provider is set to a valid token from
139+
[https://api-portal.electricitymaps.com/](https://api-portal.electricitymaps.com/).
140+
- *"Electricity Maps base URL ... could not be reached"* — Check the network connectivity of the
141+
measurement machine and that no firewall blocks `api.electricitymaps.com`.
142+
- Empty time series in the frontend — The run was likely too short and even the forecast endpoint
143+
did not return a usable value. Increase the run duration or use the Elephant provider.

‎content/en/docs/measuring/metric-providers/carbon-intensity-elephant-machine.md‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
---
2+
title: "Carbon Intensity - Elephant - Machine"
3+
description: "Documentation for CarbonIntensityElephantMachineProvider of the Green Metrics Tool"
4+
date: 2026-05-11T08:49:15+00:00
5+
weight: 221
6+
---
7+
8+
### What it does
9+
10+
This provider fetches the carbon intensity (gCO2e/kWh) of the electricity grid from a self-hosted
11+
or remotely hosted [Elephant](https://github.com/green-coding-solutions/elephant) service.
12+
Elephant is a small HTTP service developed by us which aggregates carbon
13+
intensity data from multiple upstream sources (e.g. Bundesnetzagentur, Electricity Maps, ...) and exposes
14+
them through a uniform REST interface. It also offers the ability to *simulate* arbitrary carbon
15+
intensity curves so that the same workload can be re-evaluated under different grid scenarios.
16+
17+
This provider integrates Elephant into the Green Metrics Tool so that every run automatically
18+
gets a carbon intensity time series for the configured region and data provider attached to it
19+
without leaking any information to a third party.
20+
21+
### Classname
22+
23+
- `CarbonIntensityElephantMachineProvider`
24+
25+
### Metric Name
26+
27+
- `carbon_intensity_elephant_machine`
28+
29+
### Unit
30+
31+
- `gCO2e/kWh`
32+
33+
The provider stores integer values. The API returns floating point numbers which are rounded to
34+
the nearest integer before being persisted.
35+
36+
### Prerequisites & Installation
37+
38+
The provider is a pure Python provider and does not need any binary to be compiled. It does
39+
however require a reachable Elephant service. Elephant can be self-hosted via Docker; please
40+
follow the instructions in the
41+
[Elephant repository](https://github.com/green-coding-solutions/elephant) for the setup.
42+
43+
The provider must be configured in the `config.yml`:
44+
45+
```yml
46+
measurement:
47+
metric-providers:
48+
common:
49+
carbon_intensity_elephant_machine:
50+
region: 'DE'
51+
sampling_rate: 99 # Remove if you don't want value padding
52+
provider: 'bundesnetzagentur'
53+
elephant:
54+
host: localhost
55+
port: 8085
56+
protocol: http
57+
```
58+
59+
Please see [Configuration →]({{< relref "/docs/measuring/configuration" >}}) for further info.
60+
61+
### Input Parameters
62+
63+
- `region` — The region identifier as understood by Elephant (e.g. `DE`, `FR`). The list of
64+
supported regions depends on which upstream providers your Elephant instance has configured.
65+
- `provider` — The name of the carbon intensity data provider in Elephant to use (e.g.
66+
`bundesnetzagentur`, `entsoe`).
67+
- `elephant.host` — Host name or IP of the Elephant service.
68+
- `elephant.port` — Port the Elephant service listens on (default `8085`).
69+
- `elephant.protocol` — `http` or `https`.
70+
- `sampling_rate` — The sampling rate in milliseconds. As with the Electricity Maps provider this
71+
is used to *pad* the returned values so they align with the time series of the other providers
72+
rather than to query the API more often.
73+
74+
### Output
75+
76+
The provider does not produce a continuous Stdout stream. Once per run it queries the Elephant
77+
service and writes the results directly to the database as a metric of the run.
78+
79+
Each row consists of:
80+
81+
- `time`: The timestamp of the data point in microseconds (UNIX epoch).
82+
- `value`: The carbon intensity at that point in time in `gCO2e/kWh` as an integer.
83+
- `detail_name`: The name of the Elephant carbon provider that produced the value
84+
(e.g. `bundesnetzagentur_de`). For carbon simulation runs this is the simulation UUID.
85+
86+
Any errors that occur while talking to the Elephant service are appended to the provider's stderr
87+
buffer and can be inspected in the run details in the frontend.
88+
89+
### How it works
90+
91+
On `start_profiling` and `stop_profiling` the provider records UTC timestamps. When the metrics
92+
are read at the end of the run it issues a single HTTP GET request to
93+
94+
```
95+
<protocol>://<host>:<port>/carbon-intensity/history
96+
```
97+
98+
with the configured `region`, the start/end times of the run, the `provider` filter, and
99+
`update=true` so that Elephant re-fetches stale data from its upstream sources before responding.
100+
The response is a list of `{time, carbon_intensity, provider}` entries which are sorted, rounded
101+
to integer values and expanded to the configured `sampling_rate`.
102+
103+
If the response is empty — which typically happens for very short runs because upstream providers
104+
update at intervals between 5 minutes and one hour — the provider falls back to:
105+
106+
```
107+
<protocol>://<host>:<port>/carbon-intensity/current
108+
```
109+
110+
(or `/carbon-intensity/current/primary` if no provider filter is configured) and uses the most
111+
recent value as a single data point so that the run is never left without a carbon intensity.
112+
113+
### Health check
114+
115+
When the provider starts, it issues an HTTP GET against `/health` on the configured Elephant URL.
116+
The service is expected to respond with a JSON body containing `{"status": "healthy"}`. If the
117+
service is not reachable, returns a different status, or the response cannot be parsed, the
118+
provider raises a `MetricProviderConfigurationError` and the run will not start.
119+
120+
### Carbon simulation
121+
122+
The Elephant provider is also used by the Green Metrics Tool to drive *carbon simulations*. When
123+
running the runner with `--carbon-simulation <value>`:
124+
125+
- If `<value>` is a UUID, the simulation with that ID is used.
126+
- If `<value>` is a list of carbon values, the runner first POSTs it to Elephant's `/simulation`
127+
endpoint to register a new simulation, and then uses the returned UUID for the run.
128+
- If `<value>` is a single integer, that fixed intensity is applied to the whole run.
129+
130+
For simulation runs the provider sends `simulationId=<uuid>` as an additional query parameter to
131+
Elephant which then returns the simulated carbon intensity curve instead of the real grid data.
132+
Carbon simulations are useful to answer "what if?" questions such as how a workload would have
133+
behaved on a grid with a different generation mix without re-running the workload.
134+
135+
See the `--carbon-simulation` argument of `runner.py` for more details.
136+
137+
### Caveats
138+
139+
- The provider requires that the Elephant service is reachable from the measurement machine. If
140+
Elephant is hosted on a different network, make sure that firewalls and routing allow the
141+
connection.
142+
- The smallest temporal granularity supported by this provider depends on the upstream data
143+
provider configured in Elephant. Bundesnetzagentur for example updates roughly every 15 minutes.
144+
- The fallback to the `current` endpoint will yield only a single data point. Long runs should
145+
therefore always observe enough upstream data to avoid the fallback path.
146+
- Like all carbon intensity numbers in the Green Metrics Tool the values are stored as integers.
147+
Sub-`g/kWh` precision is lost;
148+
149+
### Troubleshooting
150+
151+
- *"Elephant base URL ... could not be reached"* — Check that `elephant.host`, `elephant.port`,
152+
and `elephant.protocol` are correct and that the Elephant service is running. Try
153+
`curl <protocol>://<host>:<port>/health` from the measurement machine.
154+
- *"Elephant service health check failed"* — Elephant responded but did not report `healthy`.
155+
Check the Elephant logs for missing upstream credentials or unreachable upstream APIs.
156+
- *"Please set the provider config option ..."* — You configured the provider without a
157+
`provider` entry and the run is not a carbon simulation. Either set `provider` in the
158+
`config.yml` or run with `--carbon-simulation`.
159+
- Empty time series in the frontend — The run was too short and even the fallback endpoint did
160+
not return a value. Increase the run duration or use a simulation.

0 commit comments

Comments
 (0)