Data from energycharts is wildly different than SMARD

[ArneTR]

I have heard they use different coefficients, but currently I am seeing not only a static offset, but a heavy different shape of the chart

Should it not at least follow a similar curve?

[ArneTR]

An addition to that: Today the energycharts graph "crashed" completely jumping down to 18 g .... this looks so weird that it feels like an error

The graph on https://www.energy-charts.info/charts/power/chart.htm?c=DE does not show this behaviour ...?

[ArneTR]

I just pulled the elephant fresh, rebuild the DB and plugged in all APIs.

Different then before I am now actually facing a situation where SMARD is way lower than what EM says.

Have you seen this before?

As you can see historical data from EM shows that Germany was around 300 g per night ... not 150 as Bundesnetzagentur / SMARD suggests ...

How can I find out if data is retrocatively updated?

[ribalba]

There is a table carbon_overwrites which you can look at.

[ArneTR]

The overwrites are brutal. On average every entry is ovewritten about 10 times.
We have 3979 entries in the carbon table and 37993 in the carbon_overwrites

Here a dump from one entry:

2026-05-11 20:15:00+00 | DE     | energycharts_de |        18 |      16.1 | t              | t              | 2026-05-10 16:05:54.819025+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |      16.1 |     364.7 | t              | t              | 2026-05-10 17:06:24.045366+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     364.7 |     364.5 | t              | t              | 2026-05-10 17:36:37.99859+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     364.5 |     365.6 | t              | t              | 2026-05-10 18:06:52.403138+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     365.6 |     365.5 | t              | t              | 2026-05-10 18:37:06.817374+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     365.5 |     365.6 | t              | t              | 2026-05-10 20:07:48.366445+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     365.6 |     365.7 | t              | t              | 2026-05-10 21:08:15.968424+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     365.7 |     365.8 | t              | t              | 2026-05-10 21:38:29.263607+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     365.8 |       366 | t              | t              | 2026-05-10 22:08:43.585247+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |       366 |     366.2 | t              | t              | 2026-05-10 23:09:08.69062+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     366.2 |     366.3 | t              | t              | 2026-05-11 00:09:29.519964+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     366.3 |     366.4 | t              | t              | 2026-05-11 01:09:46.998534+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     366.4 |     367.1 | t              | t              | 2026-05-11 01:39:55.379848+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     367.1 |     367.5 | t              | t              | 2026-05-11 02:40:22.553653+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     367.5 |     367.4 | t              | t              | 2026-05-11 03:40:41.424886+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     367.4 |     367.3 | t              | t              | 2026-05-11 04:35:57.556611+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     367.3 |     367.1 | t              | t              | 2026-05-11 05:06:07.158832+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     367.1 |       367 | t              | t              | 2026-05-11 05:38:14.323189+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |       367 |     366.9 | t              | t              | 2026-05-11 06:08:23.385551+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     366.9 |     366.8 | t              | t              | 2026-05-11 06:38:32.980598+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     366.8 |     366.4 | t              | t              | 2026-05-11 07:38:50.945708+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     366.4 |     366.6 | t              | t              | 2026-05-11 08:09:00.42243+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     366.6 |       371 | t              | t              | 2026-05-11 08:39:10.288106+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |       371 |     371.2 | t              | t              | 2026-05-11 09:09:19.488573+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     371.2 |     371.1 | t              | t              | 2026-05-11 09:39:30.319825+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     371.1 |     371.2 | t              | t              | 2026-05-11 10:09:39.358444+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     371.2 |     368.2 | t              | t              | 2026-05-11 10:39:49.113606+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     368.2 |     368.3 | t              | t              | 2026-05-11 11:04:56.834785+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     368.3 |     368.4 | t              | t              | 2026-05-11 11:40:23.133417+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     368.4 |     368.1 | t              | t              | 2026-05-11 12:41:14.039893+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     368.1 |     368.3 | t              | t              | 2026-05-11 13:06:22.940736+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     368.3 |     367.7 | t              | t              | 2026-05-11 15:23:31.964569+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     367.7 |     368.1 | t              | t              | 2026-05-11 17:24:14.139023+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     368.1 |     367.2 | t              | t              | 2026-05-11 18:24:34.762888+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     367.2 |     365.6 | t              | t              | 2026-05-11 19:24:54.949063+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     365.6 |     364.1 | t              | t              | 2026-05-11 20:20:27.907582+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     364.1 |     480.2 | t              | f              | 2026-05-11 21:20:48.611852+00
 2026-05-11 20:15:00+00 | DE     | energycharts_de |     480.2 |     556.5 | f              | f              | 2026-05-11 23:21:35.72201+00

The jump at the start I feel is kind of expected. But sadly there is also a big jump at the end of the timeseries. So effectively it takes over 24 hours for a value to stabilize

Electricitymaps is not so severe with max 2 overwrites:

2026-05-09 07:00:00+00 | NO-NO2 | electricitymaps_no-no2 |        32 |        41 | t              | t              | 2026-05-09 07:31:05.092518+00
 2026-05-09 07:00:00+00 | NO-NO2 | electricitymaps_no-no2 |        41 |        51 | t              | t              | 2026-05-09 07:51:13.270579+00

So although this situation alone is quite dire and makes the energycharts data unusable.

Bundesnetzagentur

This seems quite good now. But they also had days, where the API was unuasble with first going up then down.

       provider       |          time          |     old_value      |     new_value      |        overwritten_at
----------------------+------------------------+--------------------+--------------------+-------------------------------
 bundesnetzagentur_de | 2026-05-03 22:00:00+00 | 134.46716450391486 | 155.82250749936608 | 2026-05-05 20:52:10.707539+00
 bundesnetzagentur_de | 2026-05-03 22:00:00+00 | 155.82250749936608 | 159.35965747176104 | 2026-05-05 20:11:53.390706+00
 bundesnetzagentur_de | 2026-05-03 22:00:00+00 | 159.35965747176104 | 155.82250749936608 | 2026-05-05 20:16:56.1493+00
 bundesnetzagentur_de | 2026-05-03 22:00:00+00 | 105.07785881948126 |  66.78992280274504 | 2026-05-05 20:32:01.782829+00
 bundesnetzagentur_de | 2026-05-03 22:00:00+00 |  66.78992280274504 | 143.55592496916228 | 2026-05-05 20:37:04.020435+00
 bundesnetzagentur_de | 2026-05-03 22:00:00+00 | 143.55592496916228 | 105.07785881948126 | 2026-05-05 20:42:06.992235+00
 bundesnetzagentur_de | 2026-05-03 22:00:00+00 | 105.07785881948126 | 134.46716450391486 | 2026-05-05 20:47:08.643056+00
 bundesnetzagentur_de | 2026-05-03 22:00:00+00 | 155.82250749936608 | 105.07785881948126 | 2026-05-05 20:21:58.344209+00
 bundesnetzagentur_de | 2026-05-03 22:15:00+00 | 158.43078092909482 |   106.862238682704 | 2026-05-05 20:21:58.344209+00
 bundesnetzagentur_de | 2026-05-03 22:15:00+00 |  162.2981027854283 | 158.43078092909482 | 2026-05-05 20:16:56.1493+00
 bundesnetzagentur_de | 2026-05-03 22:15:00+00 | 137.30071218018878 | 158.43078092909482 | 2026-05-05 20:52:10.707539+00
 bundesnetzagentur_de | 2026-05-03 22:15:00+00 |   106.862238682704 | 137.30071218018878 | 2026-05-05 20:47:08.643056+00
 bundesnetzagentur_de | 2026-05-03 22:15:00+00 | 158.43078092909482 |  162.2981027854283 | 2026-05-05 20:11:53.390706+00
 bundesnetzagentur_de | 2026-05-03 22:15:00+00 | 146.37095154847157 |   106.862238682704 | 2026-05-05 20:42:06.992235+00
 bundesnetzagentur_de | 2026-05-03 22:15:00+00 |  68.28351829899046 | 146.37095154847157 | 2026-05-05 20:37:04.020435+00
 bundesnetzagentur_de | 2026-05-03 22:15:00+00 |   106.862238682704 |  68.28351829899046 | 2026-05-05 20:32:01.782829+00
...

One way to good visualise that could be an error bar per data point with min-max displayed also. Hereby being min the smallest know value from the overwrite table and max the biggest. This would show the variance of the API.

Through this we coud better reason which API we would later choose if we bring in more.

For now electricitymaps seems to be the only usable in terms of variance.

Having said that: It is still unclear what the most close value is to the real value.

Can we find out why Bundesnetzagentur and energycharts are so vastly different than electricitymaps?

Maybe look in Chris, who might now more about this?

[ribalba]

Crazy. The bundesnetzagentur is so different because we calculate the carbon value our selves depending on the grid

https://github.com/green-coding-solutions/elephant/blob/main/elephant/providers/bna_helper.py

The problem is that energy charts also just uses their own model like Electricty Maps. Everyone just uses differnt multipliers I assume.

[ArneTR]

@mrchrisadams - In case this ping reaches you: How do you handle the vastly different calculation methods from vendors? I know you have worked with Ember and Electricitymaps ... and I assume they also use different methodologies to go from a power source to a carbon value.

• Where the differences that stark?
• How did you compare one value with another? Did you do transformations?

[mrchrisadams]

hi fellas! To be honest, this is something we haven't had too much experience with ourselves yet, because in CO2.js, we've mainly focussed on annual readings, and only from two providers.

We have an monthly update for annual carbon intensity from Ember in this cronjob - it means that we have to update the figures for annual carbon intensity up to twelve times a year per country, as new data becomes available and it gets recalculated:

https://github.com/thegreenwebfoundation/co2.js/blob/main/.github/workflows/update-average-intensities.yml

For Electricity Maps, we're currently assuming an annual update cycle for now, until we see any wildly off numbers, and this is our first year including their data. We're still figuring out how often recalculation might happen there.

Because these both look back at the last year, it's not normally as uncertain as I think you're seeing here.

For the go-lang library we have that does connect to electricity maps, I believe there is some notion of data being 'modelled' or 'estimated' first, before it settles down to a more stable reading. This PR has a bit more context, as when working with time series data some databases assume data is immutable, which can be a pain with data that frequently gets updated like this carbon data.
thegreenwebfoundation/grid-intensity-go#87

Hope that helps!

[ArneTR]

ty for the feedback!

@ribalba Did you know that the results from /latest are tagged as estimation typically? (at least this is indicated here in a PR from 2024)

In any case I believe it changes nothing for us as we need the value from latest anyway and then overwrite after.

But then the take away is that we need to monitor variability in the later reported data and either disable current values for some APIs after they have stabilized or disable them.

[mrchrisadams]

If it helps, this also happens with market based carbon intensity figures too.

Companies have a time window in which they can buy credits to pair with their consumption of energy from the grid in order to say they have powered a bunch of energy consumption with clean energy.

It varies from utility to utility, but this is one of the reasons big cloud providers cite for being slow in updating carbon intensity figures in their calculators.