Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/_sources/advanced_installation.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ This section describes how to deploy CodeCarbon as a system service using Ansibl
It automate the manual installation done in the previous chapter.

What the Playbook Does
--------------------
----------------------
The Ansible playbook automates the following tasks:

* Creates a dedicated system user and group for CodeCarbon
Expand All @@ -134,14 +134,14 @@ The Ansible playbook automates the following tasks:
* Starts and enables the service

Prerequisites
------------
-------------
* Ansible installed on your machine
* Debian-based target system(s)
* SSH access to target system(s)
* CodeCarbon API credentials from the dashboard

Directory Structure
-----------------
-------------------
.. code-block:: text

codecarbon/deploy/ansible/codecarbon_cli_as_a_service/
Expand All @@ -158,11 +158,11 @@ Directory Structure
└── main.yml

Quick Start
----------
-----------

1. Set the the target to install in ``hosts``:

.. code-block:: txt
.. code-block:: text

yourservername.yourdomain.com hostname=yourservername ansible_user=root ansible_ssh_private_key_file=~/.ssh/id_ed25519

Expand Down
4 changes: 2 additions & 2 deletions docs/_sources/comet.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Comet Integration
===========================


CodeCarbon can be automatically integrated with `Comet <https://www.comet.ml/site/>`_ for experiment tracking and visualization. Comet provides data scientists with powerful tools to track, compare, explain, and reproduce their experiments. Now, with CodeCarbon you can easily track the carbon footprint of your jobs along with your training metrics, hyperparameters, dataset samples, artifacts, and more.
CodeCarbon can be automatically integrated with `Comet <https://www.comet.ml/site/>`__ for experiment tracking and visualization. Comet provides data scientists with powerful tools to track, compare, explain, and reproduce their experiments. Now, with CodeCarbon you can easily track the carbon footprint of your jobs along with your training metrics, hyperparameters, dataset samples, artifacts, and more.

.. image:: ./images/comet-workspace.png
:align: center
Expand All @@ -19,7 +19,7 @@ To get started with the Comet-CodeCarbon integration, make sure you have comet-m
pip install comet_ml>=3.2.2


Go to `Comet's website <https://www.comet.ml/site/>`_ and create a free account. From your account settings page, copy your personal API key.
Go to `Comet's website <https://www.comet.ml/site/>`__ and create a free account. From your account settings page, copy your personal API key.

In the `mnist-comet.py <https://github.com/mlco2/codecarbon/blob/master/examples/mnist-comet.py>`_ example file, replace the placeholder code with your API key:

Expand Down
6 changes: 3 additions & 3 deletions docs/_sources/faq.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,16 @@ Frequently Asked Questions

- When available we use data from `ourworld in data <https://ourworldindata.org/grapher/carbon-intensity-electricity?tab=table>`_

- if not available we use the electricity mix of the country find on `globalpetrolprices.com <https://www.globalpetrolprices.com/energy_mix.php>`_ that we multiply by the carbon intensity of the source of electricity (`that you can find here <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/carbon_intensity_per_source.json>`_)
- If not available we use the electricity mix of the country find on `globalpetrolprices.com <https://www.globalpetrolprices.com/energy_mix.php>`_ that we multiply by the carbon intensity of the source of electricity (`that you can find here <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/carbon_intensity_per_source.json>`_)

- if we have neither we default to a world average of 475 gCO2.eq/KWh from `IEA <https://www.iea.org/reports/global-energy-co2-status-report-2019/emissions>`_.
- If we have neither we default to a world average of 475 gCO2.eq/KWh from `IEA <https://www.iea.org/reports/global-energy-co2-status-report-2019/emissions>`_.


* **How do I offset my emissions?**
There are many offsetting schemes, and it is hard to recommend any single one. For one-shot offsetting, the `Gold Standard <https://www.goldstandard.org/>`_ is often used, and has many offsetting projects to choose from at different prices. There are often local initiatives as well, so try researching what exists in your region/country. For a recurring offset, `Project Wren <https://projectwren.com/>`_ lets you estimate your monthly carbon emissions and offset them via a monthly subscription. Keep in mind that offsetting is a good choice, but *reducing your emissions* should be the priority.


* **How can I help?**
If you find a functionality missing in the CodeCarbon repo, please `open an issue <https://github.com/mlco2/codecarbon/issues>`_ so that you (and others!) can help add it. We did our best to cover all use cases and options, but we count on the open source community to help make the package an even greater success.
If you find any functionality missing in the CodeCarbon repo, please `open an issue <https://github.com/mlco2/codecarbon/issues>`_ so that you (and others!) can help add it. We did our best to cover all use cases and options, but we count on the open source community to help make the package an even greater success.


4 changes: 2 additions & 2 deletions docs/_sources/installation.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ For installing conda, follow the instructions on the
From PyPi repository
--------------------

The package is hosted on the pip repository `here <https://pypi.org/project/codecarbon/>`_.
The package is hosted on the pip repository `here <https://pypi.org/project/codecarbon/>`__.

To install the package, run the following command in your terminal.

Expand All @@ -26,7 +26,7 @@ To install the package, run the following command in your terminal.
From conda repository
---------------------

The package is hosted on the conda repository `here <https://anaconda.org/codecarbon/codecarbon>`_.
The package is hosted on the conda repository `here <https://anaconda.org/codecarbon/codecarbon>`__.

To install the package, run the following command in your terminal.

Expand Down
42 changes: 23 additions & 19 deletions docs/_sources/methodology.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ energy sources that are used to generate electricity, including fossil fuels and
:height: 300px
:width: 350px

When available, CodeCarbon uses global carbon intensity of electricity per cloud provider ( `here <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/cloud/impact.csv>`_ ) or per country ( `here <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/global_energy_mix.json>`_ ).
When available, CodeCarbon uses global carbon intensity of electricity per cloud provider ( `here <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/cloud/impact.csv>`__) or per country ( `here <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/global_energy_mix.json>`__ ).

If we don't have the global carbon intensity or electricity of a country, but we have its electricity mix, we used to compute the carbon intensity of electricity using this table:

Expand Down Expand Up @@ -63,9 +63,9 @@ Then, for example, if the Energy Mix of the Grid Electricity is 25% Coal, 35% Pe

Net Carbon Intensity = 0.25 * 995 + 0.35 * 816 + 0.26 * 743 + 0.14 * 29 = 731.59 kgCO₂/kWh

But it doesn't happen anymore because Our World in Data now provides the global carbon intensity of electricity per country ( `source <https://ourworldindata.org/grapher/carbon-intensity-electricity#explore-the-data>`_ ). Some countries are missing data for last year, so we use the previous year data available.
But it doesn't happen anymore because Our World in Data now provides the global carbon intensity of electricity per country ( `source <https://ourworldindata.org/grapher/carbon-intensity-electricity#explore-the-data>`__ ). Some countries are missing data for last year, so we use the previous year data available.

If ever we have neither the global carbon intensity of a country nor it's electricity mix, we apply a world average of 475 gCO2.eq/KWh ( `source <https://www.iea.org/reports/global-energy-co2-status-report-2019/emissions>`_ ).
If ever we have neither the global carbon intensity of a country nor it's electricity mix, we apply a world average of 475 gCO2.eq/KWh ( `source <https://www.iea.org/reports/global-energy-co2-status-report-2019/emissions>`__ ).

As you can see, we try to be as accurate as possible in estimating carbon intensity of electricity. Still there is room for improvement and all contributions are welcome.

Expand All @@ -86,7 +86,7 @@ Tracks Nvidia GPUs energy consumption using ``pynvml`` library (installed with t
RAM
~~~~

CodeCarbon v2 uses a 3 Watts for 8 GB ratio `source <https://www.crucial.com/support/articles-faq-memory/how-much-power-does-memory-use>`_ .
CodeCarbon v2 uses a 3 Watts for 8 GB ratio `source <https://www.crucial.com/support/articles-faq-memory/how-much-power-does-memory-use>`__ .

But this is not a good measure because it doesn't take into account the number of RAM slots used in the machine, that really drive the power consumption, not the amount of RAM.
For example, in servers you could have thousands of GB of RAM but the power consumption would not be proportional to the amount of memory used, but to the number of memory modules used.
Expand Down Expand Up @@ -143,7 +143,7 @@ CPU

- **On Windows or Mac (Intel)**

Tracks Intel processors energy consumption using the ``Intel Power Gadget``. You need to install it yourself from this `source <https://www.intel.com/content/www/us/en/developer/articles/tool/power-gadget.html>`_ . But has been discontinued. There is a discussion about it on `github issues #457 <https://github.com/mlco2/codecarbon/issues/457>`_.
Tracks Intel processors energy consumption using the ``Intel Power Gadget``. You need to install it yourself from this `source <https://www.intel.com/content/www/us/en/developer/articles/tool/power-gadget.html>`__ . But has been discontinued. There is a discussion about it on `github issues #457 <https://github.com/mlco2/codecarbon/issues/457>`_.

- **Apple Silicon Chips (M1, M2)**

Expand Down Expand Up @@ -179,7 +179,7 @@ All CPUs listed in this directory will be tracked.
CPU hardware
------------

The CPU die is the processing unit itself. It's a piece of semiconductor that has been sculpted/etched/deposited by various manufacturing processes into a net of logic blocks that do stuff that makes computing possible1. The processor package is what you get when you buy a single processor. It contains one or more dies, plastic/ceramic housing for dies and gold-plated contacts that match those on your motherboard.
The CPU die is the processing unit itself. It's a piece of semiconductor that has been sculpted/etched/deposited by various manufacturing processes into a net of logic blocks that do stuff that makes computing possible. The processor package is what you get when you buy a single processor. It contains one or more dies, plastic/ceramic housing for dies and gold-plated contacts that match those on your motherboard.

In Linux kernel, energy_uj is a current energy counter in micro joules. It is used to measure CPU core's energy consumption.

Expand All @@ -191,22 +191,22 @@ For example, on a laptop with Intel(R) Core(TM) i7-7600U, Code Carbon will read

RAPL Metrics
------------
RAPL stand for Running Average Power Limit, it is a feature of processors (CPU) that provide the energy consumption of the processor.
RAPL stands for Running Average Power Limit, it is a feature of processors (CPU) that provide the energy consumption of the processor.

See https://blog.chih.me/read-cpu-power-with-RAPL.html for more information.

Despite the name Intel RAPL, it support AMD processors since kernel 5.8.

It is some files in /sys/class/powercap/intel-rapl/subsystem/ that give the energy consumption of the CPU, and sometime RAM.
There are folder for each `domain`, and in each folder there are a file `name` with the name of the domain and a `energy_uj` for the amount of energy in micro-joules.
It is some files in /sys/class/powercap/intel-rapl/subsystem/ that give the energy consumption of the CPU, and sometimes RAM.
There are folders for each `domain`, and in each folder there is a file `name` with the name of the domain and an `energy_uj` for the amount of energy in micro-joules.

The drawback of RAPL is that not every CPU use it the same way. We focus on the `package` domain, but some CPU have more domain like `core`, `uncore`, `dram`, `psys`, `gpu`, `psys` and `psys-io`.
The drawback of RAPL is that not every CPU uses it the same way. We focus on the `package` domain, but some CPUs have more domains like `core`, `uncore`, `dram`, `psys`, `gpu`, `psys` and `psys-io`.

For example :
- Intel put all the physical cores consumption in `core` and the `package` include `core`.
- For AMD, `core` have very low energy, so we don't know if it is included in the `package` or not.
- Intel puts all the physical cores consumption in `core` and the `package` includes `core`.
- For AMD, `core` has very low energy, so we don't know if it is included in the `package` or not.

Our friend from Scaphandre, a tool to monitor energy consumption, have a good article about RAPL https://hubblo-org.github.io/scaphandre-documentation/explanations/rapl-domains.html and also a discussion with good references: https://github.com/hubblo-org/scaphandre/issues/116#issuecomment-854453231 and point out that this topic is not well documented.
Our friends from Scaphandre, a tool to monitor energy consumption, have a good article about RAPL https://hubblo-org.github.io/scaphandre-documentation/explanations/rapl-domains.html and also a discussion with good references: https://github.com/hubblo-org/scaphandre/issues/116#issuecomment-854453231 and point out that this topic is not well documented.



Expand All @@ -223,8 +223,8 @@ CodeCarbon : 137 W

Power plug measure when loaded (100% CPU): 256 W - 125W in idle = 131 W
CorWatt PkgWatt
133.13 169.82
7.54 169.82
133.13 169.82
7.54 169.82
CodeCarbon : 330 W
package-0-die-0 : 166 W
package-0-die-1 : 166 W
Expand All @@ -237,10 +237,11 @@ CPU metrics priority

CodeCarbon will first try to read the energy consumption of the CPU from low level interface like RAPL or ``powermetrics``.
If none of the tracking tools are available, CodeCarbon will be switched to a fallback mode:
- It will first detect which CPU hardware is currently in use, and then map it to a data source listing 2000+ Intel and AMD CPUs and their corresponding thermal design powers (TDPs).
- If the CPU is not found in the data source, a global constant will be applied.
- If ``psutil`` is available, CodeCarbon will try to estimate the energy consumption from the TDP and the CPU load.
- CodeCarbon assumes that 50% of the TDP will be the average power consumption to make this approximation.

- It will first detect which CPU hardware is currently in use, and then map it to a data source listing 2000+ Intel and AMD CPUs and their corresponding thermal design powers (TDPs).
- If the CPU is not found in the data source, a global constant will be applied.
- If ``psutil`` is available, CodeCarbon will try to estimate the energy consumption from the TDP and the CPU load.
- CodeCarbon assumes that 50% of the TDP will be the average power consumption to make this approximation.

Here is a drawing of the fallback mode:

Expand Down Expand Up @@ -315,13 +316,16 @@ The equivalent emissions are calculated using this formula:

.. math::
\text{Equivalent Emissions} = \frac{\text{Total Emissions (kgCO₂)}}{\text{Emission Factor (kgCO₂/unit)}}

For example:

- **Car Usage**: *1 kWh* of energy consumption is approximately equivalent to:

- *8.33 kilometers driven by a car* (*1 ÷ 0.12*).
- *11.9 hours of TV usage* (*1 ÷ 0.084*), if emissions are considered.

- **US Citizen Emissions**:

- *1 kWh* of energy consumption can be compared to a fraction of the average weekly emissions of a US citizen:

.. math::
Expand Down
2 changes: 1 addition & 1 deletion docs/_sources/model_examples.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ NB : presented on a log scale

References
----------
`Measuring the Carbon intensity of AI in Cloud Instance <https://facctconference.org/static/pdfs_2022/facct22-145.pdf>`_
`Measuring the Carbon intensity of AI in Cloud Instance <https://dl.acm.org/doi/10.1145/3531146.3533234>`_

Another source comparing models carbon intensity:
`Energy and Policy Considerations for Deep Learning in NLP <https://arxiv.org/pdf/1906.02243.pdf>`_
16 changes: 8 additions & 8 deletions docs/_sources/parameters.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ Input Parameters
* - measure_power_secs
- Interval (in seconds) to measure hardware power usage, defaults to ``15``
* - tracking_mode
- | ``machine`` measure the power consumptions of the entire machine (defaults)
- | ``machine`` measure the power consumptions of the entire machine (default)
| ``process`` try and isolate the tracked processes in isolation
* - gpu_ids
- | Comma-separated list of GPU ids to track, defaults to ``None``
| These can either be integer indexes of GPUs on the system, or prefixes
| to match against GPU identifiers as described `here <https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#cuda-environment-variables>`_
* - log_level
- | Global codecarbon log level (by order of verbosity): "debug", "info" (defaults),
- | Global codecarbon log level (by order of verbosity): "debug", "info" (default),
| "warning", "error", or "critical"
* - co2_signal_api_token
- | API token for co2signal.com (requires sign-up for free beta)
Expand All @@ -51,10 +51,10 @@ Input Parameters
| on the same machine is allowed,
| defaults to ``True`` since v3. Used to be ``False`` in v2.

PUE is a multiplication factor provided by the user, so it is up to the user to get it from his cloud provider.
Old data-centers have a PUE up to 2.2, where new green one could be as low as 1.1.
PUE is a multiplication factor provided by the user, so it is up to the user to get it from their cloud provider.
Old data-centers have a PUE up to 2.2, where new greener ones could be as low as 1.1.

If you, or your provider, use ``CUDA_VISIBLE_DEVICES`` to set the GPU you want to use, CodeCarbon will automaticly set this value to ``gpu_ids``.
If you, or your provider, use ``CUDA_VISIBLE_DEVICES`` to set the GPUs you want to use, CodeCarbon will automaticly populate this value into ``gpu_ids``.
If you set ``gpu_ids`` manually, it will override the ``CUDA_VISIBLE_DEVICES`` for CodeCarbon measures.

Output parameters
Expand Down Expand Up @@ -124,9 +124,9 @@ Specific parameters for offline mode
* - country_iso_code
- | 3-letter ISO Code of the country
| where the experiment is being run.
| Available countries are listed in `global_energy_mix.json <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/global_energy_mix.json>`_
| Available countries are listed in `global_energy_mix.json <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/global_energy_mix.json>`__
* - region
- | Optional Name of the Province/State/City, where the infrastructure is hosted
- | Optional name of the Province/State/City, where the infrastructure is hosted
| Currently, supported only for US States and Canada
| for example - California or New York, from the `list <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/2016/usa_emissions.json>`_
* - cloud_provider
Expand Down Expand Up @@ -159,7 +159,7 @@ Decorator ``track_emissions`` in addition to standard arguments, requires the fo
| defaults to ``False``
* - country_iso_code
- | 3 letter ISO Code of the country where the experiment is being run.
| Available countries are listed in `global_energy_mix.json <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/2016/global_energy_mix.json>`_
| Available countries are listed in `global_energy_mix.json <https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/2016/global_energy_mix.json>`__
* - region
- | Optional Name of the Province/State/City, where the infrastructure is hosted
| Currently, supported only for US States
Expand Down
Loading
Loading