Skip to content

Commit b4ffe30

Browse files
jmfernandezjmfernandez
committed
Finally, added support for AMD CPUs TDP identification.
Also, publicly documented the different variants of the `tdp-finder.py`. Last, but not the least important, added crawler to fetch CPUBenchmark database, as well ad documented its usage.
1 parent 5bca06f commit b4ffe30

6 files changed

Lines changed: 3944 additions & 92 deletions

File tree

‎README.md‎

Lines changed: 91 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -115,39 +115,115 @@ interesting metrics.
115115
python plotGraph.py sample-series/Wetlab2Variations_metrics/2025_05_20-02_19-14001/ dest_directory
116116
```
117117

118-
## Digestion
119-
120-
The program `tdp-finder.py` helps to obtain the TDP of an Intel processor, using the gathered metadata stored at `cpu_details.json` within the series directory.
118+
## TDP (thermal design power) of the used processor
121119

122-
Repository https://github.com/felixsteinke/cpu-spec-dataset contains at
123-
[dataset](https://github.com/felixsteinke/cpu-spec-dataset/tree/main/dataset) subdirectory several tables in CSV format with
124-
this and other details for many Intel, AMD and Ampere processors. The key column here is `ProcessorNumber`.
120+
Next accessory programs help to obtain the TDP of either an Intel or AMD processor, used later to estimate the energy consumption of the processes:
121+
* `tdp-finder.py` uses the gathered metadata stored at `cpu_details.json` within a series directory obtained by either `process-metrics-collector.py` or `execution-metrics-collector.py`.
122+
* `cpuinfo-tdp-finder.py` uses the processor model strings from `/proc/cpuinfo` (or a copy of it).
123+
* `modelname-tdp-finder.py` uses the processor model string provided through command line.
125124

126-
Forked repository https://github.com/JosuaCarl/cpu-spec-dataset contains at
127-
[dataset](https://github.com/JosuaCarl/cpu-spec-dataset/tree/main/dataset) subdirectory tables in CSV similar to the ones from original repo, but with different column names. The key column here to match the processor is `Processor Number`.
125+
Processor TDP specifications are partially available from several sources around internet.
126+
* Repository https://github.com/felixsteinke/cpu-spec-dataset contains at
127+
[dataset](https://github.com/felixsteinke/cpu-spec-dataset/tree/main/dataset) subdirectory several tables in CSV format provide details
128+
for many Intel, AMD and Ampere processors. The key column here is `ProcessorNumber` on Intel CSV file, 'Name' on AMD one, etc...
129+
* Forked repository https://github.com/JosuaCarl/cpu-spec-dataset contains at
130+
[dataset](https://github.com/JosuaCarl/cpu-spec-dataset/tree/main/dataset) subdirectory several tables in CSV similar to the ones from original repo, but with different column names. The key column here to match Intel processors is `Processor Number`, for instance.
131+
* As TDP specifications for many AMD server models are missing from the previous sources, the page https://www.cpubenchmark.net/CPU_mega_page.html provides information for many different models. The downside is that we have detected that some TDP values related to Intel laptop processors might be inaccurate.
128132

129-
Usage would be something like next:
133+
You can fetch either of the two first datasets just with next commands:
130134

131135
```bash
136+
# Recommended
137+
git clone https://github.com/JosuaCarl/cpu-spec-dataset cpu-spec-dataset_Josua
138+
# or
132139
git clone https://github.com/felixsteinke/cpu-spec-dataset
133-
python tdp-finder.py sample-series/Wetlab2Variations_metrics/2025_05_20-02_19-14001/ cpu-spec-dataset/dataset/intel-cpus.csv
140+
```
141+
142+
For the third source, there is a scraping program within this repository, which writes a trimmed version of the huge table from CPUBenchmark into a CSV file.
143+
144+
```bash
145+
python -m treecript.tdp_sources cpumark_table.csv
146+
```
147+
148+
Once these datasets are locally available, usage of `tdp-finder.py` would be something like next, using an already generated (or even ongoing) metrics directory:
149+
150+
```bash
151+
python tdp-finder.py sample-series/Wetlab2Variations_metrics/2025_05_20-02_19-14001/ cpu-spec-dataset_Josua/dataset/*.csv cpumark_table.csv
134152
```
135153

136154
```
137-
TDP (ConfigTDPMax) => 28.0 W
155+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/amd-cpus.csv
156+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/ampere-cpus.csv
157+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/benchmark-cpus.csv
158+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/cpuworld-cpus.csv
159+
Model [11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz] => TDP [Configurable TDP-up] => 28.0 W => File cpu-spec-dataset_Josua/dataset/intel-cpus.csv
138160
```
139161

140-
or
162+
Usage of `cpuinfo-tdp-finder.py` (which does not require to have a gathered metrics directory) would be something like next:
141163

142164
```bash
143-
git clone https://github.com/JosuaCarl/cpu-spec-dataset cpu-spec-dataset_Josua
144-
python tdp-finder.py sample-series/Wetlab2Variations_metrics/2025_05_20-02_19-14001/ cpu-spec-dataset_Josua/dataset/intel-cpus.csv
165+
python cpuinfo-tdp-finder.py /proc/cpuinfo cpu-spec-dataset_Josua/dataset/*.csv cpumark_table.csv
166+
```
167+
168+
```
169+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/amd-cpus.csv
170+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/ampere-cpus.csv
171+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/benchmark-cpus.csv
172+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/cpuworld-cpus.csv
173+
Model [11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz] => TDP [Configurable TDP-up] => 28.0 W => File cpu-spec-dataset_Josua/dataset/intel-cpus.csv
174+
```
175+
176+
Example with already copied contents from `/proc/cpuinfo`:
177+
178+
```bash
179+
python cpuinfo-tdp-finder.py sample_cpuinfo/cpuinfo-amd.txt cpu-spec-dataset_Josua/dataset/*.csv cpumark_table.csv
180+
```
181+
182+
```
183+
Unable to match a valid processor row for AMD EPYC 7742 64-Core Processor in file cpu-spec-dataset_Josua/dataset/amd-cpus.csv
184+
Unable to match a valid processor row for AMD EPYC 7742 64-Core Processor in file cpu-spec-dataset_Josua/dataset/ampere-cpus.csv
185+
Model [AMD EPYC 7742 64-Core Processor] => TDP [TDP] => 225.0 W => File cpu-spec-dataset_Josua/dataset/benchmark-cpus.csv
186+
```
187+
188+
If we have the processor model string, usage of `modelname-tdp-finder.py` would be something like next:
189+
190+
```bash
191+
python modelname-tdp-finder.py "11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz" cpu-spec-dataset_Josua/dataset/*.csv cpumark_table.csv
192+
```
193+
194+
```
195+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/amd-cpus.csv
196+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/ampere-cpus.csv
197+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/benchmark-cpus.csv
198+
Unable to match a valid processor row for 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz in file cpu-spec-dataset_Josua/dataset/cpuworld-cpus.csv
199+
Model [11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz] => TDP [Configurable TDP-up] => 28.0 W => File cpu-spec-dataset_Josua/dataset/intel-cpus.csv
200+
```
201+
202+
```bash
203+
python modelname-tdp-finder.py "AMD EPYC 7742 64-Core Processor" cpu-spec-dataset_Josua/dataset/*.csv cpumark_table.csv
204+
```
205+
206+
```
207+
Unable to match a valid processor row for AMD EPYC 7742 64-Core Processor in file cpu-spec-dataset_Josua/dataset/amd-cpus.csv
208+
Unable to match a valid processor row for AMD EPYC 7742 64-Core Processor in file cpu-spec-dataset_Josua/dataset/ampere-cpus.csv
209+
Model [AMD EPYC 7742 64-Core Processor] => TDP [TDP] => 225.0 W => File cpu-spec-dataset_Josua/dataset/benchmark-cpus.csv
210+
```
211+
212+
```bash
213+
python modelname-tdp-finder.py "AMD EPYC 9V74 80-Core Processor" cpu-spec-dataset_Josua/dataset/*.csv cpumark_table.csv
145214
```
146215

147216
```
148-
TDP (Configurable TDP-up) => 28.0 W
217+
Unable to match a valid processor row for AMD EPYC 9V74 80-Core Processor in file cpu-spec-dataset_Josua/dataset/amd-cpus.csv
218+
Unable to match a valid processor row for AMD EPYC 9V74 80-Core Processor in file cpu-spec-dataset_Josua/dataset/ampere-cpus.csv
219+
Unable to match a valid processor row for AMD EPYC 9V74 80-Core Processor in file cpu-spec-dataset_Josua/dataset/benchmark-cpus.csv
220+
Unable to match a valid processor row for AMD EPYC 9V74 80-Core Processor in file cpu-spec-dataset_Josua/dataset/cpuworld-cpus.csv
221+
Unable to match a valid processor row for AMD EPYC 9V74 80-Core Processor in file cpu-spec-dataset_Josua/dataset/intel-cpus.csv
222+
Model [AMD EPYC 9V74 80-Core Processor] => TDP [TDP] => 400.0 W => File cpumark_table.csv
149223
```
150224

225+
## Digestion
226+
151227
The program `metrics-aggregator.py` is an initial proof of concept to digest the gathered process tree time series. As it tries
152228
computing the Wh of each part being executed, it needs the TDP (Thermal Design Power) or similar from the CPU.
153229

0 commit comments

Comments
 (0)