You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|`avg_blockbw`|Block GB/s| Mean local block-device throughput | High values indicate local scratch/checkpoint pressure; unexpected nonzero can reveal spill to local disk. |
114
-
|`avg_cpuusage`| CPU cores | Mean CPU cores used (from user/system/nice) | Low vs allocated cores suggests under-subscription, waiting, serialization, or I/O/network stalls. |
115
-
|`avg_sharedfs_iops`|FS IOPS| Mean shared filesystem metadata/op rate | High with low MB/s points to small-file metadata bottlenecks. |
116
-
|`avg_sharedfs_bw`|FS MB/s| Mean shared filesystem bandwidth | Sustained high values indicate file I/O-heavy phases; correlate with runtime spikes/checkpoint windows. |
117
-
|`avg_ibbw`|IB MB/s| Mean InfiniBand/fabric byte throughput | High values with modest FLOP rate imply communication-heavy behavior<sup>[15](#ref-15)</sup>. |
118
-
|`avg_fabric_mb_per_gflops`|MB/GFLOP| Fabric MB per GFLOP | Communication intensity relative to compute; rising with scale often means weaker scaling efficiency. |
119
-
|`avg_tensor_active`|Tensor %| Mean tensor pipeline activity | Low on expected tensor workloads suggests kernels not reaching tensor paths. |
120
-
|`avg_fp16_active`| FP16 %| Mean GPU FP16 pipeline activity | Confirms whether mixed-precision execution is using FP16-heavy kernels as expected. |
121
-
|`avg_fp32_active`| FP32 %| Mean GPU FP32 pipeline activity | Tracks single-precision dominant GPU phases and precision-policy drift across runs. |
122
-
|`avg_fp64_active`| FP64 %| Mean GPU FP64 pipeline activity | Surfaces double-precision-heavy GPU work that may reduce peak throughput. |
123
-
|`avg_gpu_mem_bw_gbps`| GPU HBM| Mean GPU memory-bandwidth rate | High with moderate utilization can indicate memory-bound kernels. |
124
-
|`avg_fabric_mb_per_avg_tensor`|MB/tensor | Fabric MB per average tensor activity | Communication intensity normalized by tensor activity for GPU+MPI workloads. |
125
-
|`avg_flops`|GFLOP/s| Mean achieved FLOP rate | Baseline compute throughput for CPU-side arithmetic. |
126
-
|`avg_mbw`| DRAM GB/s| Mean DRAM bandwidth | High with low FLOPs suggests memory-bound CPU phases. |
127
-
|`avg_freq`| CPU GHz| Mean CPU frequency | Drops may indicate power/thermal policy or throttling. |
128
-
|`avg_ethbw`|Eth MB/s| Mean Ethernet bandwidth | Useful for TCP/object-store workflows that bypass IB paths<sup>[16](#ref-16)</sup>. |
129
-
|`detail_gpu_active`|GPU active| Number of active GPUs | Lower than allocated GPUs usually means mapping/launcher inefficiency. |
130
-
|`detail_gpu_util_max`| GPU max %| Max GPU utilization observed | Peak headroom check; high max with low mean often indicates bursty kernels. |
131
-
|`detail_gpu_util_mean`| GPU mean %| Mean GPU utilization observed | Primary “are GPUs doing work?” scalar for the job. |
132
-
|`detail_gpu_count`|GPU count| Total GPUs allocated | Sanity check against scheduler request and host topology. |
133
-
|`detail_fsio_llite_read_mb`|FSIO llite read | Total Lustre llite read MB | Aggregate client-side read volume for Lustre path. |
134
-
|`detail_fsio_llite_write_mb`|FSIO llite write | Total Lustre llite write MB | Aggregate client-side write volume for Lustre path. |
135
-
|`detail_fsio_llite_peak_mb_s`|FSIO llite peak MB/s| Peak aggregate Lustre client MB/s | Short burst combined read+write throughput versus job-total MB. |
|`vecpercent_64b`|Vec% DP| Percent of double-precision FLOPs done via vector widths > scalar | Low values on DP-heavy code suggest SIMD/vectorization opportunity<sup>[5](#ref-5)</sup>. |
164
-
|`avg_vector_width_64b`|VW DP| Average effective DP vector width | Closer to scalar indicates weak SIMD utilization in DP paths. |
165
-
|`vecpercent_32b`|Vec% SP| Percent of single-precision FLOPs done via vector widths > scalar | Low values on SP-heavy code suggest vectorization opportunity<sup>[5](#ref-5)</sup>. |
166
-
|`avg_vector_width_32b`|VW SP| Average effective SP vector width | Low average width indicates scalar/short-vector dominated SP execution. |
113
+
|`avg_blockbw`|Average local block-device throughput| Mean local block-device throughput | High values indicate local scratch/checkpoint pressure; unexpected nonzero can reveal spill to local disk. |
114
+
|`avg_cpuusage`|Average CPU cores in use| Mean CPU cores used (from user/system/nice) | Low vs allocated cores suggests under-subscription, waiting, serialization, or I/O/network stalls. |
115
+
|`avg_sharedfs_iops`|Average shared filesystem operation rate| Mean shared filesystem metadata/op rate | High with low MB/s points to small-file metadata bottlenecks. |
116
+
|`avg_sharedfs_bw`|Average shared filesystem read+write bandwidth| Mean shared filesystem bandwidth | Sustained high values indicate file I/O-heavy phases; correlate with runtime spikes/checkpoint windows. |
117
+
|`avg_ibbw`|Average high-speed fabric bandwidth| Mean InfiniBand/fabric byte throughput | High values with modest FLOP rate imply communication-heavy behavior<sup>[15](#ref-15)</sup>. |
118
+
|`avg_fabric_mb_per_gflops`|Fabric traffic per floating-point work| Fabric MB per GFLOP | Communication intensity relative to compute; rising with scale often means weaker scaling efficiency. |
119
+
|`avg_tensor_active`|Average GPU tensor-pipe activity| Mean tensor pipeline activity | Low on expected tensor workloads suggests kernels not reaching tensor paths. |
120
+
|`avg_fp16_active`|Average GPU FP16 pipeline activity| Mean GPU FP16 pipeline activity | Confirms whether mixed-precision execution is using FP16-heavy kernels as expected. |
121
+
|`avg_fp32_active`|Average GPU FP32 pipeline activity| Mean GPU FP32 pipeline activity | Tracks single-precision dominant GPU phases and precision-policy drift across runs. |
122
+
|`avg_fp64_active`|Average GPU FP64 pipeline activity| Mean GPU FP64 pipeline activity | Surfaces double-precision-heavy GPU work that may reduce peak throughput. |
123
+
|`avg_gpu_mem_bw_gbps`|Average GPU memory bandwidth| Mean GPU memory-bandwidth rate | High with moderate utilization can indicate memory-bound kernels. |
124
+
|`avg_fabric_mb_per_avg_tensor`|Fabric bandwidth per tensor activity| Fabric MB per average tensor activity | Communication intensity normalized by tensor activity for GPU+MPI workloads. |
125
+
|`avg_flops`|Average floating-point throughput| Mean achieved FLOP rate | Baseline compute throughput for CPU-side arithmetic. |
126
+
|`avg_mbw`|Average DRAM memory bandwidth| Mean DRAM bandwidth | High with low FLOPs suggests memory-bound CPU phases. |
127
+
|`avg_freq`|Average effective CPU frequency| Mean CPU frequency | Drops may indicate power/thermal policy or throttling. |
128
+
|`avg_ethbw`|Average Ethernet bandwidth| Mean Ethernet bandwidth | Useful for TCP/object-store workflows that bypass IB paths<sup>[16](#ref-16)</sup>. |
129
+
|`detail_gpu_active`|GPUs with non-zero utilization| Number of active GPUs | Lower than allocated GPUs usually means mapping/launcher inefficiency. |
130
+
|`detail_gpu_util_max`|Sum of per-GPU peak utilization| Max GPU utilization observed | Peak headroom check; high max with low mean often indicates bursty kernels. |
131
+
|`detail_gpu_util_mean`|Sum of per-GPU mean utilization| Mean GPU utilization observed | Primary “are GPUs doing work?” scalar for the job. |
132
+
|`detail_gpu_count`|Total GPUs on job| Total GPUs allocated | Sanity check against scheduler request and host topology. |
133
+
|`detail_fsio_llite_read_mb`|Total Lustre client read volume| Total Lustre llite read MB | Aggregate client-side read volume for Lustre path. |
134
+
|`detail_fsio_llite_write_mb`|Total Lustre client write volume| Total Lustre llite write MB | Aggregate client-side write volume for Lustre path. |
135
+
|`detail_fsio_llite_peak_mb_s`|Peak Lustre client read+write rate| Peak aggregate Lustre client MB/s | Short burst combined read+write throughput versus job-total MB. |
|`max_numa_remote_rate`|Peak non-local NUMA memory access rate| Peak NUMA remote-access rate | High values indicate locality/memory placement issues<sup>[4](#ref-4)</sup>. |
149
+
|`max_gpu_power`|Maximum GPU power draw| Peak GPU power draw | Detects power-cap proximity or thermal stress windows. |
150
+
|`max_node_power_est_w`|Peak estimated node power| Peak estimated node power | Useful for peak power envelope checks and cooling stress. |
151
+
|`avg_node_power_est_w`|Mean estimated node power| Mean estimated node power | Energy-to-solution comparisons across runs/configurations. |
152
+
|`max_gpu_link_gbps`|Peak GPU PCIe and NVLink data rate| Peak GPU link bandwidth (PCIe/NVLink aggregate path) | Host-device/device-device transfer pressure indicator. |
|`mem_hwm`|Peak process resident memory (high water mark)| High-water memory estimate (MemUsed-Slab-FilePages) | Compare with node RAM for host OOM risk<sup>[14](#ref-14)</sup>. |
155
+
|`node_imbalance`| CPU utilization imbalance across nodes| Node-level CPU rate imbalance | High values indicate decomposition/rank imbalance. |
156
+
|`time_imbalance`|CPU rate imbalance over job timeline| Temporal CPU imbalance across job timeline | Flags long underutilized windows or phase imbalance over time. |
157
+
|`flops_node_imbalance`|Floating-point rate imbalance across nodes| Node-level FLOP rate imbalance | Compute work unevenly distributed across nodes. |
158
+
|`fabric_node_imbalance`|Fabric bandwidth imbalance across nodes| Node-level fabric traffic imbalance | Some ranks/nodes communicate disproportionately. |
159
+
|`dram_bw_node_imbalance`| DRAM bandwidth imbalance across nodes| Node-level DRAM bandwidth imbalance | Memory pressure concentrated on subset of nodes. |
|`gpu_util_node_imbalance`| GPU utilization imbalance across nodes| Node-level GPU utilization imbalance | Multi-node training/inference skew across nodes. |
162
+
|`tensor_node_imbalance`| Tensor-pipe activity imbalance across nodes| Node-level tensor-activity imbalance | Tensor kernels unevenly distributed across participating nodes. |
163
+
|`vecpercent_64b`|Share of double-precision FLOPs from vector instructions| Percent of double-precision FLOPs done via vector widths > scalar | Low values on DP-heavy code suggest SIMD/vectorization opportunity<sup>[5](#ref-5)</sup>. |
164
+
|`avg_vector_width_64b`|Effective vector width (double precision)| Average effective DP vector width | Closer to scalar indicates weak SIMD utilization in DP paths. |
165
+
|`vecpercent_32b`|Share of single-precision FLOPs from vector instructions| Percent of single-precision FLOPs done via vector widths > scalar | Low values on SP-heavy code suggest vectorization opportunity<sup>[5](#ref-5)</sup>. |
166
+
|`avg_vector_width_32b`|Effective vector width (single precision)| Average effective SP vector width | Low average width indicates scalar/short-vector dominated SP execution. |
167
167
168
168
---
169
169
@@ -300,5 +300,6 @@ Use these numbered references when you want background on terms used throughout
300
300
| 2026-05-07 | Added GPU precision activity catalog entries (`avg_fp16_active`, `avg_fp32_active`, `avg_fp64_active`) and documented that Multiprecision Mix pies render whatever precision widths are available per job/architecture. |
301
301
| 2026-06-04 | Documented unified InfiniBand collector typename `host_ib` (replaces separate `ib_ext` / switch types in schema examples). |
302
302
| 2026-06-05 | Job list filter summary, empty-result UX, expanded-search AND/end-time semantics, navbar Find Job, job-detail breadcrumbs, shareable `?tab=` analysis tabs, partial-load retry behavior, canonical vs legacy device type names in Device data. |
303
+
| 2026-06-05 | Job detail Metrics tab short labels expanded to describe what is measured; units appear only in the bracket suffix beside each row (no unit tokens in label text). |
0 commit comments