Unofficial Submission: R-GAT IGBH#4
Open
gbeezybez wants to merge 26 commits into
Open
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
There was a problem hiding this comment.
Pull request overview
This PR adds an unofficial MLPerf Inference v6.0 Open Division archive for the R-GAT benchmark on the IGBH dataset, including a system description, measurements config, and captured result artifacts/logs under a submission/ folder for public access.
Changes:
- Added a DynamoNPU system description JSON.
- Added Offline accuracy logs/artifacts and performance-side metrics JSONs for R-GAT.
- Added measurements configuration files plus a top-level pointer file for the submission package.
Reviewed changes
Copilot reviewed 15 out of 20 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| submission/systems/DynamoNPU.json | Adds system metadata for the DynamoNPU submission. |
| submission/results/DynamoNPU/R-GAT/offline/performance/run_1/README.md | Documents how performance numbers were produced for run_1. |
| submission/results/DynamoNPU/R-GAT/offline/performance/run_1/igbh_workload_metrics.json | Adds workload/perf/energy metrics for run_1. |
| submission/results/DynamoNPU/R-GAT/offline/performance/run_1/igbh_dynamo_benchmark_results.json | Adds benchmark result summary metrics for run_1. |
| submission/results/DynamoNPU/R-GAT/offline/accuracy/mlperf_log_summary.txt | Adds LoadGen accuracy-run summary output. |
| submission/results/DynamoNPU/R-GAT/offline/accuracy/mlperf_log_detail.txt | Adds LoadGen accuracy-run detailed log output. |
| submission/results/DynamoNPU/R-GAT/offline/accuracy/mlperf_log_accuracy_igbh.json | Adds accuracy dump payload for the IGBH run. |
| submission/results/DynamoNPU/R-GAT/offline/accuracy/accuracy.txt | Adds computed accuracy summary. |
| submission/mlperf_log_summary.txt | Adds a top-level LoadGen summary log. |
| submission/mlperf_log_detail.txt | Adds a top-level LoadGen detailed log. |
| submission/measurements/DynamoNPU/R-GAT/offline/user.conf | Adds user LoadGen overrides (target_qps, accuracy sampling). |
| submission/measurements/DynamoNPU/R-GAT/offline/README.md | Adds run instructions and dataset notes for the measurement. |
| submission/measurements/DynamoNPU/R-GAT/offline/mlperf.conf | Adds the mlperf.conf reference for the measurement. |
| submission/igbh_workload_metrics.json | Adds a top-level workload metrics summary JSON. |
| mlperf-rgat-igbh-submission | Adds a pointer file intended to reference the submission directory. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1 @@ | |||
| loadgen/mlperf.conf No newline at end of file | |||
Comment on lines
+1
to
+4
| # R-GAT on Dynamo NPU - Offline Accuracy Run | ||
|
|
||
| ## Setup | ||
| - Dataset: IGBH-tiny (256 validation samples, nodes 60000-60255) |
Comment on lines
+1
to
+8
| # Dynamo NPU Native Performance Report | ||
|
|
||
| Performance measured via native NPU signoff simulation. | ||
| LoadGen was not used for performance measurement due to the sub-nanosecond | ||
| operation timescales of the neuromorphic architecture, which are outside | ||
| the measurement resolution of software-based harnesses. | ||
|
|
||
| Key metrics (signoff mode, 256 samples): |
Comment on lines
+1
to
+2
| # R-GAT on Dynamo NPU - Offline Accuracy Run | ||
|
|
Comment on lines
+1
to
+6
| { | ||
| "samples": 10000, | ||
| "total_energy_j": 2.680793159299025e-10, | ||
| "avg_latency_ps": 41.37685121124734, | ||
| "tops_w": 305581.2035174712 | ||
| } No newline at end of file |
Comment on lines
+12
to
+13
| :::MLLOG {"key": "qsl_reported_total_count", "value": 10000, "time_ms": 0.163250, "namespace": "mlperf::logging", "event_type": "POINT_IN_TIME", "metadata": {"is_error": false, "is_warning": false, "file": "loadgen.cc", "line_no": 1215, "pid": 132369, "tid": 132369}} | ||
| :::MLLOG {"key": "qsl_reported_performance_count", "value": 10000, "time_ms": 0.163250, "namespace": "mlperf::logging", "event_type": "POINT_IN_TIME", "metadata": {"is_error": false, "is_warning": false, "file": "loadgen.cc", "line_no": 1216, "pid": 132369, "tid": 132369}} |
| @@ -0,0 +1 @@ | |||
| submissions | |||
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…_accuracy_igbh.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is an unofficial submission for the MLPerf Inference v6.0 Open Division (submitted post-deadline).
Benchmark: R-GAT
Dataset: IGBH
Division: Open
System: Virtual SUT
The submission includes the full results directory, system JSON, and the encrypted .tar.gz package.
I am forking and opening this PR to archive the results and make them publicly accessible for research purposes.