Feature/ncu metric extraction (#17)

* fix: correct pytest -k examples in documentation

* feature: added ncu mtric extraction script, and updated DEVELOPMENT to reflect new changes

* style: fix ruff formatting in ncu_extract.py

Author: debashishc

.github/workflows/ruff-format.yml

@@ -5,16 +5,23 @@ on:
   push:
   pull_request:
 
+permissions:
+  contents: read
+
+concurrency:
+  group: ruff-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   format:
     runs-on: ubuntu-latest
+    timeout-minutes: 10
     steps:
       - name: Checkout code
-        uses: actions/checkout@v6
+        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332
 
       - name: Ruff check
-        uses: astral-sh/ruff-action@v3
+        uses: astral-sh/ruff-action@57714a7c8a2e59f32539362ba31877a1957dded1
       - name: Ruff format
         run: ruff format --check --diff
 
-

DEVELOPMENT.md

@@ -104,8 +104,18 @@ Key constraints to keep in mind:
 ## Testing and correctness
 
 ```bash
+# Run all tests
 uv run pytest -q
-uv run pytest tests/test_copy_transpose.py::test_copy_transpose_correctness -k "float16"
+
+# Run specific test file
+uv run pytest tests/test_copy_transpose.py
+
+# Run specific test function
+uv run pytest tests/test_copy_transpose.py::test_copy_transpose_correctness
+
+# Filter tests by name pattern with -k (matches test function names)
+uv run pytest tests/test_softmax_online.py -k correctness
+uv run pytest -k "not correctness and not compile"
 ```
 
 Correctness is the primary gate. Use explicit tolerances and document any
@@ -131,6 +141,48 @@ Add new benchmark cases to `bench/suites.yaml` and keep outputs reproducible.
 Nsight Compute with `--set full` can be slow; use lighter sets if needed
 (`--set=launchstats`, `--section=SpeedOfLight`, etc.).
 
+### Metric extraction
+
+Extract curated metrics from NCU reports using the `--extract` flag or standalone:
+
+```bash
+# Automatic extraction after profiling
+./scripts/profile.sh ncu --extract -- uv run python bench/benchmark_copy_transpose.py
+
+# Standalone extraction from existing report
+./scripts/ncu_extract.py profiles/ncu_*.ncu-rep
+
+# JSON output
+./scripts/ncu_extract.py profiles/ncu_*.ncu-rep --json
+
+# Filter by kernel name
+./scripts/ncu_extract.py profiles/ncu_*.ncu-rep --kernel "CopyTranspose"
+```
+
+Output includes three metric categories:
+- **GPU Throughput**: Memory and Compute (SM) utilization percentages
+- **Pipe Utilization**: Tensor Core, FMA, ALU activity
+- **Warp Stalls**: Long scoreboard, barrier, memory throttle stalls
+
+### Profiling tips
+
+**For NCU profiling**, use minimal iterations since you're analyzing kernel behavior, not
+timing variance. The L2 cache flush adds extra kernel launches per iteration.
+
+```bash
+# Fast profiling (recommended)
+./scripts/profile.sh ncu --extract -- uv run python bench/benchmark_copy_transpose.py --warmup 2 --iterations 1
+
+# Profile only your kernel (skip reference kernels)
+./scripts/profile.sh ncu --kernel-name "copy_transpose" --extract -- uv run python bench/benchmark_copy_transpose.py
+```
+
+| Goal | Warmup | Iterations | Notes |
+|------|--------|------------|-------|
+| Timing benchmarks | 10 | 100 | Default, good for stable p50/p90 |
+| Quick profiling | 2 | 1-3 | Sufficient for kernel analysis |
+| Deep profiling (`--set full`) | 1 | 1 | Full metrics are slow |
+
 ## Linting and formatting
 
 ```bash

README.md

@@ -90,7 +90,7 @@ Profile a kernel with helper script:
 
 ```bash
 ./scripts/profile.sh ncu -- uv run python bench/benchmark_copy_transpose.py
-./scripts/profile.sh nsys -- uv run pytest tests/test_copy_transpose.py -k "tile_size=16"
+./scripts/profile.sh nsys -- uv run pytest tests/test_copy_transpose.py
 ```
 
 Or use profiling tools directly:
@@ -140,7 +140,7 @@ uv run python -m forge_cute_py.env_check  # Verify CUDA/PyTorch setup
 ```bash
 uv run pytest -q                                      # Run all tests
 uv run pytest tests/test_copy_transpose.py            # Run specific test file
-uv run pytest -k "float16 and tile_size=16"           # Run filtered tests
+uv run pytest -k correctness                          # Filter by test name pattern
 uv run pre-commit run --all-files                     # Run linting/formatting
 ```