Skip to content

Add permutedims! benchmark suite#71

Open
lkdvos wants to merge 4 commits into
mainfrom
benchmark-suite
Open

Add permutedims! benchmark suite#71
lkdvos wants to merge 4 commits into
mainfrom
benchmark-suite

Conversation

@lkdvos

@lkdvos lkdvos commented Jul 9, 2026

Copy link
Copy Markdown
Member

Adds a BenchmarkTools.jl suite at benchmark/benchmarks.jl (defining SUITE) for tracking permutedims! performance across revisions with AirSpeedVelocity.jl.

Each case fixes a shape (dimension ratios) and a permutation and is swept over a list of total lengths, so its time can be divided by that of a same-length memory copy (the auto-generated copy group) for a machine-independent efficiency (≥ 1, approaching 1 when bandwidth-bound). Cases are defined in cases.toml in three groups, run for Float64/ComplexF64 and single/multi-threaded:

  • balanced — equal dimensions across ranks 2–6
  • skewed — one fat leading or trailing axis (the HPTT/TTC regime, as clean ratios)
  • strided — genuinely non-contiguous strided views (non-unit leading and second-leading strides on input/output), which exercise the paths a dense array never hits

Configurable through CLI options (ArgParse.jl) or a different --cases TOML file; see benchmark/README.md for usage.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

lkdvos and others added 2 commits July 9, 2026 21:42
BenchmarkTools suite covering the HPTT/TTC reference cases, small sizes
(including the strided-rs cases), and non-SIMD-aligned sizes, over
element types and thread counts, configurable via ArgParse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cases (and per-group sampling parameters) now live in
benchmark/cases.toml; an alternative file can be passed with --cases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lkdvos lkdvos force-pushed the benchmark-suite branch from 214628f to dde5c61 Compare July 10, 2026 01:42
…trided views

Cases now fix a shape (dimension ratios) and permutation and sweep a list
of total lengths, so each can be normalized against a same-length memory
copy (auto-generated `copy` group) for a machine-independent efficiency.
Adds a `strided` group whose inputs/outputs are genuinely non-contiguous
strided views (non-unit leading and second-leading strides), replacing the
explicit-size hptt/small/unaligned groups.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

Your PR no longer requires formatting changes. Thank you for your contribution!

@lkdvos lkdvos requested a review from Jutho July 10, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant