Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
- Documentation guidance for reproducible Criterion benchmark baseline capture,
local comparisons, and non-flaky performance regression reporting.

## [0.3.0] - 2026-06-06

### Added
Expand Down
33 changes: 33 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,39 @@ cargo doc --all-features --no-deps
cargo bench
```

### Benchmark Baseline Reporting

Benchmarks use Criterion and are intended to inform reviews without making CI
flaky. Do not add mandatory performance pass/fail gates unless the project later
adopts a dedicated, stable benchmarking runner.

For performance-sensitive changes, capture a reproducible local baseline before
editing and compare your branch against it:

```bash
# From the baseline branch or commit you want to compare against
cargo bench --all-features -- --save-baseline before-change

# From your feature branch
cargo bench --all-features -- --baseline before-change
```

Include the following details in the PR description when benchmark results are
relevant:

- Baseline and candidate git SHAs (`git rev-parse --short HEAD` for each)
- `rustc --version` and `cargo --version`
- CPU model/core count, operating system, and any relevant power or thermal
constraints
- Exact benchmark command, including feature flags and any Criterion filters
- A short summary of the Criterion comparison, plus the HTML report location
(`target/criterion/report/index.html`, or the equivalent path under
`$CARGO_TARGET_DIR`) or attached artifacts when helpful

Prefer local Criterion comparisons and posted artifacts over CI performance
thresholds. CI should continue checking that benchmarks compile, for example with
`cargo bench --no-run --all-features`, without failing on noisy timing deltas.

## Contribution Guidelines

### Code Style
Expand Down
28 changes: 27 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -333,12 +333,38 @@ println!("Slope estimate: {:.3} ± {:.3}",

## Performance

Run benchmarks to see performance characteristics:
Run the Criterion benchmarks to see performance characteristics:

```bash
cargo bench
```

When sharing benchmark results, include enough context for others to reproduce
and interpret them:

- Git revision, for example `git rev-parse --short HEAD`
- Rust toolchain versions from `rustc --version` and `cargo --version`
- CPU, operating system, and any notable power/thermal settings
- Exact command used, including feature flags or Criterion filters
- Criterion output location, such as `target/criterion/report/index.html` (or
the equivalent path under `$CARGO_TARGET_DIR`), when the HTML report is useful
for review

For local regression checks, capture a named Criterion baseline and compare the
current work against it:

```bash
# From the baseline branch or commit you want to compare against
cargo bench --all-features -- --save-baseline before-change

# From your feature branch
cargo bench --all-features -- --baseline before-change
```

Treat benchmark numbers as local evidence rather than CI pass/fail gates. Runtime
noise varies by machine, so prefer posting the captured context and Criterion
comparison in reviews over adding flaky performance thresholds to automation.

The library is optimized for:
- Efficient matrix operations using `nalgebra`
- Minimal memory allocations during sampling
Expand Down
Loading