Skip to content

Benchmarks timing is not independent from previous benchmarks #197

@fraya

Description

@fraya

Executing a test suite that has several benchmarks, the time of the benchmarks is dependent on the time of the previous benchmarks. For instance we can see that the time of benchmark-concatenate-for is 32 seconds:

$ _build/bin/issue-652-test-suite
Suite issue-652-test-suite:
  Benchmark benchmark-deque: PASSED in 14.929141s and 14MiB
  Benchmark benchmark-list: PASSED in 19.617091s and 15MiB
  Benchmark benchmark-current: PASSED in 24.570700s and 11MiB
  Benchmark benchmark-join-map: PASSED in 28.224557s and 12MiB
  Benchmark benchmark-concatenate-for: PASSED in 32.803978s and 13MiB
Ran 5 assertions
Ran 5 benchmarks
PASSED in 117.145467 seconds

But repeating the test, only for this benchmark:

$ _build/bin/issue-652-test-suite --run benchmark-concatenate-for
Suite issue-652-test-suite:
  Benchmark benchmark-concatenate-for: PASSED in 15.142817s and 13MiB
Ran 1 assertion
Ran 5 benchmarks: 4 skipped
PASSED in 15.142817 seconds

we can see that the time is the half.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions