Add T1 (envelope-sized) and T2 (workload-aware) tuning tiers#22
Closed
ribalba wants to merge 1 commit into
Closed
Conversation
To study energy vs. tuning effort, each scenario now comes in tiers that sit next to the default and share its flow, so only the engine configuration varies between them: - T1 (<db>.t1.yml): each vendor's own rules-of-thumb sized to the fixed 4-CPU / 8-GB container. Durability is left at default so the T0->T1 delta is pure resource sizing. - T2 (<db>.t2.yml): workload-aware indexes, partitioning and parallelism plus curated config, with a T2+ columnar sub-tier (<db>.t2col.yml) for SQL Server columnstore and Db2 BLU on TPC-H. TUNING.md documents the ladder, how each number is derived per engine, its provenance, and the threats to validity (Oracle Free / Db2 Community caps, OOM headroom, verify-on-first-run). Tooling: - run_on_cluster.py gains -t/--tier to submit a single tier. - check_repo.py compares fairness knobs within a tier rather than across all driver scripts, and adds check 4: the T1 tuning payload must be byte-identical across benchmarks per engine (comments included, since T1 is workload-agnostic, so any per-benchmark difference is drift). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Member
Author
|
More work will be done here. Closing PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split of #20 — part 2 of 2. Adds the tuning tiers on top of the default-configuration scenarios.
The tiers
Tier files sit next to the default and share its flow, so only the engine configuration varies between them and
run_on_cluster.pydiscovers them automatically.<db>.yml<db>.t1.yml<db>.t2.yml<db>.t2col.ymlTUNING.mddocuments the ladder, how each number is derived per engine, its provenance, and the threats to validity (Oracle Free / Db2 Community edition caps, OOM headroom, Oracle/Db2 verify-on-first-run).Tooling changes
run_on_cluster.pygains-t/--tierto submit a single tier (-t 0for every default,-t 1for every T1, …).check_repo.pycompares fairness knobs within a tier rather than across all driver scripts, since tier variants deliberately change tuning knobs. A sub-tier inherits knobs it doesn't restate from the engine's base tier, so the comparison is on the effective config.check_repo.pyadds check 4: per engine, the T1 tuning payload must be byte-identical across benchmarks, comments included. T1 is hardware-sized and workload-agnostic, so a per-benchmark difference is drift — and a missing comment loses the provenance a reviewer needs to reproduce the number.check_repo.pypasses on this branch with 2 expected warnings.🤖 Generated with Claude Code