Skip to content

sql: use metamorphic defaults for optimizer testing knobs in logictest#170882

Open
lohitkolluri wants to merge 2 commits into
cockroachdb:masterfrom
lohitkolluri:opt-metamorphic-optimizer-settings
Open

sql: use metamorphic defaults for optimizer testing knobs in logictest#170882
lohitkolluri wants to merge 2 commits into
cockroachdb:masterfrom
lohitkolluri:opt-metamorphic-optimizer-settings

Conversation

@lohitkolluri
Copy link
Copy Markdown

Summary

  • Use metamorphic constants for logictest defaults of optimizer-cost-perturbation and disable-opt-rule-probability flags
  • Aligns with testing_optimizer_cost_perturbation and testing_optimizer_disable_rule_probability session settings used by costfuzz and unoptimized-query-oracle
  • Improves optimizer test coverage during metamorphic logictest runs

Fixes #93825

Test plan

  • ./dev build pkg/sql/logictest

Release note: None

@trunk-io
Copy link
Copy Markdown
Contributor

trunk-io Bot commented May 25, 2026

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@blathers-crl
Copy link
Copy Markdown

blathers-crl Bot commented May 25, 2026

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

I have added a few people who may be able to assist in reviewing:

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@blathers-crl blathers-crl Bot added O-community Originated from the community X-blathers-triaged blathers was able to find an owner labels May 25, 2026
@blathers-crl blathers-crl Bot requested review from rytaft and yuzefovich May 25, 2026 09:00
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@yuzefovich yuzefovich requested review from a team and michae2 and removed request for a team and yuzefovich May 25, 2026 18:07
@lohitkolluri lohitkolluri requested review from a team as code owners May 26, 2026 05:04
@blathers-crl
Copy link
Copy Markdown

blathers-crl Bot commented May 26, 2026

Thank you for updating your pull request.

Before a member of our team reviews your PR, I have some potential action items for you:

  • We notice you have more than one commit in your PR. We try break logical changes into separate commits, but commits such as "fix typo" or "address review commits" should be squashed into one commit and pushed with --force
  • Please ensure your git commit message contains a release note.
  • When CI has completed, please ensure no errors have appeared.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@lohitkolluri lohitkolluri force-pushed the opt-metamorphic-optimizer-settings branch from fe00cda to b1d03eb Compare May 26, 2026 05:08
@blathers-crl
Copy link
Copy Markdown

blathers-crl Bot commented May 26, 2026

Thank you for updating your pull request.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@blathers-crl
Copy link
Copy Markdown

blathers-crl Bot commented May 26, 2026

Thank you for updating your pull request.

Before a member of our team reviews your PR, I have some potential action items for you:

  • We notice you have more than one commit in your PR. We try break logical changes into separate commits, but commits such as "fix typo" or "address review commits" should be squashed into one commit and pushed with --force
  • When CI has completed, please ensure no errors have appeared.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@lohitkolluri lohitkolluri force-pushed the opt-metamorphic-optimizer-settings branch from dc266d7 to b2513c6 Compare May 26, 2026 17:29
@lohitkolluri
Copy link
Copy Markdown
Author

@rytaft @michae2 — when you have a moment, could you please take another look?

@rytaft
Copy link
Copy Markdown
Collaborator

rytaft commented Jun 1, 2026

@lohitkolluri thanks for the contribution! Have you tested with COCKROACH_LOGIC_TEST_OPTIMIZER_METAMORPHIC=true on any files? What happens if you set the default value to true?

It will be nice to have this option available, but unless the metamorphic constant is enabled, it doesn't really provide us any additional test coverage we didn't already have with the existing logictest flags.

@lohitkolluri
Copy link
Copy Markdown
Author

Good point — I've flipped the default to true in 48f5d0b.

logicTestOptimizerMetamorphicEnabled = envutil.EnvOrDefaultBool(
    "COCKROACH_LOGIC_TEST_OPTIMIZER_METAMORPHIC", true)

A few things I verified:

  1. The metamorphic constants only activate in metamorphic (race) buildsConstantWithTestChoice always returns 0 in normal builds, so the env var default doesn't affect non-race CI at all.

  2. Deterministic tests are already guarded — EXPLAIN tests (execbuilder) and SQLite logictests have DisableOptimizerPerturbations: true set, so they won't be perturbed regardless.

  3. ForceProductionValues overrides — Configs that set this already zero out the knobs, so they're unaffected too.

I ran the non-EXPLAIN logictest files locally with the env var set and they passed, so this should be safe to merge as-is. Let me know if you'd like any other adjustments.

Copy link
Copy Markdown
Collaborator

@rytaft rytaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Assuming CI passes, this :lgtm:

Thanks again for the contribution!

@rytaft reviewed 21 files and all commit messages, and made 1 comment.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on michae2).

@lohitkolluri
Copy link
Copy Markdown
Author

Thanks for the review, Rebecca! I appreciate you taking the time to look through the changes.

I'll keep an eye on CI and wait for the remaining review. Thanks again!

@lohitkolluri
Copy link
Copy Markdown
Author

@michae2 could you please trigger the CI workflows when you get a chance? Rebecca has approved the change, and I'd appreciate a final look from you when convenient. Thanks!

…pt-in

Add metamorphicDisableOptRuleProbability and
metamorphicOptimizerCostPerturbation constants that mirror the
testing_optimizer_cost_perturbation and
testing_optimizer_disable_rule_probability session settings.

These metamorphic values are applied when the env var
COCKROACH_LOGIC_TEST_OPTIMIZER_METAMORPHIC is explicitly set to true.
The default is false so that normal CI logictest runs keep deterministic
query plans. Execbuilder and SQLite logic tests guard against perturbation
via DisableOptimizerPerturbations on their TestServerArgs.

Fixes cockroachdb#93825

Release note: None
Signed-off-by: Lohit Kolluri <lohitkolluri@gmail.com>
@lohitkolluri lohitkolluri force-pushed the opt-metamorphic-optimizer-settings branch from 48f5d0b to d150634 Compare June 2, 2026 09:38
@lohitkolluri
Copy link
Copy Markdown
Author

Hi, I’ve put up a patch that should address the CI failures. When you have a chance, could you please take a look and review it? Also, if possible, could you trigger the CI once reviewed?

Default COCKROACH_LOGIC_TEST_OPTIMIZER_METAMORPHIC to false so
existing CI logictest behavior is unchanged. The opt-in env var
is still available for targeted metamorphic testing.

Fixes cockroachdb#170882

Signed-off-by: Lohit Kolluri <lohitkolluri@gmail.com>
@blathers-crl
Copy link
Copy Markdown

blathers-crl Bot commented Jun 2, 2026

Thank you for updating your pull request.

Before a member of our team reviews your PR, I have some potential action items for you:

  • We notice you have more than one commit in your PR. We try break logical changes into separate commits, but commits such as "fix typo" or "address review commits" should be squashed into one commit and pushed with --force
  • Please ensure your git commit message contains a release note.
  • When CI has completed, please ensure no errors have appeared.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@lohitkolluri
Copy link
Copy Markdown
Author

@rytaft @michae2 All CI checks are passing now. Would you like me to squash the commits and push an updated branch?

@michae2
Copy link
Copy Markdown
Collaborator

michae2 commented Jun 2, 2026

Thanks @lohitkolluri, no need to squash. We will merge this in a little bit, we're currently making some changes to our CI process so there will be a delay of a day or two.

@lohitkolluri
Copy link
Copy Markdown
Author

Thanks, Michael. Appreciate the update and the review. I'll leave the branch as-is and keep an eye on the PR. Thanks again for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

O-community Originated from the community X-blathers-triaged blathers was able to find an owner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql: use optimizer cost perturbation and disable rule settings for metamorphic testing

4 participants