opt: add optimizer_span_limit and use in GenerateConstrainedScans

[michae2]

opt: add optimizer_span_limit and use in GenerateConstrainedScans

This commit creates a new session variable optimizer_span_limit, and
uses it to bound the number of spans in constrained scans produced by
GenerateConstrainedScans.

Fixes: #167620

Release note: Add a new session variable optimizer_span_limit which
bounds the number of spans the optimizer will allow in a single
constrained index scan. If a single IN set has more items than this
limit, that IN set will not be used to build a constrained index
scan. If the cross product of two or more IN sets would produce more
spans than this limit for a composite index, then only a prefix of the
IN sets will be used to produce spans.

For example, for the following table and query, only the predicates on
columns a and b will be used to construct the constrained scan of
abc_idx, because including the predicate on column c would produce
more spans than optimizer_span_limit:

CREATE TABLE abc (a INT, b INT, c INT, INDEX abc_idx (a, b, c));
SET optimizer_span_limit = 10;
SELECT * FROM abc WHERE a IN (1, 3, 5) AND b IN (2, 4, 6) AND c IN (7, 9, 11);

Co-Authored-By: roachdev-claude roachdev-claude-bot@cockroachlabs.com

---

opt: set default optimizer_span_limit to 131072

Set the default value of the optimizer_span_limit session variable to
2^17 (131072) instead of 0 (no limit). This provides a reasonable upper
bound on the number of spans the optimizer will produce, preventing
pathological cases where large cross-products of IN sets produce
excessive spans.

Informs: #167620

Release note (sql change): The session variable optimizer_span_limit
now defaults to 131072. This bounds the number of spans the optimizer
will allow in constrained index scans generated during query
optimization. Queries that would exceed this limit will use fewer looser
spans with remaining filters instead. Set to 0 to disable the limit.

Co-Authored-By: roachdev-claude roachdev-claude-bot@cockroachlabs.com

[trunk-io]

😎 Merged successfully - details.

[cockroach-teamcity]

This change is 

[michae2]

I'm just looking to backport the first commit.

[DrewKimball]

Thanks for doing this!

@DrewKimball reviewed 23 files and all commit messages, and made 1 comment.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on mw5h).

[michae2]

You're welcome! TFTR!

/trunk merge

[blathers-crl]

Based on the specified backports for this PR, I applied new labels to the following linked issue(s). Please adjust the labels as needed to match the branches actually affected by the issue(s), including adding any known older branches.

---

Issue #167620: branch-release-25.4, branch-release-26.1, branch-release-26.2.

---

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[blathers-crl]

Encountered an error creating backports. Some common things that can go wrong:

1. The backport branch might have already existed.
2. There was a merge conflict.
3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.

---

💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

merge conflict cherry-picking 52376f6 to blathers/backport-release-25.4-167180

Backport to branch 25.4.x failed. See errors above.

---

💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

merge conflict cherry-picking 52376f6 to blathers/backport-release-26.1-167180

Backport to branch 26.1.x failed. See errors above.

---

💡 Consider backporting to the fork repo instead of the main repo. See instructions for more details.

merge conflict cherry-picking 52376f6 to blathers/backport-release-26.2-167180

Backport to branch 26.2.x failed. See errors above.

---

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[michae2]

blathers auth-check

[michae2]

You shall pass