Skip to content

Build index scans when tuples are compared for equality or used in set membership.#3541

Open
nicktobey wants to merge 2 commits into
mainfrom
nicktobey/tuples
Open

Build index scans when tuples are compared for equality or used in set membership.#3541
nicktobey wants to merge 2 commits into
mainfrom
nicktobey/tuples

Conversation

@nicktobey

@nicktobey nicktobey commented May 8, 2026

Copy link
Copy Markdown
Contributor

Fixes dolthub/dolt#11003

Currently, Dolt will not optimize queries if index columns are used inside tuples. This can create problems for multi-column indexes, in situations where it is natural to represent the entire key as a tuple.

The most likely situation is one where the user wants to delete a specific set of records, and writes a query that looks like this:

delete from two_pk where (pk1, pk2) in (
  (a1, a2),
  (b1, b2),
  (c1, c2),
   ...
);

Currently, Dolt will never optimize this query to use an index. As written, the query will do a full table scan. Alternatively, the user can execute an individual delete statement for each row, but for a table with N rows, this will cause Dolt to write O(log N) garbage chunks for each row deleted. For large datasets, neither option is efficient.

Another workaround would be for the user to rewrite the query into an equivalent query that can be optimized. Such a query might look like this:

DELETE FROM two_pk WHERE (pk1 = a1 AND pk2 = a2) OR (pk1 = b1 AND pk2 = b2) OR (pk1 = c1 AND pk2 = c2)...

However, there's a compelling reason to not require the user to do this: prepared statements.

For example, the python SQL driver allows using a prepared statement like this:

tuples_to_remove = [(a1, a2), (b1, b2), (c1, c2)]
cursor.execute("delete from two_pk where (pk1, pk2) in %s", tuples_to_remove)

No equivalent exists for the workaround query: the user would need to manually construct a different query string based on the number of rows to remove, and clients dynamically constructing SQL query strings is typically a bad practice. Thus, it makes sense to support optimizing the original query.

There are two different ways to address this: either improving the analyzer so that it can analyze tuple expressions, or rewriting the filter internally to match the workaround query's filter.

This PR is an attempt to implement the latter, by allowing a single node in the costed index scan tree (the iScanLeaf struct) to describe a filter on multiple index columns.

In hindsight, rewriting the filter might have been easier, and would also better handle cases where only some of the columns in the tuple match the index keys. But such an approach could have its own pitfalls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suboptimal plan generated for DELETE on tables with multiple public key columns

1 participant