cli: add Size column to \l+ and \dt+ #170978
Draft
rafiss wants to merge 3 commits into
Draft
Conversation
Extends the table metadata cache so each row's details JSONB column records the table's primary index ID and a map of every live index ID to its replication size in bytes. The data is fetched in the same batched SpanStats RPC the cache job already uses for the table-level total: each table now contributes one extra span per live index. The per-index map and primary index ID are nullable / omitempty so upgrades remain a pure additive change — existing readers see the same JSONB shape, plus the new fields when present. This sets the stage for the SQL-surface follow-up that will: * tighten pg_relation_size and pg_table_size to return only the primary index size, * add pg_indexes_size and pg_relation_size(index_oid), while keeping pg_total_relation_size on the existing whole-table-prefix replication_size_bytes (so it still includes any dropped-index garbage that hasn't been GC'd, and the gap surfaces GC pressure to operators). Informs cockroachdb#86017. Release note: None Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
Builds on the per-index data added to system.table_metadata.details to:
- Tighten pg_relation_size(table_oid) and pg_table_size(table_oid) to
return only the primary index's cached size, matching PG's "heap
only" semantics. CockroachDB's primary index is the row data, so
"primary index size" is the natural CRDB equivalent of PG's heap.
- Have pg_relation_size also accept an index OID, returning that
single index's cached size. Resolution traverses pg_class so this
branch respects user visibility (a small documented inconsistency
with the table-OID branch, which uses system.descriptor with the
NodeUser override).
- Add pg_indexes_size(table_oid), which sums the cached sizes of the
relation's secondary indexes. CockroachDB intentionally excludes
the primary, since including it would double-count against
pg_table_size and break the identity
pg_table_size + pg_indexes_size <= pg_total_relation_size.
pg_total_relation_size is unchanged: it continues to read the
whole-table-prefix replication_size_bytes, so any dropped-index data
still occupying the keyspace shows up as the gap between
pg_total_relation_size and (pg_table_size + pg_indexes_size). That gap
is a useful GC-pressure signal for operators.
The per-index lookup falls back to replication_size_bytes via COALESCE
when a cache row predates the per-index extension (mixed-version
upgrade window). Once the cluster is fully upgraded and the cache
cycles, the fallback is dormant.
Resolves cockroachdb#86017.
Release note (sql change): pg_relation_size and pg_table_size now
return the size of the primary index only, matching PostgreSQL's
"heap only" semantics. Their previous (over-counting) behavior is
preserved by pg_total_relation_size. pg_relation_size also accepts
an index OID, returning that index's cached size. A new
pg_indexes_size built-in returns the sum of the relation's secondary
index sizes.
Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
PG's psql shows database and per-table on-disk sizes in the verbose variants of \l and \dt. CRDB's CLI carried explicit TODOs next to those columns waiting for the pg_database_size and pg_table_size builtins to exist. Now that both builtins (and pg_size_pretty) are available, wire them up. This makes the \l+ and \dt+ queries match Postgres exactly. The sizes come from the periodically-refreshed system.table_metadata cache and may lag the truth by minutes. TestDescribe gains a seed of system.table_metadata so the goldens exercise the real per-index lookup path. The details JSONB mirrors what tableMetadataUpdater writes in production (replica_count + primary_index_id + index_sizes), so we are not just relying on the mixed-version replication_size_bytes fallback. Resolves: cockroachdb#85834 Release note (cli change): The \l+ and \dt+ metacommands in the built-in SQL shell now include a Size column showing on-disk byte counts, matching PostgreSQL's psql. Database sizes come from pg_database_size; table sizes come from pg_table_size. Values are read from a periodically-refreshed internal cache and may lag the live byte count by minutes.
Contributor
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
Member
|
This change is |
rafiss
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PG's psql shows database and per-table on-disk sizes in the verbose
variants of \l and \dt. CRDB's CLI carried explicit TODOs next to
those columns waiting for the pg_database_size and pg_table_size
builtins to exist. Now that both builtins (and pg_size_pretty) are
available, wire them up.
This makes the \l+ and \dt+ queries match Postgres exactly. The
sizes come from the periodically-refreshed system.table_metadata
cache and may lag the truth by minutes.
TestDescribe gains a seed of system.table_metadata so the goldens
exercise the real per-index lookup path. The details JSONB mirrors
what tableMetadataUpdater writes in production (replica_count +
primary_index_id + index_sizes), so we are not just relying on the
mixed-version replication_size_bytes fallback.
Resolves: #85834
Release note (cli change): The \l+ and \dt+ metacommands in the
built-in SQL shell now include a Size column showing on-disk byte
counts, matching PostgreSQL's psql. Database sizes come from
pg_database_size; table sizes come from pg_table_size. Values are
read from a periodically-refreshed internal cache
and may lag the live byte count by minutes.