Skip to content

obs,sql,server,ui: add statement store and V2 SQL Activity endpoints#167502

Draft
kyle-a-wong wants to merge 6 commits intocockroachdb:masterfrom
kyle-a-wong:statement_store
Draft

obs,sql,server,ui: add statement store and V2 SQL Activity endpoints#167502
kyle-a-wong wants to merge 6 commits intocockroachdb:masterfrom
kyle-a-wong:statement_store

Conversation

@kyle-a-wong
Copy link
Copy Markdown
Contributor

Summary

Add a new statement fingerprint store and V2 SQL Activity API endpoints
with server-side pagination, sorting, and filtering. This replaces the
expensive client-side aggregation in the existing gRPC endpoints with
efficient server-side queries against the denormalized covering indexes
on system.statement_statistics and system.transaction_statistics.

  • Statement store (pkg/obs/statementstore): Write-through FIFO
    cache that deduplicates and asynchronously persists statement
    fingerprints to system.statements via a background writer goroutine.
  • Ingester integration: The SQL stats ingester calls the store on
    each statement observation, including when fingerprint IDs are
    recomputed due to implicit/explicit transaction changes.
  • Backfill migration: A v26.3 cluster upgrade backfills
    system.statements from system.statement_statistics so fingerprints
    observed before the store existed are immediately available.
  • V2 BFF endpoints: GET /api/v2/dbconsole/statements/ and
    GET /api/v2/dbconsole/transactions/ with server-side pagination,
    sorting, time range, search, app name, database, and internal
    statement filters. Datadriven EXPLAIN tests verify covering index
    usage.
  • UI pages: StatementsPageV2 and TransactionsPageV2 React
    components using SWR hooks, accessible from the Advanced Debug panel.

The individual commits speak for themselves — each is a self-contained
logical unit that builds on the previous one.

Epic: none

Release note: None

kyle-a-wong and others added 6 commits April 3, 2026 16:18
Introduce `StatementStore`, a write-through cache that deduplicates and
asynchronously persists statement fingerprints to `system.statements`.

The store uses an in-memory FIFO cache (sized via the
`obs.statements.store.cache_size` cluster setting, default 100k entries)
to skip fingerprints that have already been seen. Cache misses are
enqueued to a buffered channel and flushed in batches by a background
goroutine. When the channel is full, items are silently dropped and
retried after cache eviction.

On startup, the cache is prewarmed with the most recent 1000 fingerprint
IDs from `system.statements` to avoid redundant upserts after restarts.

The store is disabled/enabled via the `obs.statements.store.enabled`
cluster setting (default true).

Epic: none
Release note: None

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
Add a `*statementstore.StatementStore` parameter to
`NewSQLStatsIngester` so the ingester can persist statement fingerprints
as they flow through the stats pipeline.

The ingester calls `storeStatementFingerprint` at two points:
- In `processStatement`, when a statement is first observed.
- In `flushBuffer`, when the fingerprint ID is recomputed because the
  transaction's `ImplicitTxn` value changed (explicit vs implicit).

When the store is nil (e.g. in unit tests that don't need persistence),
the call is a no-op.

Epic: none
Release note: None

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
Create a `StatementStore` in `NewServer`, configure its internal
executor after the `InternalDB` is available, pass it to the
`SQLStatsIngester`, and start its background writer alongside the
other stats subsystems.

The store's internal executor and memory monitor are set up via
`SetInternalExecutor` after `NewServer` returns (breaking a dependency
cycle), and `Start` is called during `Server.Start` alongside the
ingester and persisted stats.

Epic: none
Release note: None

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
Add cluster version `V26_3_PopulateSystemStatements` (26.2, Internal 4)
and a tenant upgrade that backfills `system.statements` with distinct
fingerprints from `system.statement_statistics`.

The migration seeds the statement store so that fingerprints observed
before the store existed are immediately available for the V2 SQL
Activity endpoints without waiting for re-execution.

The backfill uses `INSERT ... ON CONFLICT DO NOTHING` so it is
idempotent and safe to re-run. The `RestoreActionNotRequired` annotation
indicates that restoring a cluster predating this migration can leave
the table empty; fingerprints will be populated on next execution.

Epic: none
Release note: None

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
Add `GetStatements` and `GetTransactions` BFF endpoints under
`/api/v2/dbconsole/` that return paginated, sorted, aggregated
statement and transaction statistics.

These endpoints query the denormalized covering indexes on
`system.statement_statistics` and `system.transaction_statistics`
directly with server-side pagination and sorting, avoiding the
expensive full-table scans and client-side aggregation of the
existing gRPC endpoints.

Key features:
- Server-side pagination (pageSize, pageNum) and sorting (sortBy,
  sortOrder) with parameterized queries to prevent SQL injection.
- Time range filtering via Unix epoch seconds (start, end).
- Optional filters: search (ILIKE on fingerprint text), appName
  (exact match), database (exact match), excludeInternal (default
  true).
- Standard deviation computed from sum-of-squares columns.
- Transaction endpoint includes a secondary query to fetch statement
  query summaries via the system.statements lookup table.
- Datadriven EXPLAIN (SHAPE) tests verify that queries use the
  expected covering indexes.

Epic: none
Release note: None

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
Add `StatementsPageV2` and `TransactionsPageV2` React components to the
DB Console, accessible via the Advanced Debug page under "SQL Activity
(V2)". These pages use the new V2 BFF endpoints with server-side
pagination and sorting via SWR hooks.

Features:
- SWR-based data fetching with automatic cache key management.
- Server-side pagination and sorting (no client-side aggregation).
- Time scale dropdown with URL state persistence.
- Search, app name, database, and internal statement filters.
- Bar chart visualizations for latency, CPU, contention, and I/O
  metrics using the existing barChartFactory.
- Transaction page shows constituent statement summaries via the
  secondary query in the transactions endpoint.

The pages are currently only linked from the debug panel to allow
iterative development before replacing the existing SQL Activity pages.

Also includes the regenerated top-level pkg/BUILD.bazel for the new
statementstore package.

Epic: none
Release note: None

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@trunk-io
Copy link
Copy Markdown
Contributor

trunk-io bot commented Apr 3, 2026

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Apr 3, 2026

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 11.04m ±9% 10.95m ±8% ~ p=0.902 n=15
allocs/op 8.128k ±0% 8.175k ±1% ~ p=0.191 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/2944efe/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2944efe978a7b7eaa17826772f9d4ca633b6f75a/bin/pkg_sql_tests benchdiff/2944efe/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2944efe/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/3944370/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/3944370be734756fdc54b5aa04934f6c4315464c/bin/pkg_sql_tests benchdiff/3944370/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/3944370/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=3944370 --new=2944efe --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.572m ±10% 3.569m ±11% ~ p=0.567 n=15
allocs/op 2.108k ±0% 2.109k ±0% ~ p=0.120 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/2944efe/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2944efe978a7b7eaa17826772f9d4ca633b6f75a/bin/pkg_sql_tests benchdiff/2944efe/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2944efe/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/3944370/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/3944370be734756fdc54b5aa04934f6c4315464c/bin/pkg_sql_tests benchdiff/3944370/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/3944370/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=3944370 --new=2944efe --memprofile ./pkg/sql/tests
🔴 Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
🔴 sec/op 3.376m ±4% 3.516m ±3% +4.16% p=0.005 n=15
allocs/op 4.238k ±0% 4.237k ±0% ~ p=0.544 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/2944efe/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2944efe978a7b7eaa17826772f9d4ca633b6f75a/bin/pkg_sql_tests benchdiff/2944efe/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2944efe/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/3944370/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/3944370be734756fdc54b5aa04934f6c4315464c/bin/pkg_sql_tests benchdiff/3944370/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/3944370/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=3944370 --new=2944efe --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/2944efe978a7b7eaa17826772f9d4ca633b6f75a/23961324784-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/3944370be734756fdc54b5aa04934f6c4315464c/23961324784-1/\* old/

built with commit: 2944efe978a7b7eaa17826772f9d4ca633b6f75a

@cockroach-teamcity cockroach-teamcity added the X-perf-check Microbenchmarks CI: Added to a PR if a performance regression is detected and should be checked label Apr 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

X-perf-check Microbenchmarks CI: Added to a PR if a performance regression is detected and should be checked

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants