PXB-3807 : Verify system and undo tablespace checksums during --check-tables

https://perconadev.atlassian.net/browse/PXB-3807

Problem:
--------
--check-tables only walks fsp_is_ibd_tablespace() B-trees, so the system
tablespace (ibdata*, which also holds the legacy rollback segments) and the
undo tablespaces -- rollback segments, change buffer, undo logs, allocation/
free pages -- received no integrity check at all.

Fix:
----
Add xb_checksum_system_undo_spaces(), a single-threaded, engine-native pass
run at the top of the check-tables phase over space 0 (TRX_SYS_SPACE) and
every undo::spaces entry. Each page is read with fil_io() into a private
aligned buffer (ut_unique_ptr) -- NOT into the buffer pool -- which maps the
page to the right ibdata file and decrypts/decompresses it, while
deliberately bypassing buf_page_io_complete()'s abort-on-corruption policy.
Each page is then run through BlockReporter::is_corrupted() (the same checksum
core innochecksum uses) and corruption is reported gracefully (non-zero exit),
not aborted. Legacy doublewrite pages in the system tablespace are skipped.

The buffer pool is flushed first (buf_flush_sync_all_buf_pools()) so the raw
on-disk image read here equals the recovered state -- post-recovery undo/
trx-sys/purge init can leave system pages dirty, which would otherwise read as
a stale/torn page and report a false positive. Flushing does not advance the
redo checkpoint, so it is safe under --apply-log-only.

Tests:
------
suites/check_tables/system_undo_checksum.sh corrupts a cold page of an undo
tablespace and of ibdata1 and asserts --check-tables reports a checksum
mismatch and exits non-zero, with a clean-run control; system_undo_checksum_
debug.sh uses the check_system_inject_corruption DBUG point plus a clean
control.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: satya-bodapati

storage/innobase/xtrabackup/src/xtrabackup.cc

@@ -102,6 +102,7 @@ Place, Suite 330, Boston, MA 02111-1307 USA
 
 #include "backup_copy.h"
 #include "backup_mysql.h"
+#include "buf0flu.h"
 #include "changed_page_tracking.h"
 #include "crc_glue.h"
 #include "ddl_tracker.h"
@@ -114,6 +115,7 @@ Place, Suite 330, Boston, MA 02111-1307 USA
 #include "read_filt.h"
 #include "redo_log.h"
 #include "space_map.h"
+#include "trx0purge.h"
 #include "utils.h"
 #include "write_filt.h"
 #include "wsrep.h"
@@ -6794,6 +6796,89 @@ Threads share a datafiles_iter and each grabs the next tablespace to check.
 On corruption, sets *ctxt->error to true but does NOT stop -- all threads
 run to completion so the operator gets a full damage report.
 @param[in]     ctxt          shared context used by all threads */
+/* PXB-3807: verify the per-page checksums of the system tablespace (ibdata*,
+which also holds the legacy rollback segments) and every undo tablespace during
+--check-tables.  The B-tree validation below (check_tables_thread_func) only
+visits fsp_is_ibd_tablespace() spaces, so without this pass the system and undo
+pages -- rollback segments, change buffer, undo logs, allocation/free pages --
+get no integrity check at all.
+
+Single-threaded and engine-native: each page is read with fil_io(), which maps
+the page number to the right ibdata file and decrypts/decompresses it from the
+space's encryption metadata, into a private buffer -- NOT into the buffer pool.
+Reading outside the buffer pool deliberately avoids buf_page_io_complete(),
+whose corrupt-page policy is to abort (or, under srv_force_recovery, silently
+ignore the corruption); instead we run each page through
+BlockReporter::is_corrupted() ourselves (the same checksum core innochecksum
+uses) and report gracefully. The caller flushes the buffer pool first
+(buf_flush_sync_all_buf_pools) so the on-disk image read here equals the
+recovered state.
+
+@return true if every page passed, false if any page is corrupt or unreadable.
+*/
+static bool xb_checksum_system_undo_spaces() {
+  /* Targets are the system tablespace (ibdata*, space 0) and every undo
+  tablespace. Collect their ids directly -- space 0 plus the global undo list
+  -- instead of scanning every fil space.  This runs at the top level of
+  --check-tables, where no other thread registers or drops undo spaces, so
+  reading undo::spaces without taking its latch is safe. */
+  std::vector<space_id_t> space_ids;
+  space_ids.push_back(TRX_SYS_SPACE);
+  for (size_t i = 0; i < undo::spaces->size(); ++i) {
+    space_ids.push_back(undo::spaces->at(i)->id());
+  }
+
+  /* One aligned page buffer, reused for every read (O_DIRECT needs alignment).
+  ut_unique_ptr frees it automatically when this function returns. */
+  ut_unique_ptr buf_holder = ut_make_unique_ptr_nokey(2 * UNIV_PAGE_SIZE);
+  byte *buf = static_cast<byte *>(ut_align(buf_holder.get(), UNIV_PAGE_SIZE));
+
+  bool ok = true;
+
+  for (const space_id_t space_id : space_ids) {
+    const page_size_t page_size(fil_space_get_flags(space_id));
+    const page_no_t n_pages = fil_space_get_size(space_id);
+    const bool is_system = fsp_is_system_tablespace(space_id);
+
+    xb::info() << "check-tables: verifying checksums of tablespace " << space_id
+               << " (" << n_pages << " pages).";
+
+    for (page_no_t page_no = 0; page_no < n_pages; ++page_no) {
+      /* Legacy doublewrite-buffer pages in the system tablespace carry
+      non-standard checksums; skip them (as innochecksum does). */
+      if (is_system && page_no >= FSP_EXTENT_SIZE &&
+          page_no < FSP_EXTENT_SIZE * 3) {
+        continue;
+      }
+
+      const page_id_t page_id(space_id, page_no);
+      const dberr_t err =
+          fil_io(IORequest(IORequest::READ), true, page_id, page_size, 0,
+                 page_size.physical(), buf, nullptr);
+      if (err != DB_SUCCESS) {
+        xb::error() << "check-tables: cannot read page " << space_id << ":"
+                    << page_no
+                    << " for checksum verification: " << ut_strerr(err) << ".";
+        ok = false;
+        continue;
+      }
+
+      BlockReporter reporter(false /* check_lsn */, buf, page_size,
+                             fsp_is_checksum_disabled(space_id));
+      bool corrupt = reporter.is_corrupted();
+      DBUG_EXECUTE_IF("check_system_inject_corruption",
+                      corrupt = (page_no > 0););
+      if (corrupt) {
+        xb::error() << "check-tables: checksum mismatch on page " << space_id
+                    << ":" << page_no << "; the tablespace is corrupted.";
+        ok = false;
+      }
+    }
+  }
+
+  return ok;
+}
+
 static void check_tables_thread_func(data_thread_ctxt_t *ctxt) {
   fil_node_t *node;
 
@@ -7229,11 +7314,26 @@ static void xtrabackup_prepare_func(int argc, char **argv) {
     uint ct_count;
     ib_mutex_t ct_count_mutex;
 
+    /* PXB-3807: redo recovery has finished, but some system/undo pages may
+    still be dirty in the buffer pool (post-recovery undo/trx-sys/purge init
+    re-dirty them). Flush once so the on-disk image read by the checksum scan
+    below equals the recovered state -- otherwise a raw read could see a stale
+    or torn page and report a false positive. Flushing pages does not advance
+    the redo checkpoint, so this is safe under --apply-log-only too. */
+    buf_flush_sync_all_buf_pools();
+
     /* --check-tables only reads; redo has already been applied. Forbid the
     Fil_shard::do_io() out-of-bounds auto-extend (PXB-1819) for the duration
     of validation so a corrupt page pointer cannot grow a backup file. */
     fil_check_tables_no_extend.store(true);
 
+    /* PXB-3807: verify the per-page checksums of the system tablespace
+    (ibdata*) and all undo tablespaces, which the B-tree validation below
+    skips entirely. */
+    if (!xb_checksum_system_undo_spaces()) {
+      check_failed = true;
+    }
+
     it = datafiles_iter_new(nullptr);
     if (it == NULL) {
       xb::error() << "datafiles_iter_new() failed.";

storage/innobase/xtrabackup/test/suites/check_tables/system_undo_checksum.sh

@@ -0,0 +1,121 @@
+############################################################################
+# PXB-3807 : --check-tables verifies the per-page checksums of the system
+# tablespace (ibdata*) and all undo tablespaces.  These spaces are skipped by
+# the B-tree validation, so without this pass their pages get no integrity
+# check at all.
+#
+# Real on-disk corruption (no debug build required): corrupt a COLD page -- one
+# InnoDB does not read during startup -- near the end of an undo tablespace and
+# of ibdata1, leaving the stored checksum stale.  --prepare --check-tables must
+# detect the mismatch, report it, and exit non-zero, WITHOUT extending the file
+# or hanging.  A clean-run control proves there are no false positives (the
+# buffer-pool flush before the scan makes the on-disk image authoritative).
+#
+# Cold pages are chosen near the high end of each file (free/unused pages that
+# recovery does not touch) so the corruption is caught by the read-only
+# checksum scan rather than by a crashing buffer-pool read at startup.
+############################################################################
+
+. inc/common.sh
+
+PAGE_SZ=16384
+
+# corrupt_last_page <file> -- flip a body byte of the file's last page, leaving
+# the stored checksum stale.  A free (all-zero) last page becomes a non-empty
+# page with a zero checksum field, which is a checksum mismatch.
+corrupt_last_page() {
+  local F="$1"
+  local SZ LASTPAGE
+  SZ=$(stat -c %s "$F")
+  LASTPAGE=$(( SZ / PAGE_SZ - 1 ))
+  vlog "corrupting $F : last page $LASTPAGE at byte offset 200"
+  mach_write_8 "$F" "$LASTPAGE" 200 0xDEADBEEFDEADBEEF
+}
+
+start_server
+
+vlog "Create a table and generate undo (rollback) so undo pages are allocated"
+mysql test <<EOF
+CREATE TABLE t1 (a INT PRIMARY KEY AUTO_INCREMENT, b VARCHAR(200));
+EOF
+for i in $(seq 1 200); do
+  run_cmd $MYSQL $MYSQL_ARGS test -e \
+    "INSERT INTO t1 (b) VALUES (REPEAT('x', 200));"
+done
+run_cmd $MYSQL $MYSQL_ARGS test -e \
+  "BEGIN; UPDATE t1 SET b = REPEAT('y', 200); ROLLBACK;"
+
+vlog "Take backup"
+xtrabackup --backup --target-dir=$topdir/backup
+
+#
+# Control: a healthy backup must pass, and the new checksum pass must run.
+#
+vlog "=== Control: clean system/undo checksum pass ==="
+cp -r $topdir/backup $topdir/backup_ok
+xtrabackup --prepare --apply-log-only --target-dir=$topdir/backup_ok
+xtrabackup --prepare --check-tables --target-dir=$topdir/backup_ok 2>&1 \
+  | tee $topdir/ok.log
+grep -q "verifying checksums of tablespace" $topdir/ok.log || \
+  die "Control: system/undo checksum pass did not run"
+grep -q "All table checks passed" $topdir/ok.log || \
+  die "Control: clean backup unexpectedly failed --check-tables"
+vlog "Control passed"
+
+#
+# Negative (undo): corrupt a cold page of an undo tablespace.
+#
+vlog "=== Negative: corrupt undo tablespace ==="
+cp -r $topdir/backup $topdir/backup_undo
+xtrabackup --prepare --apply-log-only --target-dir=$topdir/backup_undo
+UNDO=$(ls $topdir/backup_undo/undo_* 2>/dev/null | head -1)
+[ -n "$UNDO" ] || die "could not find an undo tablespace in the backup"
+corrupt_last_page "$UNDO"
+
+run_cmd_expect_failure $XB_BIN $XB_ARGS --prepare --check-tables \
+  --target-dir=$topdir/backup_undo 2>&1 | tee $topdir/undo.log
+grep -q "checksum mismatch" $topdir/undo.log || \
+  die "Negative(undo): checksum mismatch not reported"
+grep -q "Table check failed" $topdir/undo.log || \
+  die "Negative(undo): Table check failed message not found"
+grep -qi "posix_fallocate" $topdir/undo.log && \
+  die "Negative(undo): validation attempted to extend a file"
+vlog "Negative(undo) passed"
+
+#
+# Negative (ibdata): corrupt a cold page of the system tablespace.
+#
+vlog "=== Negative: corrupt system tablespace (ibdata1) ==="
+cp -r $topdir/backup $topdir/backup_sys
+xtrabackup --prepare --apply-log-only --target-dir=$topdir/backup_sys
+corrupt_last_page "$topdir/backup_sys/ibdata1"
+
+run_cmd_expect_failure $XB_BIN $XB_ARGS --prepare --check-tables \
+  --target-dir=$topdir/backup_sys 2>&1 | tee $topdir/sys.log
+grep -q "checksum mismatch" $topdir/sys.log || \
+  die "Negative(ibdata): checksum mismatch not reported"
+grep -q "Table check failed" $topdir/sys.log || \
+  die "Negative(ibdata): Table check failed message not found"
+vlog "Negative(ibdata) passed"
+
+#
+# Negative (ibdata startup-range / "hot" page): page 5 of the system tablespace
+# is TRX_SYS, read by InnoDB during startup recovery -- BEFORE the checksum
+# scan.  Corrupting it is therefore caught by the normal buffer-pool read
+# during recovery (which aborts after retries with "Unable to read page"),
+# not by our graceful scan.  Either way --prepare must fail (non-zero); this
+# documents that startup-range corruption is surfaced, just via a different
+# path than the cold-page scan above.
+#
+vlog "=== Negative: corrupt ibdata1 startup-range page (TRX_SYS, page 5) ==="
+cp -r $topdir/backup $topdir/backup_hot
+# Do NOT --apply-log-only first: we want recovery to read the corrupt page.
+mach_write_8 "$topdir/backup_hot/ibdata1" 5 200 0xDEADBEEFDEADBEEF
+run_cmd_expect_failure $XB_BIN $XB_ARGS --prepare --check-tables \
+  --target-dir=$topdir/backup_hot 2>&1 | tee $topdir/hot.log
+grep -Eqi "Unable to read page|page corruption|checksum mismatch|is corrupted|corrupt" \
+  $topdir/hot.log || \
+  die "Negative(ibdata hot): startup-range corruption not surfaced"
+vlog "Negative(ibdata hot) passed"
+
+vlog "All PXB-3807 sub-tests passed"

storage/innobase/xtrabackup/test/suites/check_tables/system_undo_checksum_debug.sh

@@ -0,0 +1,61 @@
+############################################################################
+# PXB-3807 : --check-tables verifies the per-page checksums of the system
+# tablespace (ibdata*) and all undo tablespaces, which the B-tree validation
+# skips entirely.
+#
+# This debug test uses a DBUG injection (check_system_inject_corruption) for a
+# deterministic positive: with the injection the checksum pass flags pages and
+# --prepare --check-tables must fail.  It also includes a clean-run control to
+# prove the pass does not false-positive on a healthy backup (the buffer-pool
+# flush before the scan makes the on-disk image authoritative).
+############################################################################
+
+. inc/common.sh
+
+require_debug_pxb_version
+
+start_server
+
+vlog "Create a table and generate some undo (rollback) so undo pages are used"
+mysql test <<EOF
+CREATE TABLE t1 (a INT PRIMARY KEY AUTO_INCREMENT, b VARCHAR(200));
+EOF
+for i in $(seq 1 100); do
+  run_cmd $MYSQL $MYSQL_ARGS test -e \
+    "INSERT INTO t1 (b) VALUES (REPEAT('x', 200));"
+done
+run_cmd $MYSQL $MYSQL_ARGS test -e \
+  "BEGIN; UPDATE t1 SET b = REPEAT('y', 200); ROLLBACK;"
+
+vlog "Take backup"
+xtrabackup --backup --target-dir=$topdir/backup
+
+#
+# Control: a healthy backup must pass, and the new checksum pass must run.
+#
+vlog "=== Control: clean system/undo checksum pass ==="
+cp -r $topdir/backup $topdir/backup_ok
+xtrabackup --prepare --check-tables --target-dir=$topdir/backup_ok 2>&1 \
+  | tee $topdir/ok.log
+grep -q "verifying checksums of tablespace" $topdir/ok.log || \
+  die "Control: system/undo checksum pass did not run"
+grep -q "All table checks passed" $topdir/ok.log || \
+  die "Control: clean backup unexpectedly failed --check-tables"
+vlog "Control passed"
+
+#
+# Injected corruption: the checksum pass must report and fail the prepare.
+#
+vlog "=== Injected corruption: check_system_inject_corruption ==="
+cp -r $topdir/backup $topdir/backup_inj
+run_cmd_expect_failure $XB_BIN $XB_ARGS --prepare --check-tables \
+  --debug=d,check_system_inject_corruption \
+  --target-dir=$topdir/backup_inj 2>&1 | tee $topdir/inj.log
+
+grep -q "checksum mismatch" $topdir/inj.log || \
+  die "Injected: checksum mismatch not reported"
+grep -q "Table check failed" $topdir/inj.log || \
+  die "Injected: Table check failed message not found"
+vlog "Injected corruption sub-test passed"
+
+vlog "All PXB-3807 debug sub-tests passed"