fix(bb): clamp BatchMergeProver degree-check loop to fix nightly debug SIGABRT

[AztecBot]

Summary

Unblocks the Nightly Debug Build on next. With _GLIBCXX_DEBUG enabled, BatchMergeTests/2.TooManySubtablesFails aborts (exit 134) with:

Error: attempt to subscript container with out-of-bounds index 40,
       but container only holds 40 elements.

Root cause

TooManySubtablesFails uses BB_DISABLE_ASSERTS() so it can drive the verifier with N = max_subtables + 1 subtables. With BB asserts demoted to warnings, control flow falls past the BB_ASSERT_LTE(N, M, ...) guard at the top of BatchMergeProver::construct_proof and reaches compute_degree_check_polynomial, which iterates over flattened_columns.size() (sized for N) but indexes degree_check_challenges (sized for (M+1) * NUM_WIRES). In Release the over-read returns garbage that the verifier rejects — exactly what the test wants. In Debug, libstdc++'s _GLIBCXX_DEBUG (set by the debug preset's CXXFLAGS) bounds-checks std::vector::operator[] and traps. BB_DISABLE_ASSERTS() cannot suppress this — the bounds check is libstdc++'s, not bb's.

Why the previously-merged fix didn't fix anything

PR #23019 (already merged into merge-train/barretenberg via the open #23025) was titled the same as this PR but only modified TweakableBatchMergeProver::construct_proof. The actual TooManySubtablesFails test path uses prove_and_verify with the default fault_mode = FaultMode::NONE, which routes through BatchMergeProver (the base class), not TweakableBatchMergeProver. I confirmed this in this container by building goblin_tests against origin/merge-train/barretenberg HEAD with the debug preset — the same OOB still aborts. The merge train cannot deliver this fix without a real prover-side change.

Fix

Clamp the loop to min(flattened_columns.size(), degree_check_challenges.size()) inside BatchMergeProver::compute_degree_check_polynomial. Normal paths are unaffected because both sizes equal (M+1) * NUM_WIRES whenever N <= M (the assert holds). The misuse path now produces a partial degree-check polynomial that the verifier still rejects in both Debug and Release — same observable behaviour, no UB.

This is the same one-file change as the open draft #22976; opening this fresh PR against current next so it doesn't sit silently in draft.

Verification

Reproduced inside the ClaudeBox container on f23aa82c52 (current next HEAD) with the same flags as the nightly:

cd barretenberg/cpp
HOME=/tmp cmake --preset debug -DAVM_TRANSPILER_LIB=
cd build-debug && ninja goblin_tests
./bin/goblin_tests --gtest_filter='*TooManySubtablesFails*'

• Pre-fix: aborts on BatchMergeTests/2.TooManySubtablesFails with the OOB shown above (exit 134). Confirmed identical abort on origin/merge-train/barretenberg HEAD.
• Post-fix: 2 passed, 2 skipped (the native-curve cases self-skip).

Full goblin_tests post-fix: 68 passed / 7 skipped / 0 failed (342s).

Related PRs

• fix(bb): allocate gemini masking poly with virtual size == dyadic #22937 (merged) — fixed the prior abort site (gemini_masking_poly virtual size).
• fix(bb): clamp BatchMergeProver degree-check loop to fix nightly debug SIGABRT #23019 (merged into merge-train/barretenberg via open feat: merge-train/barretenberg #23025) — same title, but the diff only modified TweakableBatchMergeProver, which is unused on this test path; it is a no-op for the nightly.
• fix(bb): clamp BatchMergeProver degree-check loop to fix nightly debug SIGABRT #22976 (open draft) — same one-file prover clamp as this PR, against an older base.

Detailed analysis: https://gist.github.com/AztecBot/7be72c96a1d3d18458dce92a828116a2

ClaudeBox log: https://claudebox.work/s/8ad866e315acbe92?run=1

ClaudeBox log: https://claudebox.work/s/8ad866e315acbe92?run=1