Skip to content

rbd: add debug logging to ProcessMetadata for CBT diagnostics#6203

Open
kaovilai wants to merge 1 commit into
ceph:develfrom
kaovilai:rbd-add-debug-logging-ProcessMetadata
Open

rbd: add debug logging to ProcessMetadata for CBT diagnostics#6203
kaovilai wants to merge 1 commit into
ceph:develfrom
kaovilai:rbd-add-debug-logging-ProcessMetadata

Conversation

@kaovilai
Copy link
Copy Markdown
Contributor

@kaovilai kaovilai commented Mar 24, 2026 •
edited
Loading

Summary

Add debug log statements to ProcessMetadata in snap_diff.go to improve observability of the CBT (Changed Block Tracking) metadata streaming path.

Why this is useful beyond a specific bug

The ProcessMetadata function involves multiple steps (snapshot ID resolution, snap context setup, DiffIterateByID call) but currently has no logging between them. When GetMetadataAllocated returns unexpected results (empty streams, wrong block counts), there's no visibility into:

  • Which snapshot ID was resolved and whether it matches expectations
  • What Offset/Length/FromSnapID parameters were passed to DiffIterateByID
  • How many blocks were buffered but not yet sent

These are all DebugLog calls (not visible at default log levels), so there's no performance impact in production.

Changes

Three log.DebugLog calls added:

  1. After snapshot ID resolution — logs snapshot name, ID, and volume size
  2. Before DiffIterateByID — logs config parameters (Offset, Length, FromSnapID)
  3. After DiffIterateByID completes — logs remaining buffered block count

Fixes: #6206

Note

Responses generated with Claude

Copilot AI review requested due to automatic review settings March 24, 2026 23:43
@mergify mergify Bot added component/rbd Issues related to RBD bug Something isn't working labels Mar 24, 2026
@kaovilai kaovilai force-pushed the rbd-add-debug-logging-ProcessMetadata branch from afeabf6 to b94a798 Compare March 24, 2026 23:45
Copilot AI reviewed Mar 24, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds targeted debug logging to the RBD snapshot metadata streaming path (ProcessMetadata) to help diagnose CBT cases where metadata streams are unexpectedly empty.

Changes:

  • Log resolved snapshot ID and volume size before setting the snapshot context.
  • Log DiffIterateByID configuration parameters (offset/length/from-snap).
  • Log completion status and the number of buffered blocks after DiffIterateByID returns.

Comment thread internal/rbd/snap_diff.go Outdated
Comment thread internal/rbd/snap_diff.go
Comment thread internal/rbd/snap_diff.go Outdated
@nixpanic nixpanic requested a review from Rakshith-R March 26, 2026 15:52
@nixpanic
Copy link
Copy Markdown
Member

nixpanic commented Mar 26, 2026

Is this debugging useful outside the scope where a potential bug needs to be addressed?

Can you explain a little more about the issue that you are facing?

To get the PR accepted, you will need to squash the extra/correction commits into the original one.

@kaovilai kaovilai force-pushed the rbd-add-debug-logging-ProcessMetadata branch from 0c8b85e to f68485a Compare March 26, 2026 19:13
@kaovilai
Copy link
Copy Markdown
Contributor Author

kaovilai commented Mar 26, 2026

squashed, pr body updated.

nixpanic
nixpanic previously approved these changes Mar 27, 2026
View reviewed changes
@nixpanic
Copy link
Copy Markdown
Member

nixpanic commented Mar 27, 2026

@Rakshith-R , please have a look at this and #6200

Rakshith-R
Rakshith-R reviewed Apr 8, 2026
View reviewed changes
Comment thread internal/rbd/snap_diff.go Outdated
Rakshith-R
Rakshith-R reviewed Apr 8, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@Rakshith-R Rakshith-R left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a small nit, LGTM

@mergify mergify Bot dismissed nixpanic’s stale review April 9, 2026 15:11

Pull request has been modified.

@kaovilai kaovilai requested a review from Rakshith-R April 9, 2026 15:46
@Rakshith-R
Copy link
Copy Markdown
Contributor

Rakshith-R commented Apr 14, 2026

@kaovilai Can you please squash into single commit ?
That will fix commitlint failure too

@kaovilai kaovilai force-pushed the rbd-add-debug-logging-ProcessMetadata branch from e58fbf1 to 37cb3c9 Compare April 28, 2026 07:08
@kaovilai
Copy link
Copy Markdown
Contributor Author

kaovilai commented Apr 28, 2026

fixed

@nixpanic
Copy link
Copy Markdown
Member

nixpanic commented Apr 28, 2026

@Mergifyio rebase

@nixpanic nixpanic requested a review from a team April 28, 2026 16:13
@kaovilai @Rakshith-R @ceph-csi-bot
rbd: add debug logging to ProcessMetadata
729342d 
Add debug log statements to ProcessMetadata to log key parameters
at each stage: snapshot ID and volume size after resolution,
DiffIterateByID config (Offset, Length, FromSnapID) before the
call, and remaining buffered block count after completion.

These logs are useful for diagnosing CBT (Changed Block Tracking)
issues in production where GetMetadataAllocated may return
unexpected results without visibility into the internal state.

Fixes: ceph#6206
Co-authored-by: Rakshith R <rakshith.r.0001@gmail.com>
Signed-off-by: Tiger Kaovilai <passawit.kaovilai@gmail.com>
@ceph-csi-bot ceph-csi-bot force-pushed the rbd-add-debug-logging-ProcessMetadata branch from 37cb3c9 to 729342d Compare April 28, 2026 16:13
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 28, 2026

Deprecation notice: This pull request comes from a fork and was rebased using bot_account impersonation. This capability will be removed on July 1, 2026. After this date, the rebase action will no longer be able to rebase fork pull requests with this configuration. Please switch to the update action/command to ensure compatibility going forward.

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 28, 2026

rebase

✅ Branch has been successfully rebased

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@nixpanic nixpanic nixpanic approved these changes

@Rakshith-R Rakshith-R Awaiting requested review from Rakshith-R

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

bug Something isn't working component/rbd Issues related to RBD

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

rbd: getRBDSnapID and ProcessMetadata issues in snapshot metadata path

4 participants

@kaovilai @nixpanic @Rakshith-R