Skip to content

log the direct source IP and node_id of NewBlockHashes of new PoW blocks - #3583

Open
Hecate2 wants to merge 4 commits into
Conflux-Chain:masterfrom
Hecate2:block-source-logger
Open

log the direct source IP and node_id of NewBlockHashes of new PoW blocks#3583
Hecate2 wants to merge 4 commits into
Conflux-Chain:masterfrom
Hecate2:block-source-logger

Conversation

@Hecate2

@Hecate2 Hecate2 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Add the ability to log the direct source IP and NodeId for each received NewBlockHashes message. This feature is controlled by the log_block_source config flag (default: false). To track PoW block propagation patterns.

[BLOCK_SOURCE] hash=0x... from_node=0x... from_addr=1.2.3.4:32323

Changes:

  • Add get_peer_addr() method to NetworkContext trait
  • Implement get_peer_addr() in network service to look up session address
  • Add peer_addr field to sync message Context struct
  • Populate peer_addr when dispatching messages
  • Log [BLOCK_SOURCE] with block hash, source NodeId and IP when enabled
  • Add log_block_source config option in ProtocolConfiguration and configuration.rs

This change is Reviewable

Add the ability to log the direct source IP and NodeId for each received
NewBlockHashes message. This feature is controlled by the log_block_source
config flag (default: false) and is designed for deployment on public
bootnodes to track block propagation patterns.

Changes:
- Add get_peer_addr() method to NetworkContext trait
- Implement get_peer_addr() in network service to look up session address
- Add peer_addr field to sync message Context struct
- Populate peer_addr when dispatching messages
- Log [BLOCK_SOURCE] with block hash, source NodeId and IP when enabled
- Add log_block_source config option in ProtocolConfiguration and configuration.rs
.map(|s| s.as_str())
.unwrap_or("unknown");
for hash in &self.block_hashes {
info!(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Per-hash info!() logging may produce very high log volume on a busy bootnode

A single NewBlockHashes message can carry many hashes, and a bootnode with many connected peers can receive a large number of these messages per second. When log_block_source is true, this loop emits one info! line per hash per message, which could produce tens of thousands of log lines per second under load — potentially causing I/O saturation or log-file bloat.

Consider logging a single aggregated line per message instead, e.g.:

Suggested change
info!(
info!(

Alternatively, restructure to log once per message rather than once per hash:

let hashes_str: Vec<String> = self.block_hashes.iter()
    .map(|h| format!("{:#x}", h))
    .collect();
info!(
    "[BLOCK_SOURCE] hashes=[{}] from_node={} from_addr={}",
    hashes_str.join(","), ctx.node_id, peer_addr
);

This reduces the log volume proportionally to the average number of hashes per message.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@kilo-code-bot

kilo-code-bot Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Incremental Review Notes

This review covers changes since commit 49f0ab7b3acf373e3114e2322947c55f5575c78c.

The latest commit ("use simple impl") removes the #[cfg(test)] module that was added earlier in this PR:

  • Test module deleted: Four RLP round-trip and decode-rejection unit tests for NewBlockHashes are removed. These were added as part of this PR and are now being reverted along with the rest of the log_block_source complexity.
  • No functional changes: No production code was modified in this increment. The handle logic, all struct fields, and all config options remain as reviewed in the previous pass.
  • Existing inline comment: The previous WARNING comment on line 44 (info!() log volume) has line: null — it is outdated; the code it targeted no longer exists in the current diff and was already marked resolved in the prior review.
Files Reviewed (1 file)
  • crates/cfxcore/core/src/sync/message/new_block_hashes.rs - no issues (test module deletion only)
Previous Review Summaries (4 snapshots, latest commit 49f0ab7)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 49f0ab7)

Status: No Issues Found | Recommendation: Merge

Incremental Review Notes

This review covers changes since commit 9cd735f9b60e1436863b1ff654a1425d937b3c73.

The latest commit ("use simple impl") reverts the log_block_source feature entirely:

  • log_block_source removed: The [BLOCK_SOURCE] info!() logging, the log_block_source config option, and the ProtocolConfiguration.log_block_source field are all gone. The previously flagged WARNING (per-hash info!() log volume) is now resolved.
  • Simplified handle logic: The unknown_hashes Vec and lazy-computation optimization are replaced with a straightforward inline .filter() chain. Equivalent correctness, less complexity.
  • debug!() now includes from_node/from_addr: The original debug!("on_new_block_hashes, msg={:?}", self) is extended to include peer identity at DEBUG level. Since debug!() is a no-op in release builds, this adds no production overhead.
  • peer_addr context field retained: Still populated on message dispatch (io.get_peer_addr(peer) acquires a session read lock), but now only consumed by this one debug!() call. This is pre-existing (added in commit 1, not changed in this commit) and outside the incremental review scope.
Files Reviewed (3 files)
  • crates/cfxcore/core/src/sync/message/new_block_hashes.rs - no issues
  • crates/cfxcore/core/src/sync/synchronization_protocol_handler.rs - no issues
  • crates/config/src/configuration.rs - no issues

Previous review (commit 9cd735f)

Status: No Issues Found | Recommendation: Merge

Incremental Review Notes

This review covers changes since commit 7388f07fd1d17dfc6c2785054623524c540e0d01.

This commit is a clean performance-focused refactor of new_block_hashes.rs:

  • HashSet<H256>Vec<&H256>: For the typical 1–2 hash payload, linear scan is cheaper than HashSet allocation and hashing. The comment accurately documents this trade-off.
  • Lazy computation: unknown_hashes is only populated when log_block_source || !in_catch_up_mode, skipping all block_header_by_hash DB lookups in catch-up mode with logging off. This restores pre-logging performance in that path.
  • catch_up_mode() caching: Result stored in in_catch_up_mode and reused — avoids double evaluation.
  • Borrow safety: unknown_hashes: Vec<&H256> holds immutable references into self.block_hashes; self.block_hashes.iter() on line 107 is also immutable — no borrow conflict.
  • headers_to_request construction: unknown_hashes.into_iter().cloned().collect() correctly yields Vec<H256> from Vec<&H256>.

The previously flagged WARNING (per-hash info!() log volume) remains intentional behavior — logging fires only for unknown hashes with log_block_source explicitly opt-in (default: false).

Files Reviewed (1 file)
  • crates/cfxcore/core/src/sync/message/new_block_hashes.rs - no issues

Previous review (commit 7388f07)

Status: No Issues Found | Recommendation: Merge

Incremental Review Notes

This review covers changes since commit 66f46651e97ca7730252c6ae67397ad020e7d189.

The previous WARNING (per-hash info!() logging potentially causing high log volume on a busy bootnode) has been partially mitigated in this commit:

  • known_hashes is now pre-computed from block_header_by_hash lookups
  • The logging loop now guards with if !known_hashes.contains(hash), suppressing log entries for hashes whose headers we already have
  • This prevents repeated logging for already-downloaded blocks across subsequent NewBlockHashes messages from different peers

Per-hash logging for new/unknown hashes remains intentional behavior (first-seen propagation tracking). The duplicate ratio is bounded by the short window before header download, as documented in the new code comments.

No new issues were found in the incremental diff.

Files Reviewed (1 file)
  • crates/cfxcore/core/src/sync/message/new_block_hashes.rs - no issues

Previous review (commit 66f4665)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
crates/cfxcore/core/src/sync/message/new_block_hashes.rs 44 Per-hash info!() logging may produce very high log volume on a busy bootnode
Files Reviewed (7 files)
  • crates/cfxcore/core/src/sync/message/handleable.rs - no issues
  • crates/cfxcore/core/src/sync/message/new_block_hashes.rs - 1 issue
  • crates/cfxcore/core/src/sync/state/snapshot_chunk_sync.rs - no issues
  • crates/cfxcore/core/src/sync/synchronization_protocol_handler.rs - no issues
  • crates/config/src/configuration.rs - no issues
  • crates/network/src/lib.rs - no issues
  • crates/network/src/service.rs - no issues

Fix these issues in Kilo Cloud


Reviewed by claude-sonnet-4.6 · Input: 9 · Output: 2.2K · Cached: 127.1K

Hecate2 added 2 commits July 29, 2026 15:14
Replace the custom BlockSourceTracker with approximate first-seen
deduplication that reuses the existing block_header_by_hash lookup.
Only block hashes whose headers are not yet downloaded generate a
[BLOCK_SOURCE] log entry. This avoids adding any new data structure,
consuming zero additional memory and zero extra disk I/O.

Trade-offs documented in new_block_hashes.rs:
- Not a strict first-seen guarantee: multiple peers propagating the
  same hash within the header-download window (ms to seconds) each
  generate a log entry. In practice this window is small.
- After the header is cached, all subsequent NewBlockHashes for the
  same block are silently suppressed, which is the desired behavior.

Changes:
- Remove BlockSourceTracker struct and its unit tests
- Remove block_source_tracker field from SynchronizationProtocolHandler
- Remove log_block_source_capacity config option
- Reuse block_header_by_hash filter for dedup in NewBlockHashes::handle
- Unify the header existence check so logging and request filtering
  share a single lookup
@Hecate2

Hecate2 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Changed to reuse the original on_new_block_hashes debug log

@Hecate2
Hecate2 force-pushed the block-source-logger branch from 49f0ab7 to 758bb18 Compare July 31, 2026 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant