Skip to content

Commit efd2b64

Browse files
committed
Use deterministic port allocation in integration tests
Integration tests allocate listening ports for test nodes by binding to 127.0.0.1:0, reading the OS-assigned port, then immediately dropping the socket. Later, Node::start() tries to bind to that same port. This has two problems: Parallel test collisions: with 35 tests running concurrently, each creating 2-4 nodes with 2 ports each, the OS can reassign a freed port to another test's node before the original node calls start(). This is a classic TOCTOU race. In CI, it caused ~50% of Rust test runs to fail with InvalidSocketAddress, always exactly one random test per run. Restart instability: when a test stops and restarts a node, the port is released during stop() and must be re-acquired during start(). Another test's node can grab it in between. The peer store still has the old address, so auto-reconnection also fails. Both problems are inherent to any scheme that allocates a port and later releases it. The only way to guarantee a port stays yours is to never release it, or to never share the port space. A deterministic atomic counter starting at a fixed base port (20000) solves both problems: each fetch_add returns a unique value, so no two nodes in the same process ever get the same port, and ports are stable across restarts because the same node keeps the same config. There is no external contention because CI runners are isolated. AI tools were used in preparing this commit.
1 parent a555133 commit efd2b64

File tree

1 file changed

+8
-10
lines changed

1 file changed

+8
-10
lines changed

tests/common/mod.rs

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ use std::collections::{HashMap, HashSet};
1414
use std::env;
1515
use std::future::Future;
1616
use std::path::PathBuf;
17+
use std::sync::atomic::{AtomicU16, Ordering};
1718
use std::sync::{Arc, RwLock};
1819
use std::time::Duration;
1920

@@ -268,17 +269,14 @@ pub(crate) fn random_storage_path() -> PathBuf {
268269
temp_path
269270
}
270271

271-
pub(crate) fn random_listening_addresses() -> Vec<SocketAddress> {
272-
let num_addresses = 2;
273-
let mut listening_addresses = HashSet::new();
274-
275-
while listening_addresses.len() < num_addresses {
276-
let socket = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
277-
let address: SocketAddress = socket.local_addr().unwrap().into();
278-
listening_addresses.insert(address);
279-
}
272+
static NEXT_PORT: AtomicU16 = AtomicU16::new(20000);
280273

281-
listening_addresses.into_iter().collect()
274+
pub(crate) fn random_listening_addresses() -> Vec<SocketAddress> {
275+
let port = NEXT_PORT.fetch_add(2, Ordering::Relaxed);
276+
vec![
277+
SocketAddress::TcpIpV4 { addr: [127, 0, 0, 1], port },
278+
SocketAddress::TcpIpV4 { addr: [127, 0, 0, 1], port: port + 1 },
279+
]
282280
}
283281

284282
pub(crate) fn random_node_alias() -> Option<NodeAlias> {

0 commit comments

Comments
 (0)