Skip to content

Commit 3588dae

Browse files
committed
feat(security): enforce per-database idle timeout and typed kill reasons
Introduces KillReason, an idle-session sweep loop, per-database idle timeout configuration, and richer SHOW SESSIONS output. KillReason enum replaces the bool kill signal on the session watch channel, allowing the session drop path to surface a precise error (IdleTimeout, TokenExpired, AdminKill, UserDropped) rather than a generic "killed" state. IdleTimeoutCache caches the idle_session_timeout_secs per database in memory so the sweep loop avoids catalog round-trips. Populated at startup from the catalog and updated synchronously by the ALTER DATABASE SET IDLE_TIMEOUT handler. ALTER DATABASE SET IDLE_TIMEOUT = <secs> (0 = disable) is gated on ClusterAdmin/Superuser, writes the updated descriptor to the catalog, updates the live IdleTimeoutCache, and emits a DatabaseIdleTimeoutChanged audit event. idle_sweep::sweep_once() runs on a 5-second Tokio tick, snapshots sessions via sweep_snapshot(), computes the earlier of the per-database idle deadline and the OIDC token expiry deadline, and signals the appropriate KillReason. The sweep loop never calls unregister(); only the session's own drop path does. SessionRegistry gains: - current_database, idle_timeout_secs, token_expiry_ms, bytes_in, bytes_out, current_statement_digest fields on RegisteredSession. - kill_session_by_id() for targeted one-shot kill with a KillReason. - lookup_session_database() for pre-kill authority checks. - sweep_snapshot() producing SweepEntry rows consumed by the sweep loop. SHOW SESSIONS gains idle_seconds, current_database, bytes_in, bytes_out, current_statement, and token_expires_in_secs columns. KILL SESSION is extended to allow ClusterAdmin and DatabaseOwner of the session's database to kill sessions (not just Superuser). A race between authority check and kill is surfaced as a distinct audit row. kill_sessions_for_user() and kill_sessions_for_username() now accept a KillReason parameter so callers (bus consumer, blacklist DDL, emergency DDL) can declare intent rather than relying on the sentinel bool.
1 parent 311db2e commit 3588dae

16 files changed

Lines changed: 725 additions & 50 deletions

File tree

nodedb/src/bootstrap/background_loops.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,11 @@ pub fn spawn_background_loops(
198198
},
199199
);
200200

201+
// Idle session sweep (5-second timer).
202+
// Closes sessions whose idle timeout or OIDC token expiry has elapsed.
203+
crate::control::security::sessions::spawn_idle_sweep_loop(shared);
204+
info!("idle session sweep loop running");
205+
201206
// Audit log flush (10-second timer).
202207
let shared_audit = Arc::clone(shared);
203208
crate::control::shutdown::spawn_loop(

nodedb/src/control/mirror/restart.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,7 @@ mod tests {
132132
status,
133133
}),
134134
audit_dml: nodedb_types::AuditDmlMode::None,
135+
idle_session_timeout_secs: 0,
135136
}
136137
}
137138

nodedb/src/control/security/audit/event.rs

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,11 @@ pub enum AuditEvent {
133133
/// The AUDIT_DML mode for a database was changed via
134134
/// `ALTER DATABASE SET AUDIT_DML`.
135135
DatabaseAuditDmlChanged = 44,
136+
/// The idle session timeout for a database was changed via
137+
/// `ALTER DATABASE SET IDLE_TIMEOUT`.
138+
DatabaseIdleTimeoutChanged = 45,
139+
/// An OIDC provider was created, altered, or dropped.
140+
OidcProviderChanged = 46,
136141
}
137142

138143
impl AuditEvent {
@@ -187,6 +192,8 @@ impl AuditEvent {
187192
Self::DatabaseRestored => 42,
188193
Self::DmlAudit => 43,
189194
Self::DatabaseAuditDmlChanged => 44,
195+
Self::DatabaseIdleTimeoutChanged => 45,
196+
Self::OidcProviderChanged => 46,
190197
}
191198
}
192199

@@ -237,7 +244,9 @@ impl AuditEvent {
237244
| Self::DatabaseBackedUp
238245
| Self::DatabaseRestored
239246
| Self::DmlAudit
240-
| Self::DatabaseAuditDmlChanged => false,
247+
| Self::DatabaseAuditDmlChanged
248+
| Self::DatabaseIdleTimeoutChanged
249+
| Self::OidcProviderChanged => false,
241250
}
242251
}
243252

@@ -291,6 +300,8 @@ impl AuditEvent {
291300
Self::DatabaseRestored => "database_restored",
292301
Self::DmlAudit => "dml_audit",
293302
Self::DatabaseAuditDmlChanged => "database_audit_dml_changed",
303+
Self::DatabaseIdleTimeoutChanged => "database_idle_timeout_changed",
304+
Self::OidcProviderChanged => "oidc_provider_changed",
294305
}
295306
}
296307

@@ -341,7 +352,9 @@ impl AuditEvent {
341352
| Self::TenantMoved
342353
| Self::DatabaseBackedUp
343354
| Self::DatabaseRestored
344-
| Self::DatabaseAuditDmlChanged => AuditLevel::Standard,
355+
| Self::DatabaseAuditDmlChanged
356+
| Self::DatabaseIdleTimeoutChanged
357+
| Self::OidcProviderChanged => AuditLevel::Standard,
345358
Self::DmlAudit => AuditLevel::Forensic,
346359
}
347360
}

nodedb/src/control/security/buses/consumer.rs

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,10 @@ fn handle_session_invalidated(
157157

158158
// Hard revoke: signal every open session for this user to close.
159159
if event.reason.is_hard_revoke() {
160-
session_registry.kill_sessions_for_user(event.user_id);
160+
session_registry.kill_sessions_for_user(
161+
event.user_id,
162+
crate::control::security::sessions::KillReason::UserDropped,
163+
);
161164
}
162165
// Soft revoke: identity rehydrate is driven by the per-user version
163166
// counter at the next request-entry boundary — no explicit signal needed.
@@ -240,6 +243,8 @@ mod tests {
240243
auth_method: "password".into(),
241244
tenant_id: 1,
242245
credential_version: 0,
246+
current_database: None,
247+
token_expiry_ms: None,
243248
};
244249
let mut kill_rx = registry.register("session-42", &params).unwrap();
245250

@@ -266,7 +271,11 @@ mod tests {
266271
kill_rx.has_changed().unwrap_or(false),
267272
"kill_rx must be signalled for hard revoke"
268273
);
269-
assert!(*kill_rx.borrow_and_update(), "kill value must be true");
274+
assert_ne!(
275+
*kill_rx.borrow_and_update(),
276+
crate::control::security::sessions::KillReason::Alive,
277+
"kill value must not be Alive"
278+
);
270279
}
271280

272281
#[tokio::test]
Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
// SPDX-License-Identifier: BUSL-1.1
2+
3+
//! Idle-session sweep loop (Control Plane, Tokio).
4+
//!
5+
//! `idle_sweep_loop` runs on the Tokio runtime at a 5-second tick. Each tick
6+
//! it calls `sweep_once`, which:
7+
//!
8+
//! 1. Takes a lock-free snapshot of every session via `sweep_snapshot`.
9+
//! 2. For each session, computes the earliest close deadline from:
10+
//! - The per-database idle timeout (from the session's cached
11+
//! `idle_timeout_secs`, which mirrors the `IdleTimeoutCache`).
12+
//! - The OIDC token expiry (`token_expiry_ms`), if set.
13+
//! - The global idle timeout stored in `SharedState::idle_timeout_secs`.
14+
//! 3. If `now_ms >= deadline`, sends `KillReason::IdleTimeout` or
15+
//! `KillReason::TokenExpired` (whichever is earlier) to that session.
16+
//!
17+
//! The session's own drop path calls `unregister` — the sweep loop only signals
18+
//! via the `kill_tx`; it never calls `unregister` itself.
19+
20+
use std::sync::Arc;
21+
use std::time::Duration;
22+
23+
use crate::control::security::sessions::{KillReason, SessionRegistry};
24+
use crate::control::security::time::now_ms;
25+
use crate::control::state::SharedState;
26+
27+
/// Compute the next close deadline (milliseconds since epoch) for a session,
28+
/// given:
29+
/// - `token_exp_ms`: OIDC token expiry in ms (0 = no token expiry).
30+
/// - `last_active_ms`: last-active timestamp in milliseconds.
31+
/// - `idle_cap_secs`: idle timeout in seconds (0 = no idle cap).
32+
///
33+
/// Returns `None` if neither deadline applies.
34+
/// Returns the earlier of the two deadlines when both apply.
35+
pub fn next_close_deadline_ms(
36+
token_exp_ms: u64,
37+
last_active_ms: u64,
38+
idle_cap_secs: u64,
39+
) -> Option<u64> {
40+
let idle_deadline = if idle_cap_secs > 0 {
41+
Some(last_active_ms.saturating_add(idle_cap_secs * 1_000))
42+
} else {
43+
None
44+
};
45+
let token_deadline = if token_exp_ms > 0 {
46+
Some(token_exp_ms)
47+
} else {
48+
None
49+
};
50+
match (idle_deadline, token_deadline) {
51+
(Some(a), Some(b)) => Some(a.min(b)),
52+
(Some(a), None) => Some(a),
53+
(None, Some(b)) => Some(b),
54+
(None, None) => None,
55+
}
56+
}
57+
58+
/// Determine the kill reason for a deadline breach, preferring
59+
/// `TokenExpired` when the token deadline is earlier than the idle deadline.
60+
fn kill_reason_for(
61+
token_exp_ms: u64,
62+
last_active_ms: u64,
63+
idle_cap_secs: u64,
64+
now_ms: u64,
65+
) -> KillReason {
66+
// Evaluate which deadline fired first.
67+
let token_fired = token_exp_ms > 0 && now_ms >= token_exp_ms;
68+
let idle_fired =
69+
idle_cap_secs > 0 && now_ms >= last_active_ms.saturating_add(idle_cap_secs * 1_000);
70+
71+
match (token_fired, idle_fired) {
72+
(true, false) => KillReason::TokenExpired,
73+
(false, true) => KillReason::IdleTimeout,
74+
(true, true) => {
75+
// Both fired: prefer the one with the earlier deadline.
76+
let token_dl = token_exp_ms;
77+
let idle_dl = last_active_ms.saturating_add(idle_cap_secs * 1_000);
78+
if token_dl <= idle_dl {
79+
KillReason::TokenExpired
80+
} else {
81+
KillReason::IdleTimeout
82+
}
83+
}
84+
(false, false) => KillReason::Alive,
85+
}
86+
}
87+
88+
/// Single sweep pass.
89+
///
90+
/// Exported for testing; production code calls this via `idle_sweep_loop`.
91+
pub fn sweep_once(session_registry: &Arc<SessionRegistry>, global_idle_secs: u64) {
92+
let now = now_ms();
93+
let entries = session_registry.sweep_snapshot();
94+
for entry in entries {
95+
// Per-session cap: the session's cached `idle_timeout_secs` (loaded from
96+
// `IdleTimeoutCache` at session start and on cache invalidation).
97+
// Falls back to the global cap when the per-database value is 0.
98+
let idle_secs = if entry.idle_timeout_secs > 0 {
99+
entry.idle_timeout_secs
100+
} else {
101+
global_idle_secs
102+
};
103+
// last_active is stored in seconds; convert to ms for deadline arithmetic.
104+
let last_active_ms = entry.last_active_secs * 1_000;
105+
let deadline = next_close_deadline_ms(entry.token_expiry_ms, last_active_ms, idle_secs);
106+
let Some(dl) = deadline else {
107+
continue;
108+
};
109+
if now >= dl {
110+
let reason = kill_reason_for(entry.token_expiry_ms, last_active_ms, idle_secs, now);
111+
if reason != KillReason::Alive {
112+
let _ = session_registry.kill_session_by_id(&entry.session_id, reason);
113+
}
114+
}
115+
}
116+
}
117+
118+
/// Spawn the idle sweep loop on the Tokio runtime.
119+
///
120+
/// Runs at a 5-second tick. Respects `SharedState::shutdown` for graceful
121+
/// termination. Registered in `loop_registry` for drain-on-shutdown.
122+
pub fn spawn_idle_sweep_loop(shared: &Arc<SharedState>) {
123+
let shared_sweep = Arc::clone(shared);
124+
crate::control::shutdown::spawn_loop(
125+
&shared.loop_registry,
126+
&shared.shutdown,
127+
"idle_session_sweep",
128+
move |mut shutdown| async move {
129+
let mut tick = tokio::time::interval(Duration::from_secs(5));
130+
loop {
131+
tokio::select! {
132+
_ = shutdown.wait_cancelled() => break,
133+
_ = tick.tick() => {}
134+
}
135+
if shutdown.is_cancelled() {
136+
break;
137+
}
138+
let global_idle = shared_sweep.idle_timeout_secs();
139+
sweep_once(&shared_sweep.session_registry, global_idle);
140+
}
141+
},
142+
);
143+
}
144+
145+
#[cfg(test)]
146+
mod tests {
147+
use super::*;
148+
149+
#[test]
150+
fn no_deadline_when_both_zero() {
151+
assert_eq!(next_close_deadline_ms(0, 1_000_000, 0), None);
152+
}
153+
154+
#[test]
155+
fn idle_deadline_only() {
156+
// last_active_ms = 1000s, cap = 300s → deadline = 1300s in ms
157+
let dl = next_close_deadline_ms(0, 1_000_000, 300);
158+
assert_eq!(dl, Some(1_300_000));
159+
}
160+
161+
#[test]
162+
fn token_deadline_only() {
163+
let dl = next_close_deadline_ms(9_999_000, 1_000_000, 0);
164+
assert_eq!(dl, Some(9_999_000));
165+
}
166+
167+
#[test]
168+
fn picks_earlier_idle_over_token() {
169+
// idle fires at 1_300_000, token at 9_999_000 → idle wins
170+
let dl = next_close_deadline_ms(9_999_000, 1_000_000, 300);
171+
assert_eq!(dl, Some(1_300_000));
172+
}
173+
174+
#[test]
175+
fn picks_earlier_token_over_idle() {
176+
// token fires at 500_000, idle at 1_300_000 → token wins
177+
let dl = next_close_deadline_ms(500_000, 1_000_000, 300);
178+
assert_eq!(dl, Some(500_000));
179+
}
180+
181+
#[test]
182+
fn kill_reason_token_when_token_earlier() {
183+
let now = 600_000u64;
184+
let reason = kill_reason_for(500_000, 1_000_000, 300, now);
185+
assert_eq!(reason, KillReason::TokenExpired);
186+
}
187+
188+
#[test]
189+
fn kill_reason_idle_when_idle_fires() {
190+
let now = 1_400_000u64;
191+
// idle deadline = 1_300_000, token at 9_999_000 → idle
192+
let reason = kill_reason_for(9_999_000, 1_000_000, 300, now);
193+
assert_eq!(reason, KillReason::IdleTimeout);
194+
}
195+
196+
#[test]
197+
fn alive_when_not_expired() {
198+
let now = 100_000u64;
199+
let reason = kill_reason_for(9_999_000, 1_000_000, 300, now);
200+
assert_eq!(reason, KillReason::Alive);
201+
}
202+
}
Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,31 @@
11
// SPDX-License-Identifier: BUSL-1.1
22

3-
//! Active session tracking (registry, cap enforcement, revocation).
3+
//! Active session tracking: registry, cap enforcement, revocation, idle sweep.
4+
//!
5+
//! ## Lifecycle
6+
//!
7+
//! `SessionRegistry` is constructed once during server startup and shared
8+
//! via `SharedState::session_registry`. Every authenticated connection
9+
//! (native, pgwire, HTTP) calls `register` on bind and `unregister` on drop.
10+
//!
11+
//! `spawn_idle_sweep_loop` MUST be called exactly once at server startup,
12+
//! after `SharedState` is fully built. It registers a Tokio task with the
13+
//! shared `loop_registry` (so it drains on shutdown) and ticks every five
14+
//! seconds, signalling `KillReason::IdleTimeout` or `KillReason::TokenExpired`
15+
//! to sessions whose per-database idle cap or OIDC token expiry has elapsed.
16+
//! Calling it more than once would spawn duplicate sweepers — there is no
17+
//! internal idempotency guard, so the call site (currently
18+
//! `nodedb::bootstrap::background_loops`) is responsible for the
19+
//! exactly-once invariant.
20+
//!
21+
//! Callers never invoke `unregister` from the sweep path; the sweep loop
22+
//! only sends on `kill_tx`, and the session's own drop path removes the
23+
//! row.
424
25+
pub mod idle_sweep;
526
pub mod registry;
627

7-
pub use registry::{SessionCapExceeded, SessionInfo, SessionParams, SessionRegistry};
28+
pub use idle_sweep::spawn_idle_sweep_loop;
29+
pub use registry::{
30+
KillReason, SessionCapExceeded, SessionInfo, SessionParams, SessionRegistry, SweepEntry,
31+
};

0 commit comments

Comments
 (0)