Skip to content

Commit e2b5a30

Browse files
committed
docs(administration): add database management pages
Add seven new pages covering the database management subsystem: databases (lifecycle, DDL, backup), database-clone (CoW semantics, materialization, lineage), database-mirror (async/sync replication, promotion), move-tenant (offline relocation procedure), quotas (per-database and per-tenant resource limits), session-management (idle timeout, KILL SESSION, revocation), and oidc-sso (OIDC provider setup, claim mapping, token refresh). Register all new pages under a "Databases" group and expand the "Security" group with oidc-sso and session-management in oxidoc.toml.
1 parent ed35d09 commit e2b5a30

8 files changed

Lines changed: 1592 additions & 0 deletions

File tree

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
---
2+
title: Cloning Databases
3+
description: Copy-on-write CLONE DATABASE for branching, point-in-time staging, and forensic snapshots.
4+
---
5+
6+
# Cloning Databases
7+
8+
A **clone** is a copy-on-write (CoW) database created at a point-in-time snapshot of a source. Reads delegate to the source; writes go to the clone. Returns in milliseconds regardless of source size.
9+
10+
## What is a Clone?
11+
12+
When you clone a database, NodeDB:
13+
14+
1. Records the source database and a point-in-time LSN (Log Sequence Number)
15+
2. Creates a new database with the same collections
16+
3. Copies only the catalog metadata — zero storage copying
17+
4. Routes reads to the source for data not yet written in the clone
18+
5. Copies rows to the clone on first write (copy-on-write)
19+
20+
This means a clone of a 100 GB database returns in milliseconds and initially uses negligible storage.
21+
22+
## Creating a Clone
23+
24+
Clone at the latest commit:
25+
26+
```sql
27+
CLONE DATABASE staging FROM prod;
28+
```
29+
30+
Clone at a specific point in time:
31+
32+
```sql
33+
CLONE DATABASE prod_yesterday FROM prod AS OF SYSTEM TIME 1730000000000;
34+
```
35+
36+
The timestamp is in milliseconds since epoch. NodeDB resolves it to the nearest LSN and captures that snapshot. Reads on `prod_yesterday` see `prod`'s state at that moment.
37+
38+
## Read-Path Delegation
39+
40+
Until a row is written in the clone, reads are served from the source at the chosen LSN:
41+
42+
```
43+
1. Query the clone for a row
44+
2. If the row exists in the clone → return it
45+
3. If the row was explicitly deleted in the clone → return not-found
46+
4. If the row exists only in the source → return it (delegated read)
47+
5. If the row doesn't exist anywhere → return not-found
48+
```
49+
50+
This delegation is transparent — your query is unaware it is reading from the source.
51+
52+
## Write-on-Clone (Copy-Up)
53+
54+
When you first modify a row that exists only in the source:
55+
56+
```sql
57+
UPDATE staging.users SET status = 'active' WHERE id = 100;
58+
```
59+
60+
If row 100 exists only in the source:
61+
62+
1. The row is **copied up** from the source to the clone
63+
2. The UPDATE is applied
64+
3. The copy is durably recorded
65+
4. Subsequent reads see the clone's copy
66+
67+
Delete on a source-only row inserts a **tombstone** — the row is invisible to subsequent reads without being physically removed.
68+
69+
All writes are durable through the WAL and Raft replication.
70+
71+
## Bitemporal Correctness
72+
73+
If the source is [bitemporal](bitemporal), the clone preserves time-travel semantics:
74+
75+
```sql
76+
SELECT * FROM staging.events AS OF SYSTEM TIME 1729000000000 WHERE event_id = 42;
77+
```
78+
79+
- If query time ≤ clone's LSN → read from source at that time
80+
- If query time > clone's LSN → read from clone (clone did not exist before its creation)
81+
- If query time < clone creation → empty result with metadata note
82+
83+
This is why cloning from a point-in-time staging database works: you see the exact historical state at clone time, then your own edits afterward.
84+
85+
## Materializing a Clone
86+
87+
Background materialization gradually copies all rows from source to clone, freeing the clone from source dependency:
88+
89+
```sql
90+
ALTER DATABASE staging MATERIALIZE;
91+
```
92+
93+
This blocks until all rows are copied. Useful when you want to:
94+
95+
- Drop the source database
96+
- Stop relying on source's read-path performance
97+
- Create an independent snapshot for long-term archival
98+
99+
Before materializing completes, the clone remains usable — reads continue delegating if needed.
100+
101+
## Viewing Lineage
102+
103+
See the ancestor chain of a clone:
104+
105+
```sql
106+
SHOW DATABASE LINEAGE FOR staging;
107+
```
108+
109+
Returns:
110+
111+
```
112+
source_database | created_at_ms | as_of_ms | status
113+
-----------------+---------------+----------+----------
114+
prod | 1730000000000 | null | Shadowed
115+
```
116+
117+
For nested clones (clone of a clone):
118+
119+
```
120+
source_database | created_at_ms | as_of_ms | status
121+
-----------------+---------------+----------+----------
122+
prod | 1730000000000 | null | Shadowed
123+
staging | 1730086400000 | null | Shadowed
124+
```
125+
126+
## Clone Depth Limits
127+
128+
Clone depth is limited to 8 levels to prevent query-path explosion. Attempting to clone from a clone-8 returns `CLONE_DEPTH_EXCEEDED`.
129+
130+
To exceed the limit, materialize the source clone first:
131+
132+
```sql
133+
ALTER DATABASE staging MATERIALIZE;
134+
CLONE DATABASE stage2 FROM staging; -- now staging is Materialized, so this succeeds
135+
```
136+
137+
## Restrictions and Errors
138+
139+
| Restriction | Solution |
140+
| ----------------------- | ------------------------------------ |
141+
| Cannot clone a mirror | Promote the mirror first; then clone |
142+
| Clone depth > 8 | Materialize source; retry |
143+
| Source dropped with dependents | Use `DROP DATABASE … FORCE` |
144+
145+
## Practical Examples
146+
147+
### Point-in-Time Staging
148+
149+
Clone production at 1 AM daily for staging/QA tests:
150+
151+
```sql
152+
-- 1 AM: Create daily snapshot
153+
CLONE DATABASE qa_daily_$(date +%Y%m%d) FROM prod AS OF SYSTEM TIME (EXTRACT(EPOCH FROM '2026-05-10 01:00:00'::timestamp) * 1000)::bigint;
154+
155+
-- QA team tests against the snapshot
156+
-- After testing: DROP or materialize for archival
157+
```
158+
159+
### Forensic Snapshots
160+
161+
Clone at the time of a suspected incident:
162+
163+
```sql
164+
-- Incident at 1730000000000 (ms)
165+
CLONE DATABASE incident_snapshot FROM prod AS OF SYSTEM TIME 1730000000000;
166+
167+
-- Investigate without modifying production
168+
SELECT * FROM incident_snapshot.events WHERE status = 'failed';
169+
```
170+
171+
### Blue-Green Deployment
172+
173+
Create a staging clone, apply schema changes, validate, then promote:
174+
175+
```sql
176+
CLONE DATABASE blue FROM prod;
177+
ALTER TABLE blue.users ADD COLUMN new_field INT;
178+
-- Run tests on blue
179+
-- Once validated, swap DNS / connection strings
180+
```
181+
182+
## Composition with Other Features
183+
184+
**Quotas:** A clone inherits the source's quota settings. Adjust independently:
185+
186+
```sql
187+
ALTER DATABASE staging SET QUOTA quota_storage_bytes = 53687091200;
188+
```
189+
190+
**Mirroring:** A clone can be mirrored (preview-environment DR). A mirror cannot be cloned — promote first.
191+
192+
**Multi-engine:** Clones work across all engines (vector, graph, columnar, etc.) transparently.
193+
194+
## Performance Notes
195+
196+
- **Clone creation**: O(catalog size), typically < 100 ms even for TB sources
197+
- **Materialization**: Background, respects database maintenance budget. Small rows (< 1 KB) typically materialize at 100K–500K rows/sec
198+
- **Read overhead while Shadowed**: Negligible — clone check is a fast hashtable lookup
199+
- **Write overhead while Shadowed**: Single copy-up latency per source row; subsequent writes unaffected

0 commit comments

Comments
 (0)