Skip to content

Commit d7f43f1

Browse files
mr: add package-level documentation (#167714)
mr: add package-level documentation
2 parents cd06503 + 46afb01 commit d7f43f1

6 files changed

Lines changed: 413 additions & 4 deletions

File tree

pkg/sql/catalog/multiregion/BUILD.bazel

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
33
go_library(
44
name = "multiregion",
55
srcs = [
6+
"doc.go",
67
"region_config.go",
78
"region_util.go",
89
"telemetry.go",

pkg/sql/catalog/multiregion/doc.go

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
// Copyright 2026 The Cockroach Authors.
2+
//
3+
// Use of this software is governed by the CockroachDB Software License
4+
// included in the /LICENSE file.
5+
6+
/*
7+
Package multiregion provides the core abstractions for CockroachDB's
8+
multi-region SQL layer. It defines the read-only RegionConfig struct that
9+
represents a database's user-configured multi-region state and the utilities
10+
that consume it.
11+
12+
# Multi-Region Architecture
13+
14+
CockroachDB's multi-region implementation spans several packages. This package
15+
sits at the center, providing the metadata layer that other packages depend on.
16+
The key packages and their roles:
17+
18+
- pkg/sql/catalog/multiregion (this package) — Defines RegionConfig, the
19+
central read-only abstraction that captures a database's multi-region state:
20+
regions, primary and secondary regions, survival goal, placement policy,
21+
super regions, and zone config extensions. All downstream operations
22+
consume a RegionConfig rather than reading descriptors directly.
23+
24+
- pkg/sql/regions — Zone config synthesis engine. Given a RegionConfig and
25+
a table's locality, it deterministically produces the zone configuration
26+
that enforces the desired replica placement. This includes voter and
27+
non-voter constraint generation, lease preference synthesis, and
28+
super-region-aware constraint scoping.
29+
30+
- pkg/sql/regionliveness — Region availability probing. Uses the
31+
system.region_liveness table to detect and quarantine unavailable regions.
32+
This is consulted before DDL operations to prevent changes that reference
33+
unreachable regions.
34+
35+
- pkg/ccl/multiregionccl — Contains the multi-region initialization.
36+
Constructs the initial RegionConfig when a database first becomes
37+
multi-region (CREATE DATABASE ... PRIMARY REGION or ALTER DATABASE ...
38+
SET PRIMARY REGION). This package also determines the sorted insertion
39+
position for new region enum values.
40+
41+
# Data Flow
42+
43+
The typical flow when multi-region state changes is:
44+
45+
1. The database descriptor and the multi-region enum type descriptor store
46+
the persistent multi-region state. Regions are represented as enum values
47+
in a special enum type descriptor associated with the database. This
48+
enum-based representation enables transactional region addition and
49+
removal: a region being added has its enum member Direction set to ADD;
50+
once the transition completes, the Direction becomes NONE and the
51+
member becomes fully public. A region being removed transitions through
52+
the REMOVE direction before the member is dropped entirely.
53+
54+
2. SynthesizeRegionConfig reads those descriptors and produces a RegionConfig.
55+
This is an immutable snapshot of the database's multi-region state at that
56+
point in time. For DDL operations, the config is synthesized fresh (not
57+
cached) so it reflects in-flight changes. For DML operations, a cached
58+
version may be used.
59+
60+
3. The zone config synthesis layer (pkg/sql/regions) consumes the RegionConfig
61+
along with each table's locality to produce zone configurations. These zone
62+
configs specify voter constraints, non-voter constraints, replica counts,
63+
and lease preferences that enforce the multi-region semantics.
64+
65+
4. Zone configs are applied to the database and all affected tables, either
66+
directly during DDL execution or via the databaseRegionChangeFinalizer
67+
that runs after region enum value transitions complete.
68+
69+
# RegionConfig
70+
71+
Key fields of RegionConfig:
72+
73+
- regions: The set of PUBLIC regions in the database. These are the regions
74+
currently available for replica placement.
75+
- transitioningRegions: Regions being added or removed (in-flight enum
76+
transitions). These are tracked separately so that DDL validation can
77+
account for them.
78+
- addingRegions: The subset of transitioningRegions that are specifically
79+
being added (not removed). Exposed via the AddingRegions() method.
80+
- regionEnumID: The ID of the multi-region enum type descriptor that
81+
this RegionConfig was synthesized from. Links the config back to its
82+
source descriptor and is used during validation.
83+
- primaryRegion: The database's primary region. Tables without an explicit
84+
home region default to this region for leaseholder placement.
85+
- secondaryRegion: An optional failover region. If the primary region becomes
86+
unavailable, leaseholders move to the secondary region rather than to an
87+
arbitrary region.
88+
- survivalGoal: Either SURVIVE ZONE FAILURE (default) or SURVIVE REGION
89+
FAILURE. This controls how many voters and replicas are created and how
90+
they are distributed.
91+
- placement: Either DEFAULT or RESTRICTED. RESTRICTED suppresses non-voting
92+
replicas outside of the home region or super region, reducing replication
93+
cost at the expense of cross-region read performance.
94+
- superRegions: Named groupings of regions that define domiciling boundaries.
95+
See the Super Regions section below.
96+
- zoneCfgExtensions: Per-locality zone config overrides that let users
97+
customize zone config properties at different granularities.
98+
99+
# Table Localities
100+
101+
Multi-region databases support three table locality types, each of which
102+
produces different zone configurations:
103+
104+
GLOBAL tables replicate data to all database regions with non-blocking
105+
transactions (GlobalReads = true). Reads are fast everywhere because every
106+
region has a local replica. Writes pay the cost of writing to all regions.
107+
GLOBAL tables are not constrained by super regions — they intentionally
108+
replicate everywhere.
109+
110+
REGIONAL BY TABLE tables have an affinity to a single region (either the
111+
database primary region or an explicitly specified region). Leaseholders are
112+
placed in the affinity region. Voters are placed according to the survival
113+
goal: for zone survival, all voters go in the home region; for region
114+
survival, voters are distributed across regions to tolerate a region loss.
115+
116+
REGIONAL BY ROW tables are partitioned by a crdb_region column. Each
117+
partition behaves like a REGIONAL BY TABLE with affinity to the partition's
118+
region. This enables data domiciling: different rows in the same table can
119+
reside in different regions, and when super regions are configured, each
120+
row's replicas are confined to the super region that contains its region.
121+
122+
# Super Regions
123+
124+
A super region is a named grouping of database regions that defines a
125+
domiciling boundary. When data is homed in a region that belongs to a super
126+
region, all replicas — both voters and non-voters — are confined to the
127+
member regions of that super region. This is the foundation for data
128+
domiciling use cases where regulatory requirements mandate that data stays
129+
within a geographic or legal boundary.
130+
131+
Super regions are orthogonal to the survival goal. A super region with zone
132+
survivability confines replicas to its member regions and tolerates zone
133+
loss within those regions. A super region with region survivability also
134+
confines replicas but tolerates the loss of an entire member region. In
135+
both cases, no replica is placed outside the super region.
136+
137+
There are two kinds of super regions:
138+
139+
Explicit super regions are created by the user via ALTER DATABASE ... ADD
140+
SUPER REGION. They are stored in the multi-region enum type descriptor and
141+
appear in RegionConfig.SuperRegions(). A region may belong to at most one
142+
explicit super region.
143+
144+
Implicit super regions arise when a database has exactly three regions and
145+
uses SURVIVE REGION FAILURE. In this case, the system treats all three
146+
regions as a single implicit super region so that tables receive explicit
147+
per-region replica constraints (e.g. a 2+2+1 voter distribution) rather
148+
than inheriting the looser database-level zone config. This produces more
149+
deterministic replica placement across the three regions.
150+
HasImplicitSuperRegion returns true in this case.
151+
152+
RegionConfig provides methods to query super region membership, retrieve
153+
the member regions of a containing super region, and determine the
154+
effective survival goal for a given region (the super region's own goal
155+
if set, otherwise the database-level goal).
156+
157+
Super regions interact with table localities as follows:
158+
159+
- REGIONAL BY ROW: Each row's replicas are confined to the super region
160+
containing the row's crdb_region value. Different rows may be in
161+
different super regions within the same table.
162+
- REGIONAL BY TABLE: Replicas are confined to the super region containing
163+
the table's home region.
164+
- GLOBAL: Unaffected by super regions. GLOBAL tables replicate everywhere.
165+
166+
# Zone Config Extensions
167+
168+
Zone config extensions allow users to customize zone config properties at
169+
different levels of the multi-region hierarchy. Extensions are applied in
170+
a fixed precedence order during zone config synthesis:
171+
172+
1. Regional — Applies to all REGIONAL tables and partitions.
173+
2. Super Region — Applies to tables/partitions homed in a specific super
174+
region. Overrides the regional extension.
175+
3. Regional In — Applies to tables/partitions homed in a specific region.
176+
Overrides both the regional and super region extensions.
177+
178+
For GLOBAL tables, only the global extension applies.
179+
180+
Extension inheritance works by having the extension "fill in" any fields
181+
not explicitly set from the base zone config, then replacing the base
182+
with the result. This means an extension only overrides the fields it
183+
explicitly sets.
184+
185+
The ExtendZoneConfigWithRegionalIn method applies all three regional tiers
186+
in order. The ExtendZoneConfigWithGlobal method applies the global
187+
extension. Both methods validate that the resulting zone config does not
188+
violate survival goal requirements (e.g., num_voters cannot be reduced
189+
below the minimum required for the survival goal).
190+
191+
# Validation
192+
193+
RegionConfig provides several validation functions:
194+
195+
- ValidateRegionConfig: Validates overall consistency — region enum ID is
196+
set, at least one region exists, placement and survival goal are
197+
compatible, super regions are valid, and zone config extensions reference
198+
valid regions.
199+
- ValidateSuperRegions: Validates that super region names are unique and
200+
sorted, regions within each super region are unique, sorted, and belong
201+
to the database, no region appears in multiple super regions, and each
202+
super region has enough regions for its effective survival goal.
203+
- CanSatisfySurvivalGoal: Checks whether the given number of regions is
204+
sufficient for the survival goal (REGION_FAILURE requires at least 3).
205+
- CanDropRegion: Checks whether dropping a region would violate the
206+
survival goal or break a super region membership constraint.
207+
*/
208+
package multiregion

pkg/sql/catalog/multiregion/region_config.go

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,6 @@
33
// Use of this software is governed by the CockroachDB Software License
44
// included in the /LICENSE file.
55

6-
// Package multiregion provides functions and structs for interacting with the
7-
// static multi-region state configured by users on their databases.
86
package multiregion
97

108
import (

pkg/sql/regions/BUILD.bazel

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ go_library(
55
srcs = [
66
"cached_db_regions.go",
77
"db_regions.go",
8+
"doc.go",
89
"region_provider.go",
910
"region_util.go",
1011
],

0 commit comments

Comments
 (0)