Skip to content

Commit 8495be9

Browse files
committed
save
1 parent 546555a commit 8495be9

9 files changed

Lines changed: 161 additions & 135 deletions

File tree

docs/configuration/pgdog.toml/rewrite.md

Lines changed: 8 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,9 @@
22
icon: material/alpha-r-box-outline
33
---
44

5-
# Rewrite
5+
# Rewrite engine
66

7-
The `rewrite` section controls PgDog's automatic SQL rewrites for sharded clusters. It affects shard-key updates and multi-row INSERT statements, and can be toggled globally or per-policy.
8-
9-
## Options
7+
The `rewrite` section controls PgDog's automatic SQL rewrites for sharded databases. It affects sharding key updates and multi-tuple inserts. Either one can be toggled separately:
108

119
```toml
1210
[rewrite]
@@ -15,58 +13,29 @@ shard_key = "error"
1513
split_inserts = "error"
1614
```
1715

18-
| Field | Description | Default |
16+
| Setting | Description | Default |
1917
| --- | --- | --- |
20-
| `enabled` | Master toggle: when `false`, PgDog parses but never applies rewrite plans. | `false` |
18+
| `enabled` | Enables/disables the query rewrite engine. | `false` |
2119
| `shard_key` | Behavior when an `UPDATE` changes a sharding key: `error` rejects the statement,<br>`rewrite` migrates the row between shards,<br>`ignore` forwards it unchanged. | `"error"` |
2220
| `split_inserts` | Behavior when a sharded table receives a multi-row `INSERT`: `error` rejects the statement, `rewrite` fans the rows out to their shards, `ignore` forwards it unchanged. | `"error"` |
2321

2422
!!! note "Two-phase commit"
25-
PgDog recommends enabling [two-phase commit](../../features/sharding/2pc.md) when either policy is set to `rewrite`. Without it, rewrites are committed shard-by-shard and can leave partial changes if a shard fails.
23+
Consider enabling [two-phase commit](../../features/sharding/2pc.md) when either feature is set to `rewrite`. Without it, rewrites are committed shard-by-shard and can leave partial changes if a transaction fails.
2624

2725
## Runtime overrides
2826

2927
The admin database exposes these toggles via the `SET` command:
3028

3129
```postgresql
32-
SET rewrite_enabled TO true; -- mirrors [rewrite].enabled
30+
SET rewrite_enabled TO true; -- enable/disable rewrite engine
3331
SET rewrite_shard_key_updates TO rewrite; -- error | rewrite | ignore
3432
SET rewrite_split_inserts TO rewrite; -- error | rewrite | ignore
3533
```
3634

3735
The setting changes are applied immediately. These overrides allow canary testing before persisting them in `pgdog.toml`.
3836

39-
## Limitations
40-
41-
### Sharding key updates
42-
43-
Sharding key rewrites in an `UPDATE` clause have to resolve to a single row. If the sharding key isn't unique or the `WHERE` clause has an incorrect `OR` condition, for example, PgDog will rollback the transaction and raise an error.
44-
45-
For example:
46-
47-
```postgresql
48-
UPDATE users SET id = 5 WHERE admin = true;
49-
```
50-
51-
On a single-shard deployment, this would raise a unique index violation error. On a cross-shard deployment, the PgDog rewrite engine will block cross-shard updates that could potentially affect multiple rows.
52-
53-
### Multi-tuple inserts
54-
55-
`INSERT` statements with multiple tuples have to be executed outside of an explicit transaction. PgDog needs to start a cross-shard transaction to safely commit the rows to multiple shards, and an existing transaction will interfere with its internal state.
56-
57-
For example:
58-
59-
```postgresql
60-
BEGIN;
61-
INSERT INTO users VALUES ($1, $2), ($3, $4);
62-
```
63-
64-
This scenario will raise an error (code `25001`).
65-
66-
### Default behavior
67-
68-
Both split inserts and sharding key updates fallback to raising an error if `enabled` is set to `false`.
6937

7038
### Read more
7139

72-
- [Rewrite behavior](../../features/sharding/sharding-functions.md#rewrite-behavior)
40+
- [Cross-shard INSERT](../../features/sharding/cross-shard-queries/insert.md#multiple-tuples)
41+
- [Cross-shard UPDATE](../../features/sharding/cross-shard-queries/update.md#sharding-key-updates)

docs/features/sharding/.pages

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
nav:
22
- 'index.md'
33
- 'basics.md'
4+
- 'supported-queries.md'
45
- 'query-routing.md'
56
- 'manual-routing.md'
67
- 'cross-shard-queries'

docs/features/sharding/cross-shard-queries/.pages

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
title: Cross-shard queries
12
nav:
23
- 'index.md'
34
- 'select.md'

docs/features/sharding/cross-shard-queries/copy.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ icon: material/upload
33
---
44
# COPY
55

6-
`COPY` is a special PostgreSQL command that ingests a file directly into a specified database table. This allows for writing data faster than by using individual `INSERT` queries.
6+
`COPY` is a special PostgreSQL command that can ingest a file directly into a specified database table. This allows for writing data faster than by using individual `INSERT` queries.
77

88
PgDog supports parsing the `COPY` command, splitting the input data stream automatically, and sending the rows to each shard in parallel.
99

@@ -36,6 +36,18 @@ By using _N_ nodes in a sharded database cluster, the performance of `COPY` incr
3636

3737
The cost of parsing and sharding the CSV stream in PgDog is negligibly small.
3838

39+
## COPY out
40+
41+
All `COPY [...] TO STDOUT` statements are treated as cross-shard and are executed on all shards concurrently. The rows are streamed directly, without buffering or sorting, which allows to read large amounts of data from all shards quickly.
42+
43+
PgDog doesn't currently support routing `COPY` statements based on its query. For example, the following statement will be sent to all shards even if it contains a sharding key:
44+
45+
```postgresql
46+
COPY (SELECT * FROM users WHERE id IN ($1, $2, $3)) TO STDOUT;
47+
```
48+
49+
If the query fetches rows from more than one shard, PgDog will also ignore any `ORDER BY` predicates and return rows in whatever order they arrive from the shards.
50+
3951
## Read more
4052

4153
- [Two-phase commit](../2pc.md)

docs/features/sharding/cross-shard-queries/insert.md

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ icon: material/table-plus
44

55
# Cross-shard INSERT
66

7-
If the `INSERT` statement specifies only one sharding key, it's [routed directly](../query-routing.md#insert) to one of the shards. Otherwise, it becomes a cross-shard `INSERT` statement.
7+
If an `INSERT` statement specifies only one sharding key, it's sent [directly](../query-routing.md#insert) to one of the shards. Otherwise, it becomes a cross-shard `INSERT` statement.
88

99
## How it works
1010

@@ -27,7 +27,7 @@ VALUES
2727
($1, $2, $3, $4)
2828
```
2929

30-
A row will be created on each shard. Each shard can then use joins to fetch this data with [direct-to-shard](../query-routing.md#select) queries.
30+
An identical row will be created on each shard. [Direct-to-shard](../query-routing.md#select) queries can then either fetch them directly or join with other sharded or omnisharded tables.
3131

3232
### Consistency
3333

@@ -80,10 +80,6 @@ Each call to `pgdog.unique_id()` generates a unique value, so it's possible to u
8080

8181
This function can be used with any tables, not just omnisharded ones, or independently of any tables at all.
8282

83-
Statements that target sharded tables but don't specify the sharding key are sent to one of the shards only. Which shard receives the statement is controlled by the round robin load balancing algorithm, ensuring all shards serve an equal number of statements.
84-
85-
Finally, if the `INSERT` statement has more than one tuple, the statement is automatically rewritten to create one row at a time, and each row is sent to the matching shard.
86-
8783
## Sharded tables
8884

8985
`INSERT` statements targetting sharded tables will commonly provide the sharding key. A notable exception to this rule is tables that shard on the primary key, which is often database-generated, e.g., using a sequence.
@@ -120,7 +116,7 @@ VALUES
120116
($4, $5, $6);
121117
```
122118

123-
In sharded databases, however, the individual tuples are likley to belong on different shards. To make this work, PgDog can automatically rewrite the statement and send each tuple to the right shard. Using the example above, the result of that operation produces two single-tuple statements:
119+
In sharded databases, however, the individual tuples are likely to belong on different shards. To make this work, PgDog can automatically rewrite the statement and send each tuple to the right shard. Using the example above, the result of that operation produces two single-tuple statements:
124120

125121
=== "Statement 1"
126122
```postgresql
@@ -157,7 +153,7 @@ INSERT INTO users (email, created_at) VALUES ($1, $2), ($3, $4);
157153
COMMIT; -- or ROLLBACK;
158154
```
159155

160-
If a transaction isn't started and a multi-tuple statement is sent by the application, PgDog will return an error and abort the request.
156+
If a transaction isn't started and a multi-tuple statement are sent by the application, PgDog will return an error and abort the request.
161157

162158
Requiring transactions ensures that if one of the `INSERT` statements fails, e.g., because of a unique constraint violation, the transaction can be rolled back, leaving the database in a consistent state.
163159

docs/features/sharding/cross-shard-queries/update.md

Lines changed: 107 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,110 @@
22
icon: material/table-edit
33
---
44

5-
# UPDATE
5+
# Cross-shard UPDATE and DELETE
6+
7+
`UPDATE` and `DELETE` statements that provide none or more than one sharding key in the `WHERE` clause are cross-shard and will be sent to all shards concurrently.
8+
9+
For example, assuming the `users` table is sharded on the `id` column, filtering rows by any other column produces a cross-shard query:
10+
11+
```postgresql
12+
UPDATE users SET is_admin = true WHERE email LIKE '%@pgdog.dev';
13+
```
14+
15+
## Consistency
16+
17+
Much like cross-shard [`INSERT`](insert.md) statements, any updates to multiple rows on multiple databases outside a [two-phase](../2pc.md) transaction are not guaranteed to be atomic. It's always best to send updates inside a transaction, like so:
18+
19+
```postgresql
20+
BEGIN;
21+
UPDATE orders SET delivered_at = now() WHERE delivered_at IS NULL;
22+
DELETE FROM orders WHERE user_id = $1;
23+
COMMIT;
24+
```
25+
26+
Both `DELETE` and `UPDATE` statements follow the same rules and work largely the same, with the notable exception of sharding key updates.
27+
28+
## Sharding key updates
29+
30+
When the sharding key of a row is updated, its sharded mapping is no longer valid. Queries searching for that row will be sent to the wrong shard, causing data loss. To avoid this, PgDog supports moving the row between shards automatically.
31+
32+
### How it works
33+
34+
Taking the same `users` table as an example (sharded on the `id` column), a statement mutating the sharding key will, frequently, take the following form:
35+
36+
```postgresql
37+
UPDATE users SET id = $1 WHERE id = $2;
38+
```
39+
40+
When a client sends such a statement, PgDog will rewrite it into three statements and execute them automatically:
41+
42+
1. `SELECT` statement to fetch the row from the database
43+
2. `INSERT` statement to create that row on the new shard, with updated values
44+
2. `DELETE` statement to remove the now stale row from the old shard
45+
46+
The entire exchange looks like this:
47+
48+
```postgresql
49+
-- Old `id` value.
50+
SELECT * FROM users WHERE id = $1;
51+
52+
-- New `id` value, with columns retrieved by the previous query.
53+
INSERT INTO users (id, email, created_at)
54+
VALUES ($1, $2, $3);
55+
56+
-- Old `id` value.
57+
DELETE FROM users WHERE id = $1;
58+
```
59+
60+
### Transaction required
61+
62+
Since PgDog needs to execute multiple statements across different databases, a transaction is required to maintain data consistency (as per [MVCC](https://www.postgresql.org/docs/current/mvcc.html)). PgDog expects all sharding key updates to be executed inside a transaction, like so:
63+
64+
```postgresql
65+
BEGIN;
66+
UPDATE users SET id = $1 WHERE id = $2 RETURNING *;
67+
COMMIT;
68+
```
69+
70+
If the application doesn't start a transaction, PgDog will return an error and abort the request.
71+
72+
### Configuration
73+
74+
This feature is **disabled** by default and can be enabled with configuration in [`pgdog.toml`](../../../configuration/pgdog.toml/rewrite.md):
75+
76+
```toml
77+
[rewrite]
78+
enabled = true
79+
shard_key = "rewrite"
80+
```
81+
82+
### Updating multiple rows
83+
84+
While multi-row updates are supported, changing multiple rows' sharding keys is not. If the `UPDATE` statement `WHERE` clause returns more than one row, PgDog will abort the transaction and return an error.
85+
86+
This check happens early in the transaction and will not create partial updates to the database.
87+
88+
### Efficiency
89+
90+
It's common practice for ORMs, like ActiveRecord or SQLAlchemy to mutate entire rows when saving records. Take the following example:
91+
92+
=== "ActiveRecord"
93+
```ruby
94+
user = User.find(1)
95+
user.email = "test@pgdog.dev"
96+
user.save
97+
```
98+
=== "SQL"
99+
```postgresql
100+
UPDATE users SET
101+
id = $1, email = $2, created_at = $4, updated_at = $3
102+
WHERE
103+
id = $1;
104+
```
105+
106+
While the sharding key (`id`) is technically updated in the query, its value doesn't change. To avoid unnecessary overhead, PgDog performs multiple checks on the new row before performing the [three statement](#how-it-works) sharding key update flow:
107+
108+
1. If the value of the sharding key parameter (`SET id = $1`) is the same as it is in the `WHERE` clause (`WHERE id = $1`), the query is routed directly to the shard without modifications.
109+
2. If the values are different, or the `WHERE` clause doesn't specify the sharding key, PgDog will check the value of the key returned by the first `SELECT` statement in the flow. If both keys map to the _same_ shard, will send the query directly to that shard, without modifications.
110+
111+
Updating the sharding key isn't a frequent operation and both of these mechanisms ensure that the more expensive algorithm isn't triggered unnecessarily.

docs/features/sharding/sharding-functions.md

Lines changed: 0 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -180,66 +180,6 @@ shard = 0
180180

181181
This will send all queries that don't specify a schema or use a schema without a mapping to shard zero.
182182

183-
## Rewrite behavior
184-
185-
PgDog can transparently move writes between shards when the [`rewrite`](../../configuration/pgdog.toml/rewrite.md) feature is enabled.
186-
187-
### Sharding key updates
188-
189-
Sharding key updates handle the situation when a query is changing the value of the sharding key, which could require the row to be moved to a different shard.
190-
191-
For example:
192-
193-
```postgresql
194-
UPDATE users SET id = $2 WHERE id = $1;
195-
```
196-
197-
If configured, PgDog can rewrite this query into three statements, executed inside a cross-shard transaction:
198-
199-
```postgresql
200-
SELECT * FROM users WHERE id = $1; /* query 1 */
201-
DELETE FROM users WHERE id = $1; /* query 2 */
202-
INSERT INTO users VALUES ($1, $2); /* row fetched in query #1 */
203-
```
204-
205-
#### Limitations
206-
207-
The row returned by _query 1_, constructed from the `WHERE` clause of the original `UPDATE` statement, has to match exactly one row. If that's not the case, the operation will be aborted and PgDog will raise an error.
208-
209-
### Multi-tuple inserts
210-
211-
Multi-tuple `INSERT` statements may write rows that belong on separate shards. To handle this situation, if configured, PgDog can rewrite the statement into multiple, single-tuple statements, and send them to their respective shards in a cross-shard transaction.
212-
213-
For example:
214-
215-
```postgresql
216-
INSERT INTO users (id, email) VALUES ($1, $2), ($3, $4);
217-
```
218-
219-
This statement will be rewritten into the following two queries:
220-
221-
```postgresql
222-
INSERT INTO users (id, email) VALUES ($1, $2);
223-
INSERT INTO users (id, email) VALUES ($3, $4);
224-
```
225-
226-
#### Limitations
227-
228-
Since PgDog starts a cross-shard transaction to make this operation atomic, the original `INSERT` statement must not be sent inside an explicit transaction by the client. If that's the case, PgDog will abort the operation and return an error.
229-
230-
### Configuration
231-
232-
Both features require the `enabled` flag to be set to `true`, for example:
233-
234-
```toml
235-
[rewrite]
236-
enabled = true
237-
split_inserts = "rewrite"
238-
shard_key = "rewrite"
239-
```
240-
241-
If a safe rewrite plan cannot be determined, PgDog will abort the transaction and return an error. To guarantee cross-shard atomicity of the operation, consider enabling [two-phase commit](2pc.md).
242-
243183
## Read more
244184

245185
- [COPY command](cross-shard-queries/copy.md)

0 commit comments

Comments
 (0)