You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/architecture/comparison.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ PgDog aims to be the de facto PostgreSQL proxy and pooler. Below is a feature co
28
28
|[Manual routing](../features/sharding/manual-routing.md)| Only using comments (regex), doesn't work with prepared statements | :material-check-circle-outline: |
29
29
|[Automatic routing](../features/sharding/query-routing.md)| No | :material-check-circle-outline: |
30
30
|[Primary key generation](../features/sharding/schema_management/primary_keys.md)| No | :material-check-circle-outline: |
31
-
|[Cross-shard queries](../features/sharding/cross-shard.md)| No | Partial support |
32
-
|[COPY](../features/sharding/copy.md)| No | :material-check-circle-outline: |
31
+
|[Cross-shard queries](../features/sharding/cross-shard-queries/index.md)| No | Partial support |
32
+
|[COPY](../features/sharding/cross-shard-queries/copy.md)| No | :material-check-circle-outline: |
33
33
|[Postgres-compatible sharding functions](../features/sharding/sharding-functions.md)| No | Same functions as declarative partitioning |
34
34
| Two-Phase Commit | No | :material-check-circle-outline: |
Copy file name to clipboardExpand all lines: docs/features/sharding/cross-shard-queries/copy.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,7 +22,7 @@ PgDog supports sharding data sent via `COPY`, using any one of the following for
22
22
| Text | PostgreSQL version of CSV, with `<tab>` (`\t`) as the delimiter. |`hello\tworld\t1\t2\t3`|
23
23
| Binary | PostgreSQL-specific format that encodes data using the format used to store it on disk. ||
24
24
25
-
Each row is extracted from the data stream, inspected for the sharding key, and sent to a data node. The sharding key should be specified in the [configuration](../../configuration/pgdog.toml/sharded_tables.md) and provided in the command statement, for example:
25
+
Each row is extracted from the data stream, inspected for the sharding key, and sent to a data node. The sharding key should be specified in the [configuration](../../../configuration/pgdog.toml/sharded_tables.md) and provided in the command statement, for example:
26
26
27
27
```postgresql
28
28
COPY users (id, email) FROM STDIN;
@@ -38,5 +38,5 @@ The cost of parsing and sharding the CSV stream in PgDog is negligibly small.
Copy file name to clipboardExpand all lines: docs/features/sharding/cross-shard-queries/ddl.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ icon: material/table-cog
8
8
9
9
## Atomicity
10
10
11
-
DDL statements should be atomic across all shards. This is to protect against a single shard failure to create a table or index, which could result in an inconsistent schema. PgDog can use [two-phase commit](2pc.md) to ensure this is the case, however that means that all DDL statements must be executed inside a transaction, for example:
11
+
DDL statements should be atomic across all shards. This is to protect against a single shard failure to create a table or index, which could result in an inconsistent schema. PgDog can use [two-phase commit](../2pc.md) to ensure this is the case, however that means that all DDL statements must be executed inside a transaction, for example:
12
12
13
13
```postgresql
14
14
BEGIN;
@@ -22,7 +22,7 @@ COMMIT;
22
22
23
23
## Idempotency
24
24
25
-
Some statements, like `CREATE INDEX CONCURRENTLY`, cannot run inside transactions. To make sure these are safely executed, you have two options: use [manual routing](manual-routing.md) and execute it on each shard individually, or write idempotent schema migrations, for example:
25
+
Some statements, like `CREATE INDEX CONCURRENTLY`, cannot run inside transactions. To make sure these are safely executed, you have two options: use [manual routing](../manual-routing.md) and execute it on each shard individually, or write idempotent schema migrations, for example:
Copy file name to clipboardExpand all lines: docs/features/sharding/index.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,9 +19,9 @@ PgDog is a query router. It can extract sharding hints directly from the SQL usi
19
19
| Query router feature | Description |
20
20
|-|-|
21
21
|[**Direct-to-shard queries**](query-routing.md)| Automatic sharding key detection which sends the query to one shard only. |
22
-
|[**Cross-shard queries**](cross-shard.md)| Queries that don't have a sharding key are sent to all shards with results collected and transformed, as if they came from one database. |
22
+
|[**Cross-shard queries**](cross-shard-queries/index.md)| Queries that don't have a sharding key are sent to all shards with results collected and transformed, as if they came from one database. |
23
23
|[**Manual routing**](manual-routing.md)| Provide the sharding key in a query comment, or separately with a `SET` PostgreSQL command. |
24
-
|[**Sharded COPY**](copy.md)| Data sent via `COPY` commands is automatically split between all shards, using the configured [sharding function](sharding-functions.md). |
24
+
|[**Sharded COPY**](cross-shard-queries/copy.md)| Data sent via `COPY` commands is automatically split between all shards, using the configured [sharding function](sharding-functions.md). |
Copy file name to clipboardExpand all lines: docs/features/sharding/query-routing.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,15 +40,15 @@ The `SELECT` query can express complex filtering logic and not all of it is curr
40
40
| Column equals to a value |`payments.user_id = $1`|
41
41
| Column matches against a list | `payments.user_id IN ($1, $2, $3)`
42
42
43
-
All other variations will be ignored and the query will be sent to [all shards](cross-shard.md).
43
+
All other variations will be ignored and the query will be sent to [all shards](cross-shard-queries/index.md).
44
44
45
45
!!! note "Query router improvements"
46
46
This is an area of constant improvement. Check back here for updates or [create an issue](https://github.com/pgdogdev/pgdog/issues/new) to request
47
47
support for a particular filter you're using.
48
48
49
49
If the query has multiple sharding key filters, all of them will be extracted and converged to a set of unique shard numbers.
50
50
51
-
For example, when filtering by a list of values, e.g., `WHERE user_id IN ($1, $2, $3)`, if all of them map to a single shard, the query will be sent to that shard only. If they map to two or more shards, it will be sent to all corresponding shards [concurrently](cross-shard.md).
51
+
For example, when filtering by a list of values, e.g., `WHERE user_id IN ($1, $2, $3)`, if all of them map to a single shard, the query will be sent to that shard only. If they map to two or more shards, it will be sent to all corresponding shards [concurrently](cross-shard-queries/index.md).
52
52
53
53
## INSERT
54
54
@@ -77,7 +77,7 @@ VALUES ($1, $2), ($3, $4) -- More than one tuple.
77
77
78
78
Both `UPDATE` and `DELETE` queries work identically to [`SELECT`](#select) queries. The query router looks inside the `WHERE` clause for sharding keys, and routes the query to the corresponding shard.
79
79
80
-
If no `WHERE` clause is present, or it's filtering on a column not used for sharding, the query is sent to all shards [concurrently](cross-shard.md), for example:
80
+
If no `WHERE` clause is present, or it's filtering on a column not used for sharding, the query is sent to all shards [concurrently](cross-shard-queries/index.md), for example:
81
81
82
82
```postgresql
83
83
UPDATE users SET banned = true;
@@ -144,5 +144,5 @@ The latter will match queries referring to the `users.id` column only. Together
Copy file name to clipboardExpand all lines: docs/features/sharding/resharding/hash.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,7 +39,7 @@ Required (*) and optional parameters for this command are as follows:
39
39
40
40
The first thing PgDog will do when data sync is started is create a replication slot on each primary database in the source cluster. This will prevent Postgres from removing the WAL, while we copy data for each table to the destination.
41
41
42
-
Next, each table will be copied, in parallel, to the destination database, using [sharded COPY](../copy.md). Once that's done, table changes are synchronized, in real-time, with logical replication from the replication slot created earlier.
42
+
Next, each table will be copied, in parallel, to the destination database, using [sharded COPY](../cross-shard-queries/copy.md). Once that's done, table changes are synchronized, in real-time, with logical replication from the replication slot created earlier.
43
43
44
44
The whole process happens entirely online, and doesn't require database reboots or pausing writes to the source database.
45
45
@@ -95,7 +95,7 @@ PgDog will distribute the table copy load evenly between all replicas in the con
95
95
96
96
#### Binary `COPY`
97
97
98
-
PgDog uses the binary `COPY` format to send and receive data, which has been shown to be consistently faster than text, because it avoids the (de)serialization overhead of sending tuples in text form. PgDog decodes tuples in-flight and splits them evenly between destination shards, using the [sharded COPY](../copy.md) implementation.
98
+
PgDog uses the binary `COPY` format to send and receive data, which has been shown to be consistently faster than text, because it avoids the (de)serialization overhead of sending tuples in text form. PgDog decodes tuples in-flight and splits them evenly between destination shards, using the [sharded COPY](../cross-shard-queries/copy.md) implementation.
99
99
100
100
!!! note "Binary compatibility"
101
101
While the data format used by PostgreSQL databases hasn't changed in decades, binary `COPY` sends rows exactly as they are stored on disk.
Copy file name to clipboardExpand all lines: docs/features/sharding/sharding-functions.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,7 +23,7 @@ The sharding functions are configurable in [`pgdog.toml`](../../configuration/pg
23
23
Since sharding is configured for each table or column name, this allows storing tables
24
24
with different sharding functions in the same database.
25
25
26
-
While this works for some [cross-shard](cross-shard.md) queries, joins between tables using a different sharding function are not possible for [direct-to-shard](query-routing.md) queries.
26
+
While this works for some [cross-shard](cross-shard-queries/index.md) queries, joins between tables using a different sharding function are not possible for [direct-to-shard](query-routing.md) queries.
27
27
28
28
29
29
## Hash
@@ -81,7 +81,7 @@ values = [1, 2, 3]
81
81
shard = 0
82
82
```
83
83
84
-
This example will route all queries with `user_id` equal to one, two, or three to shard zero. Unlike [hash](#hash) sharding, a value <-> shard mapping is required for _all_ values of the sharding key. If a value is used that doesn't have a mapping, the query will be sent to [all shards](cross-shard.md).
84
+
This example will route all queries with `user_id` equal to one, two, or three to shard zero. Unlike [hash](#hash) sharding, a value <-> shard mapping is required for _all_ values of the sharding key. If a value is used that doesn't have a mapping, the query will be sent to [all shards](cross-shard-queries/index.md).
85
85
86
86
!!! note "Required configuration"
87
87
The `[[sharded_tables]]` configuration entry is still required for list and range sharding. It specifies the data type of the column, which tells PgDog how to parse its value at runtime.
@@ -242,5 +242,5 @@ If a safe rewrite plan cannot be determined, PgDog will abort the transaction an
0 commit comments