save

levkk · levkk · commit e297f66169a8 · 2025-12-29T15:02:29.000-08:00
diff --git a/docs/features/sharding/.pages b/docs/features/sharding/.pages
@@ -3,9 +3,8 @@ nav:
   - 'basics.md'
   - 'query-routing.md'
   - 'manual-routing.md'
-  - 'cross-shard.md'
+  - 'cross-shard-queries'
   - 'sharding-functions.md'
-  - 'copy.md'
   - '...'
   - 'resharding'
   - 'internals'
diff --git a/docs/features/sharding/cross-shard-queries/.pages b/docs/features/sharding/cross-shard-queries/.pages
@@ -0,0 +1,7 @@
+nav:
+    - 'index.md'
+    - 'select.md'
+    - 'insert.md'
+    - 'update.md'
+    - 'ddl.md'
+    - '...'
diff --git a/docs/features/sharding/cross-shard-queries/copy.md b/docs/features/sharding/cross-shard-queries/copy.md
@@ -1,7 +1,7 @@
 ---
 icon: material/upload
 ---
-# COPY command
+# COPY
 
 `COPY` is a special PostgreSQL command that ingests a file directly into a specified database table. This allows for writing data faster than by using individual `INSERT` queries.
 
diff --git a/docs/features/sharding/cross-shard-queries/ddl.md b/docs/features/sharding/cross-shard-queries/ddl.md
@@ -0,0 +1,30 @@
+---
+icon: material/table-cog
+---
+
+# CREATE, ALTER, DROP
+
+`CREATE`, `ALTER` and `DROP`, also known as **D**ata **D**efinition **L**anguage (DDL), are by design, cross-shard statements. When a client sends over a DDL command, PgDog will send it to all shards in parallel, ensuring the table, index, view and sequence definitions are identical across the database cluster.
+
+## Atomicity
+
+DDL statements should be atomic across all shards. This is to protect against a single shard failure to create a table or index, which could result in an inconsistent schema. PgDog can use [two-phase commit](2pc.md) to ensure this is the case, however that means that all DDL statements must be executed inside a transaction, for example:
+
+```postgresql
+BEGIN;
+CREATE TABLE users (
+    id BIGSERIAL PRIMARY KEY,
+    email VARCHAR NOT NULL,
+    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
+);
+COMMIT;
+```
+
+## Idempotency
+
+Some statements, like `CREATE INDEX CONCURRENTLY`, cannot run inside transactions. To make sure these are safely executed, you have two options: use [manual routing](manual-routing.md) and execute it on each shard individually, or write idempotent schema migrations, for example:
+
+```postgresql
+DROP INDEX IF EXISTS user_id_idx;
+CREATE INDEX CONCURRENTLY user_id_idx ON users USING btree(user_id);
+```
diff --git a/docs/features/sharding/cross-shard-queries/index.md b/docs/features/sharding/cross-shard-queries/index.md
@@ -0,0 +1,74 @@
+---
+icon: material/multicast
+---
+
+# Cross-shard queries overview
+
+If a client can't or doesn't specify a sharding key in the query, PgDog will send that query to all shards in parallel, and combine the results automatically. To the client, this looks like the query was executed by a single database.
+
+<center style="margin-top: 2rem;">
+    <img src="/images/cross-shard.png" width="95%" alt="Cross-shard queries" />
+</center>
+
+## How it works
+
+PgDog understands the Postgres protocol and query language. It can connect to multiple database servers, send the query to all of them, and collect [`DataRow`](#under-the-hood) messages as they are returned by each connection.
+
+Once all servers finish executing the request, PgDog processes the result, performs any requested sorting, aggregation or row disambiguation, and sends the complete result back to the client, as if all rows came from one database server.
+
+Just like with [direct-to-shard](../query-routing.md) queries, each SQL command is handled differently, as documented below:
+
+- [`SELECT`](select.md)
+- [`INSERT`](insert.md)
+- [`UPDATE`, `DELETE`](update.md)
+- [`CREATE`, `ALTER`, `DROP`](ddl.md)(and other DDL statements)
+- [`COPY`](copy.md)
+
+
+## Under the hood
+
+PgDog implements the PostgreSQL wire protocol, which is well documented and stable. The messages sent by Postgres clients and servers contain all the necessary information about data types, column names and executed statements, which PgDog can use to present multi-database results as a single stream of data.
+
+The following protocol messages are especially relevant:
+
+| Message | Description |
+|-|-|
+| `DataRow` | Each `DataRow` message contains one tuple, for each row returned by the query. |
+| `RowDescription` | This message has the column names and data types returned by the query. |
+| `CommandComplete` | Indicates that the query has finished returning results. PgDog uses it to start sorting and aggregation. |
+
+The protocol has two formats for encoding tuples: text and binary. Text format is equivalent to calling the `to_string()` method on native types, while binary encoding sends them in network-byte order. For example:
+
+=== "Data"
+    ```postgresql
+    SELECT 1::bigint, 2::integer, 'three'::VARCHAR;
+    ```
+=== "Encoding"
+    | Data type | Text | Binary |
+    |-|-|-|
+    | `BIGINT` | `"1"` | `00 00 00 00 00 00 00 01` |
+    | `INTEGER` | `"2"` | `00 00 00 02` |
+    | `VARCHAR` | `"three"` | `three` |
+
+Since PgDog needs to process rows before sending them to the client, we implemented parsing both formats for [most data types](select.md#supported-data-types).
+
+
+## Disabling cross-shard queries
+
+If you don't want PgDog to route cross-shard queries, e.g., because you have a [multitenant](../../multi-tenancy.md) system with no interdependencies, cross-shard queries can be disabled with a configuration setting:
+
+```toml
+[general]
+cross_shard_disabled = true
+```
+
+If this feature is enabled, and a query doesn't have a sharding key, instead of executing the query, PgDog will return an error and abort the transaction.
+
+## Read more
+
+- [Sharding functions](../sharding-functions.md)
+- [Cross-shard `SELECT`](select.md)
+- [Cross-shard `INSERT`](insert.md)
+- [Cross-shard `UPDATE` and `DELETE`](update.md)
+- [DDL, e.g., `CREATE TABLE`](ddl.md)
+- [`COPY` command](copy.md)
diff --git a/docs/features/sharding/cross-shard-queries/insert.md b/docs/features/sharding/cross-shard-queries/insert.md
@@ -0,0 +1,99 @@
+---
+icon: material/table-plus
+---
+
+# Cross-shard INSERT
+
+If the `INSERT` statement specifies only one sharding key, it's [routed directly](../query-routing.md#insert) to one of the shards. Otherwise, it becomes a cross-shard `INSERT` statement.
+
+## How it works
+
+Cross-shard `INSERT` statements fall into one of three categories, each one handled differently:
+
+1. `INSERT` targetting [omnisharded tables](#omnisharded-tables)
+2. `INSERT` targetting a [sharded table](#sharded-insert), with no sharding key specified
+3. `INSERT` with [multiple tuples](#multiple-tuples), each destined for a different shard
+
+By design, applications using PgDog don't need to concern themselves with this and can use the database normally. However, there are some trade-offs when using cross-shard queries, documented below.
+
+## Omnisharded tables
+
+Queries that target [omnisharded](../omnishards.md) tables, the statement is sent to all shards concurrently. This ensures that the data is identical on all shards, for example:
+
+```postgresql
+INSERT INTO request_logs
+    (client_ip, request_path, response_code, created_at)
+VALUES
+    ($1, $2, $3, $4)
+```
+
+A row will be created on each shard. Each shard can then use joins to fetch this data with [direct-to-shard](../query-routing.md#select) queries.
+
+### Consistency
+
+Unless [two-phase commit](../2pc.md) is enabled, inserts into omnisharded tables are not guaranteed to be atomic. It is possible for the statement to succeed on some of the shards and not others. If you don't want to or can't enable 2pc, consider sending cross-shard inserts inside a transaction:
+
+```postgresql
+BEGIN;
+INSERT INTO request_logs
+    (client_ip, request_path, response_code, created_at)
+VALUES
+    ($1, $2, $3, $4);
+-- You will receive an ack or an error from all shards here.
+COMMIT;
+```
+
+This gives you a much higher chance of recording rows on all shards, since you will know if your statement violated some contstraint (e.g., unique index or `NOT NULL` check) before committing the transaction.
+
+### Primary keys
+
+It's common practice to use `BIGSERIAL` columns as primary keys. These are powered by a sequence to ensure non-recurring values are automatically generated for each new row.
+
+Sharded databases can't use sequences directly because they are not aware of other shards and will produce duplicate values across databases. To circumvent this, PgDog provides a way to generate [unique integers](../unique-ids.md) in the proxy using a distributed and shard-aware algorithm.
+
+To use unique IDs as primary keys (or in any other column) in omnisharded tables, you can call the `pgdog.unique_id()` function in the `VALUES` clause. For example:
+
+```postgresql
+INSERT INTO ip_logs
+    (id, client_ip, created_at)
+VALUES
+    (pgdog.unqiue_id(), $1, now());
+```
+
+The function is evaluated inside PgDog which places the value it returns directly into the query. This works for all queries, including prepared statements.
+
+Each call to `pgdog.unique_id()` generates a unique value, so it's possible to use it multiple times inside the same query and get different numbers, for example:
+
+=== "Query"
+    ```postgresql
+    SELECT
+        pgdog.unique_id() AS one,
+        pgdog.unique_id() AS two;
+    ```
+=== "Result"
+    ```
+           one         |        two
+    -------------------+-------------------
+     12014338348945408 | 12014338348945409
+    (1 row)
+    ```
+
+This function can be used with any tables, not just omnisharded ones, or independently of any tables at all.
+
+Statements that target sharded tables but don't specify the sharding key are sent to one of the shards only. Which shard receives the statement is controlled by the round robin load balancing algorithm, ensuring all shards serve an equal number of statements.
+
+Finally, if the `INSERT` statement has more than one tuple, the statement is automatically rewritten to create one row at a time, and each row is sent to the matching shard.
+
+## Sharded tables
+
+If the `INSERT` is creating a row in a sharded table, but the primary key is [database-generated](schema_management/primary_keys.md) _and_ used for sharding that table, the statement is sent to only one of the shards, using the round robin algorithm.
+
+For example:
+
+```postgresql
+INSERT INTO users (id, email) VALUES (DEFAULT, 'test@acme.com') RETURNING *;
+```
+
+Instead of creating one user per shard, which would cause duplicate entries, PgDog will let the database generate a _globally_ unique primary key and place it on one of the shards only. This ensures even data distribution across the entire database cluster.
+
+## Multiple tuples
diff --git a/docs/features/sharding/cross-shard-queries/select.md b/docs/features/sharding/cross-shard-queries/select.md
diff --git a/docs/features/sharding/cross-shard-queries/update.md b/docs/features/sharding/cross-shard-queries/update.md
diff --git a/docs/features/sharding/cross-shard.md b/docs/features/sharding/cross-shard.md
