dark

Author: levkk

docs/features/connection-pooler/index.md

@@ -39,6 +39,12 @@ If your application uses `LISTEN`/`NOTIFY`, e.g., [DBOS](https://dbos.dev) or an
 
 This allows applications to use `LISTEN`/`NOTIFY` in [transaction mode](pub_sub.md), just like any other Postgres feature.
 
+### Connection recovery
+
+Unlike other poolers, PgDog takes extra care not to close connections to Postgres unless absolutely necessary. It goes as far as to roll back abandoned transactions and drain abandoned queries. This protects the database against connection storms created by buggy applications.
+
+You can read more about connection recovery methods [here](connection-recovery.md).
+
 ## Read more
 
 {{ next_steps_links([

docs/features/load-balancer/healthchecks.md

@@ -9,7 +9,8 @@ All databases load balanced by PgDog are regularly checked with health checks. A
 If a replica database fails a health check, it's temporarily removed from the load balancer, preventing it from serving queries for a configurable period of time.
 
 <center>
-  <img src="/images/healthchecks.png" width="80%" alt="Healthchecks"/>
+  <img src="/images/healthchecks.png" width="80%" alt="Healthchecks" class="theme-aware-image" />
+  <p>Health checks prevent broken databases from serving queries.</p>
 </center>
 
 ## How it works
@@ -23,8 +24,7 @@ PgDog performs two kinds of health checks to ensure applications don't accidenta
 
 If a connection or database fails a health check, it is **temporarily removed** from the load balancer and cannot serve any more queries. This prevents applications from continuously hitting a broken database until it's restarted by an administrator.
 
-!!! note "99.99% uptime"
-    This strategy is very effective at reducing error rates in busy applications. If you are operating a large number of databases, hardware failures are relatively common and an effective load balancer is required to maintain 99.99% database uptime.
+This strategy is very effective at reducing error rates in busy applications. If you are operating a large number of databases, hardware failures are relatively common and an effective load balancer is required to maintain 99.99% database uptime.
 
 ### Connection health check
 
@@ -85,6 +85,13 @@ When PgDog is first started, it's possible that the database or the network is n
 
 The **default** value for this setting is **5 seconds** (`5_000` milliseconds).
 
+!!! note "Disabling health checks"
+    If you want to disable database health checks, you can set the `idle_healthcheck_delay` setting to a large number, e.g. 100 years, in milliseconds:
+
+    ```toml
+    [general]
+    idle_healthcheck_delay = 3155760000000
+    ```
 
 ### Primary database exception
 
@@ -108,10 +115,7 @@ The amount of time the database is banned from serving traffic is controlled wit
 
 The **default** value is **5 minutes** (`300_000` milliseconds).
 
-!!! note
-    A database will not be placed back into the load balancer until it passes a health check again.
-
-    Make sure that `idle_healthcheck_interval` is set to a lower value than `ban_timeout`, so health checks have time to run before you expect the database to resume serving traffic.
+A database will not be placed back into the load balancer until it passes a health check again. Make sure that `idle_healthcheck_interval` is set to a lower value than `ban_timeout`, so health checks have time to run before you expect the database to resume serving traffic.
 
 ### False positives
 
@@ -170,7 +174,10 @@ This is configurable on startup only and will spin up an HTTP server on `http://
 
 This health check looks at all configured connection pools, and if **at least one** is online, responds with `HTTP/1.1 200 OK`. If _all_ connection pools are down because of failed health checks, PgDog will respond with `HTTP/1.1 502 Bad Gateway`.
 
-!!! note "Handling a lot of requests"
-    The HTTP health check uses existing internal state to answer requests and doesn't send queries to the connection pools. This makes it very quick and inexpensive, which ensures that massively distributed load balancers (like the AWS NLB) don't cause an unexpected influx of requests to the database.
+#### Handling a lot of requests
+
+The HTTP health check uses existing internal state to answer requests and doesn't send queries to the connection pools. This makes it very quick and inexpensive, which ensures that massively distributed load balancers (like the AWS NLB) don't cause an unexpected influx of requests to the database.
+
+#### HTTPS
 
 To make configuration easier, the health check endpoint doesn't support HTTPS, so make sure to configure your load balancer to use plain HTTP only.

docs/features/load-balancer/index.md

@@ -9,7 +9,7 @@ icon: material/lan
 
 # Load balancer overview
 
-PgDog understands the PostgreSQL wire protocol and uses its SQL parser to understand queries. This allows it to split read queries from write queries and distribute traffic evenly between databases.
+PgDog understands the PostgreSQL wire protocol and uses the native PostgreSQL parser to understand queries. This allows it to split read queries from write queries and distribute traffic evenly between databases.
 
 Applications can connect to a single PgDog [endpoint](#single-endpoint), without having to manually manage multiple connection pools.
 
@@ -18,13 +18,15 @@ Applications can connect to a single PgDog [endpoint](#single-endpoint), without
 When a query is received by PgDog, it will inspect it using the native Postgres SQL parser. If the query is a `SELECT` and the [configuration](../../configuration/pgdog.toml/databases.md) contains both primary and replica databases, PgDog will send it to one of the replicas. For all other queries, PgDog will send them to the primary.
 
 <center>
-  <img src="/images/replicas.png" width="80%" alt="Load balancer" />
+  <img src="/images/replicas.png" width="95%" alt="Load balancer" class="theme-aware-image" />
+  <p>Load balancer topology</p>
 </center>
 
 Applications don't have to manually route queries between databases or maintain several connection pools internally.
 
-!!! note "SQL compatibility"
-    PgDog's query parser is powered by the `pg_query` library, which extracts the Postgres native SQL parser directly from its source code. This makes it **100% compatible** with the PostgreSQL query language and allows PgDog to understand all valid PostgreSQL queries.
+### SQL compatibility
+
+PgDog's query parser is powered by the `pg_query` library, which extracts the Postgres native SQL parser directly from its source code. This makes it **100% compatible** with the PostgreSQL query language and allows PgDog to understand all valid PostgreSQL queries.
 
 ## Load distribution
 

docs/features/load-balancer/manual-routing.md

@@ -139,17 +139,17 @@ SET LOCAL "pgdog"."role" TO "primary";
 
 In this example, all transaction statements (including the `BEGIN` statement) will be sent to the primary database. Whether the transaction is committed or reverted, the value of `pgdog.role` will be reset to its previous value.
 
-!!! note "Statement ordering"
-    To make sure PgDog intercepts the routing hint early enough in the transaction flow, you need to send all hints _before_ executing actual queries.
+#### Statement ordering
+To make sure PgDog intercepts the routing hint early enough in the transaction flow, you need to send all hints _before_ executing DDL/DML statements.
 
-    The following flow, for example, _will not_ work:
+The following flow, for example, _will not_ work:
 
-    ```postgresql
-    BEGIN;
-    SELECT * FROM users WHERE id = $1;
-    SET LOCAL pgdog.role TO "primary"; -- The client is already connected to a server.
-    INSERT INTO users (id) VALUES ($1); -- If connected to a replica, this will fail.
-    ```
+```postgresql
+BEGIN;
+SELECT * FROM users WHERE id = $1;
+SET LOCAL pgdog.role TO "primary"; -- The client is already connected to a server.
+INSERT INTO users (id) VALUES ($1); -- If connected to a replica, this will fail.
+```
 
 
 
@@ -171,6 +171,24 @@ If you've configured the desired database role (and/or shard) for each of your a
 
 Once it's disabled, PgDog will rely solely on the `pgdog.role` and `pgdog.shard` parameters to make its routing decisions.
 
-### Session state & `SET`
+### Session state and SET
 
 The query parser is used to intercept and interpret `SET` commands. If the parser is disabled and your application uses `SET` commands to configure the connection, PgDog will not be able to guarantee that all connections have the correct session settings in [transaction mode](../connection-pooler/transaction-mode.md).
+
+You can keep the parser enabled for handling `SET` commands only as follows:
+
+=== "pgdog.toml"
+    ```toml
+    [general]
+    query_parser = "session_state"
+    ```
+=== "Helm chart"
+    ```yaml
+    queryParser: session_state
+    ```
+
+The internal implementation is using a very fast Regex to detect `SET` commands and will turn on the query parser for that statement only. The regex supports comments as well, so the following example will be detected:
+
+```postgresql
+/* api: users.create */ SET application_name TO 'sidekiq';
+```

docs/features/load-balancer/replication-failover.md

@@ -4,7 +4,7 @@ icon: material/chart-timeline-variant
 
 # Replication and failover
 
-PgDog has built-in functionality for monitoring the state of Postgres replica databases. If configured, it can also automatically detect when a replica is promoted and redirect write queries to the new primary, and ban replicas from serving traffic if they have fallen far behind in the replication stream.
+PgDog has built-in functionality for monitoring the state of Postgres replica databases. If configured, it can also automatically detect when a replica is promoted and redirect write queries to the new primary, or block replicas from serving traffic if they have fallen far behind in the replication stream.
 
 ## Replication
 
@@ -25,15 +25,15 @@ In addition to fetching raw metrics, PgDog can calculate the replication lag (al
 | Primary LSN | Get the LSN from the primary using `pg_current_wal_lsn()`. |
 | Replica LSN | Get the LSN from each replica using `pg_last_wal_replay_lsn()` or `pg_last_wal_receive_lsn()`. |
 | LSN check | If the two LSNs are identical, replication lag is 0. |
-| Calculate lag | If the two LSNs are different, replication lag is `now() - pg_last_xact_replay_timestamp()`. |
+| Calculate lag | If the two LSNs are different, replication lag is `now() - pg_last_xact_replay_timestamp()` retrieved from the replica. |
 
 This formula assumes that when the replica's LSN is behind the primary, the primary is still receiving write requests. While this is not always the case, it will show replication lag growing over time if the replication stream is falling behind or is broken.
 
 !!! note "Formula accuracy"
     It is possible to calculate the exact replication delay in bytes by subtracting a replica LSN from the primary LSN. While this provides an exact measurement,
     that metric isn't very useful: it's hard to translate bytes into a measurement of how stale the data on the replica truly is.
 
-    Approximating the lag in milliseconds is more informative and will be reasonably accurate the majority of the time.
+    Approximating the lag in milliseconds is more informative and will be reasonably accurate most of the time.
 
 ### Configuration
 
@@ -66,10 +66,15 @@ Decreasing the value of `lsn_check_interval` will produce more accurate statisti
 
 It's common for PgDog deployments to be serving upwards of 30,000-50,000 queries per second per pooler process, so you can run the LSN check query quite frequently without noticeable impact on system latency.
 
+#### Saturated connection pool
+
+If the connection pool is at capacity, either due to a database incident or an inefficient query, PgDog will create a standalone connection to fetch the LSN from each database in the configuration. This is to make sure that it can't miss
+a [failover](#failover) event during a database incident.
+
 ### Replica lag ban
 
-!!! note "Experimental feature"
-    This feature is new and experimental. Please report any issues you encounter.
+!!! note "New feature"
+    This feature is new and experimental. Please report any issues you may encounter.
 
 If a replica has fallen far behind the primary, it may start serving stale data to the application. This can cause hard to debug issues, so it's often best to remove this replica from the load balancer until it's able to catch up.
 
@@ -98,14 +103,15 @@ Unlike [health check-triggered](healthchecks.md) bans, replica lag ban is not cl
 
 ## Failover
 
-<center>
-  <img src="/images/failover.png" width="95%" alt="Failover" />
-</center>
-
 If the `pg_is_in_recovery()` function returns `true`, the database is configured as a standby. It can only serve read queries (e.g. `SELECT`) and is expected to be reasonably up-to-date with the primary database.
 
 Replica databases can be promoted to serve write queries. If that happens, `pg_is_in_recovery()` will start returning `false`. You can read more about this in the [PostgreSQL documentation](https://www.postgresql.org/docs/18/functions-admin.html#FUNCTIONS-RECOVERY-CONTROL).
 
+<center>
+  <img src="/images/failover.png" width="90%" alt="Failover" class="theme-aware-image" />
+  <p>Failover event</p>
+</center>
+
 !!! warning "Failover trigger"
     PgDog does not detect primary failure and **will not** call `pg_promote()`. It is expected that the databases are managed externally by another tool, like Patroni or AWS RDS, which handle replica promotion.
 
@@ -140,6 +146,10 @@ Failover is disabled by default. To enable it, change all configured databases i
 
 On startup, PgDog will connect to each database, find out if they are in recovery, and automatically reload its configuration with the determined roles.
 
+!!! info "Replication lag monitoring"
+    In order for PgDog to detect failover events, it needs to query the database for its [replication status](#configuration). Make sure to set
+    `lsn_check_delay` to a reasonable value (e.g., `0`) before enabling this feature.
+
 ### Split brain
 
 If a replica is promoted while the existing primary is alive and serving queries, write queries can be routed to either database, causing data loss. This type of error is called "split brain", indicating that the database cluster no longer has an authoritative source of data it's managing.