-
Notifications
You must be signed in to change notification settings - Fork 393
Draft: Support Rails 8.1 + pure-Ruby PostgreSQL parity #1217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
e088992
90eeb46
3cd3e35
d015b27
9fe3b51
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| # activerecord-jdbcpostgresql-adapter | ||
|
|
||
| * https://github.com/jruby/activerecord-jdbc-adapter/ | ||
|
|
||
| ## Description | ||
|
|
||
| This is an ActiveRecord driver for PostgreSQL using JDBC running under JRuby. | ||
|
|
||
| It is part of the [activerecord-jdbc-adapter](https://github.com/jruby/activerecord-jdbc-adapter) | ||
| project; see the top-level README for general installation and configuration. | ||
|
|
||
| ## Connection and resilience options | ||
|
|
||
| In addition to the standard ActiveRecord PostgreSQL settings, the adapter | ||
| recognizes the following `database.yml` keys. Each is translated to the | ||
| corresponding [pgjdbc connection parameter](https://jdbc.postgresql.org/documentation/use/#connection-parameters). | ||
| Setting timeouts and TCP keep-alive is strongly recommended when running behind a | ||
| connection pooler or a firewall that drops idle connections, so a dead/reaped | ||
| backend surfaces as an error instead of hanging the JVM thread. | ||
|
|
||
| | `database.yml` key | pgjdbc property | Meaning | Default | | ||
| | --- | --- | --- | --- | | ||
| | `connect_timeout` | `connectTimeout` | Timeout (seconds) for establishing a new connection. Also read from `PGCONNECT_TIMEOUT`. | pgjdbc default (10s) | | ||
| | `socket_timeout` | `socketTimeout` | Per-query read timeout (seconds) on an established connection. `0` = infinite. | unset (infinite) | | ||
| | `login_timeout` | `loginTimeout` | Timeout (seconds) for the connection + authentication handshake. | unset | | ||
| | `prepare_threshold` | `prepareThreshold` | How many executions before a query is promoted to a *named* server-side prepared statement. `0` keeps statements unnamed. Overrides the value derived from `prepared_statements`. | derived from `prepared_statements` | | ||
| | `keepalives` | `tcpKeepAlive` | Enable TCP keep-alive probes. | unset (off) | | ||
|
|
||
| You can also pass any pgjdbc property directly under `properties:` (e.g. | ||
| `tcpKeepAlive: true`). Example: | ||
|
|
||
| ```yml | ||
| production: | ||
| adapter: postgresql | ||
| database: blog | ||
| username: blog | ||
| password: blog | ||
| connect_timeout: 5 # cap connection establishment (pgjdbc connectTimeout) | ||
| socket_timeout: 30 # cap per-query reads (pgjdbc socketTimeout) | ||
| properties: | ||
| tcpKeepAlive: true # detect dead backends instead of hanging | ||
| ``` | ||
|
|
||
| > **Behavior change:** prior to this release `connect_timeout` was mapped to | ||
| > pgjdbc's `socketTimeout` (a per-query read timeout). It now maps to | ||
| > `connectTimeout`, matching the pg gem / libpq meaning of `connect_timeout`. If | ||
| > you relied on the old behavior to bound query execution time, set | ||
| > `socket_timeout` instead. | ||
|
|
||
| ## Running behind PgBouncer (or another connection pooler) | ||
|
|
||
| The PostgreSQL adapter works out of the box with PgBouncer in **`session`** | ||
| pooling mode — every checkout keeps one backend, so per-session state is stable | ||
| and no special configuration is required. | ||
|
|
||
| In **`transaction`** or **`statement`** pooling mode PgBouncer reassigns the | ||
| server backend between transactions, which breaks anything that relies on | ||
| per-session server state. See the next section before using those modes. | ||
|
|
||
| | Concern | `session` mode | `transaction` / `statement` mode | | ||
| | --- | --- | --- | | ||
| | Prepared statements | works as-is | `prepared_statements: false` (or PgBouncer >= 1.21, see below) | | ||
| | Advisory locks (migrations) | works as-is | `advisory_locks: false` | | ||
| | Session `SET`s (timezone, search_path, intervalstyle) | applied at connect | must be enforced per backend (see below) | | ||
|
|
||
| ## PgBouncer transaction (and statement) pooling mode | ||
|
|
||
| Use this configuration as a starting point: | ||
|
|
||
| ```yml | ||
| production: | ||
| adapter: postgresql | ||
| host: 127.0.0.1 | ||
| port: 6432 # PgBouncer listening port | ||
| database: blog | ||
| username: blog | ||
| password: blog | ||
| prepared_statements: false # required: named server statements can't span backends | ||
| advisory_locks: false # required: session-scoped locks can't span backends | ||
| connect_timeout: 5 | ||
| socket_timeout: 30 | ||
| properties: | ||
| tcpKeepAlive: true | ||
| ``` | ||
|
|
||
| **Prepared statements.** With `prepared_statements: true` (the default) the | ||
| pgjdbc driver promotes a query to a *named* server-side prepared statement after | ||
| it has executed `prepareThreshold` (5 by default) times. That named statement | ||
| lives on one backend, so once PgBouncer hands the next transaction a different | ||
| backend you get `ERROR: prepared statement "S_1" does not exist`. Two options: | ||
|
|
||
| * Set `prepared_statements: false` — this forces pgjdbc `prepareThreshold=0`, so | ||
| only unnamed statements are used, which are safe across backends. | ||
| * Or run PgBouncer >= 1.21 with `max_prepared_statements > 0` and keep prepared | ||
| statements on. Tune the promotion point with `prepare_threshold:` if needed. | ||
|
|
||
| **Advisory locks.** Schema migrations take a session-level advisory lock that is | ||
| acquired and released across transaction boundaries. Under transaction/statement | ||
| pooling the unlock may land on a different backend and orphan the lock, so set | ||
| `advisory_locks: false`. | ||
|
|
||
| **Session `SET` statements.** On connect the adapter issues `SET TIME ZONE`, | ||
| `SET intervalstyle = iso_8601`, applies `schema_search_path`, etc. These do | ||
| **not** follow backend reassignment, which can *silently* corrupt results | ||
| (interval parsing, timestamps, querying the wrong schema). Because PgBouncer in | ||
| transaction/statement mode disallows per-session state, enforce these per backend | ||
| instead — for example: | ||
|
|
||
| * `ALTER ROLE blog SET intervalstyle = 'iso_8601';` | ||
| * `ALTER ROLE blog SET timezone = 'UTC';` (when using ActiveRecord UTC time zone support) | ||
| * set `search_path` via `ALTER ROLE`/`ALTER DATABASE` or PgBouncer's server-side | ||
| connect options instead of `schema_search_path:` | ||
|
|
||
| If you cannot enforce that state globally, use `session` pooling for that database. | ||
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -58,7 +58,7 @@ def translate_exception(exception, message:, sql:, binds:) | |
| end | ||
|
|
||
| # this version of log() automatically fills type_casted_binds from binds if necessary | ||
| def log(sql, name = "SQL", binds = [], type_casted_binds = [], async: false, &block) | ||
| def log(sql, name = "SQL", binds = [], type_casted_binds = [], async: false, allow_retry: false, &block) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| if binds.any? && (type_casted_binds.nil? || type_casted_binds.empty?) | ||
| type_casted_binds = lambda { | ||
| # extract_raw_bind_values | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,13 +19,16 @@ def exec_insert(sql, name = nil, binds = NO_BINDS, pk = nil, sequence_name = nil | |
| binds = convert_legacy_binds_to_attributes(binds) if binds.first.is_a?(Array) | ||
|
|
||
| with_raw_connection do |conn| | ||
| if without_prepared_statement?(binds) | ||
| log(sql, name) { conn.execute_insert_pk(sql, pk) } | ||
| else | ||
| log(sql, name, binds) do | ||
| conn.execute_insert_pk(sql, binds, pk) | ||
| result = | ||
| if without_prepared_statement?(binds) | ||
| log(sql, name) { conn.execute_insert_pk(sql, pk) } | ||
| else | ||
| log(sql, name, binds) do | ||
| conn.execute_insert_pk(sql, binds, pk) | ||
| end | ||
| end | ||
| end | ||
| handle_warnings(sql) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Roughly equivalent to the following section, but Rails moves some of this boilerplate into a Also, this change and several below are a single method in Rails: |
||
| result | ||
| end | ||
| end | ||
|
|
||
|
|
@@ -41,15 +44,18 @@ def internal_exec_query(sql, name = nil, binds = NO_BINDS, prepare: false, async | |
| binds = convert_legacy_binds_to_attributes(binds) if binds.first.is_a?(Array) | ||
|
|
||
| with_raw_connection do |conn| | ||
| if without_prepared_statement?(binds) | ||
| log(sql, name, async: async) { conn.execute_query(sql) } | ||
| else | ||
| log(sql, name, binds, async: async) do | ||
| # this is different from normal AR that always caches | ||
| cached_statement = fetch_cached_statement(sql) if prepare && @jdbc_statement_cache_enabled | ||
| conn.execute_prepared_query(sql, binds, cached_statement) | ||
| result = | ||
| if without_prepared_statement?(binds) | ||
| log(sql, name, async: async) { conn.execute_query(sql) } | ||
| else | ||
| log(sql, name, binds, async: async) do | ||
| # this is different from normal AR that always caches | ||
| cached_statement = fetch_cached_statement(sql) if prepare && @jdbc_statement_cache_enabled | ||
| conn.execute_prepared_query(sql, binds, cached_statement) | ||
| end | ||
| end | ||
| end | ||
| handle_warnings(sql) | ||
| result | ||
| end | ||
| end | ||
|
|
||
|
|
@@ -63,11 +69,14 @@ def exec_update(sql, name = 'SQL', binds = NO_BINDS) | |
| binds = convert_legacy_binds_to_attributes(binds) if binds.first.is_a?(Array) | ||
|
|
||
| with_raw_connection do |conn| | ||
| if without_prepared_statement?(binds) | ||
| log(sql, name) { conn.execute_update(sql) } | ||
| else | ||
| log(sql, name, binds) { conn.execute_prepared_update(sql, binds) } | ||
| end | ||
| result = | ||
| if without_prepared_statement?(binds) | ||
| log(sql, name) { conn.execute_update(sql) } | ||
| else | ||
| log(sql, name, binds) { conn.execute_prepared_update(sql, binds) } | ||
| end | ||
| handle_warnings(sql) | ||
| result | ||
| end | ||
| end | ||
| alias :exec_delete :exec_update | ||
|
|
@@ -90,22 +99,37 @@ def convert_legacy_binds_to_attributes(binds) | |
| end | ||
| end | ||
|
|
||
| def preprocess_query(sql) | ||
| check_if_write_query(sql) if respond_to?(:check_if_write_query, true) | ||
| mark_transaction_written_if_write(sql) if respond_to?(:mark_transaction_written_if_write, true) | ||
| sql | ||
| # AR >= 8.1 removed these helpers, folding them into #preprocess_query via | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fine for now but we are introducing more divergence from the original. |
||
| # the primitives #write_query?, #ensure_writes_are_allowed and | ||
| # #mark_transaction_written. The JDBC adapters' custom query paths | ||
| # (e.g. #exec_insert, #execute_and_clear) still call them, so reintroduce | ||
| # them here on top of those primitives. | ||
| def mark_transaction_written_if_write(sql) | ||
| mark_transaction_written if write_query?(sql) | ||
| end | ||
|
|
||
| def check_if_write_query(sql) | ||
| ensure_writes_are_allowed(sql) if write_query?(sql) | ||
| end | ||
|
|
||
| def raw_execute(sql, name, binds = [], prepare: false, async: false, allow_retry: false, materialize_transactions: true, batch: false) | ||
| log(sql, name, async: async) do | ||
| with_raw_connection(allow_retry: allow_retry, materialize_transactions: materialize_transactions) do |conn| | ||
| result = conn.execute(sql) | ||
| verified! | ||
| handle_warnings(sql) | ||
| result | ||
| end | ||
| end | ||
| end | ||
|
|
||
| # Drains and dispatches any SQL warnings collected during the last | ||
| # execution. No-op by default; adapters that surface DB warnings (e.g. | ||
| # PostgreSQL) override this. Mirrors AbstractAdapter#handle_warnings, which | ||
| # the JDBC query paths bypass since they don't route through #raw_execute. | ||
| def handle_warnings(sql) | ||
| end | ||
|
|
||
| end | ||
| end | ||
| end | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this content generated? I cannot verify all of this without knowing where it came from.