Skip to content

Commit 0d14a5c

Browse files
jpmckinneyclaude
andcommitted
Simplify psycopg requirements guidance #162
Trim the requirements-psycopg section to the page's operational register: drop the per-extra rationale, the runtime-preference exposition, the startup-import note and the boxed admonitions, keeping the extras, the three-file base layout, the pre-commit hooks and the lint.yml line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013mEwtfE72tX2eUWEHJb1Zs
1 parent a03d1c1 commit 0d14a5c

4 files changed

Lines changed: 11 additions & 52 deletions

File tree

docs/docker/django.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ When using the ``gthreads`` worker class, a main thread `handles the heartbeat <
4646
Number of threads
4747
~~~~~~~~~~~~~~~~~
4848

49-
Ensure your code is thread safe. Notably, `psycopg cursors are not thread safe <https://www.psycopg.org/psycopg3/docs/api/cursors.html>`__, though this isn't a concern for typical usage of `Django <https://docs.djangoproject.com/en/5.2/ref/databases/>`__.
49+
Ensure your code is thread safe. Notably, `psycopg cursors are not thread safe <https://www.psycopg.org/psycopg3/docs/advanced/async.html>`__, though this isn't a concern for typical usage of `Django <https://docs.djangoproject.com/en/5.2/ref/databases/>`__.
5050

5151
`When using threads <https://docs.gunicorn.org/en/stable/design.html#how-many-threads>`__, the application is loaded by the worker and some memory is shared between its threads (thus consuming less memory than additional workers would).
5252

docs/python/preferences.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,11 @@ Command-line interface
2929
DataFrames
3030
`Polars <https://pola.rs>`__, unless end-users are unfamiliar (`pandas <https://pandas.pydata.org/docs/>`__).
3131
Object Relational Mapper (ORM)
32-
Django. If you don't need an ORM, use `psycopg <https://www.psycopg.org/psycopg3/docs/>`__ (psycopg 3). Do not use `SQLAlchemy <https://www.sqlalchemy.org/>`__, except with FastAPI or in low-level libraries with limited scope *where an ORM is needed*.
32+
Django. If you don't need an ORM, use `psycopg <https://www.psycopg.org/psycopg3/docs/>`__. Do not use `SQLAlchemy <https://www.sqlalchemy.org/>`__, except with FastAPI or in low-level libraries with limited scope *where an ORM is needed*.
3333

3434
.. note::
3535

36-
Use ``psycopg[c]`` in production and ``psycopg[binary]`` in development. :ref:`See instructions<requirements-psycopg>`.
36+
Use ``psycopg[c]`` in production and ``psycopg[binary]`` in development, `as recommended and preferred <https://www.psycopg.org/psycopg3/docs/basic/install.html#local-installation>`__. :ref:`See instructions<requirements-psycopg>`.
3737

3838
HTTP client
3939
`Requests <https://docs.python-requests.org/en/latest/>`__ for synchronous code and `niquests <https://niquests.readthedocs.io/en/latest/>`__ for asynchronous code, unless a framework uses another, like Scrapy (Twisted).

docs/python/requirements.rst

Lines changed: 7 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,6 @@ The requirements of *applications* (not :doc:`packages<packages>`) are managed b
3737
- ``requirements.txt`` names all direct and indirect requirements needed in the production environment, all locked to specific versions by `uv <https://docs.astral.sh/uv/>`__.
3838
- ``requirements_dev.txt`` names all direct and indirect requirements needed in the development environment, all locked to specific versions by ``uv``.
3939

40-
Projects that install a different build of the same package in production and development – notably :ref:`psycopg<requirements-psycopg>` – add two more files, ``requirements_base.in`` and ``requirements_base.txt``, to hold the shared requirements.
41-
4240
This ensures that:
4341

4442
- All environments use the same versions of production requirements, to ensure consistent and replicable deployments and to avoid errors or surprises during or after deployment due to differences between versions (e.g. a new version of Django requires upgrading application code).
@@ -116,36 +114,18 @@ If the project has a :ref:`requirements_base.in<requirements-psycopg>`, add shar
116114
psycopg
117115
~~~~~~~
118116

119-
`psycopg <https://www.psycopg.org/psycopg3/docs/>`__ (psycopg 3) replaces ``psycopg2``.
120-
121-
``psycopg2`` and ``psycopg2-binary`` were two distributions of the same import name, so installing the binary distribution shadowed a source build. psycopg 3 is instead a **single package with extras**, which install differently-named modules:
122-
123-
``psycopg``
124-
Pure Python. Requires the ``libpq`` library at runtime.
125-
``psycopg[c]``
126-
Compiled against the system ``libpq``. Requires a C compiler and the ``libpq`` development files at install time. This is the fast, faithful successor to a source-built ``psycopg2``.
127-
``psycopg[binary]``
128-
A self-contained wheel that bundles ``libpq``. Requires no build step.
129-
130-
At runtime, psycopg prefers the C implementation, then the binary, then the pure Python implementation. The old approach of inheriting the production lock and adding the binary distribution on top therefore **no longer selects the binary build**.
117+
Use `psycopg <https://www.psycopg.org/psycopg3/docs/>`__ (psycopg 3), not the older ``psycopg2``. It is a single package with extras: use ``psycopg[c]`` (compiled against the system ``libpq``) in production, and ``psycopg[binary]`` (a self-contained wheel) in development, keeping the ``psycopg`` version in sync.
131118

132-
Instead, use ``psycopg[c]`` in production and ``psycopg[binary]`` in development (the build-free equivalent of ``psycopg2-binary``), keeping the core ``psycopg`` version in sync across the locked files.
133-
134-
Three-tier layout
135-
^^^^^^^^^^^^^^^^^^
136-
137-
Because each lock file is compiled independently, shared transitive dependencies can resolve to different versions unless they share a locked base. Split the requirements into three tiers:
119+
Unlike ``psycopg2`` and ``psycopg2-binary``, the extras install different modules, so you can't add the binary build on top of the production lock. Instead, share a locked base across three files:
138120

139121
``requirements_base.in``
140-
The shared direct requirements (everything except psycopg). Compiles to ``requirements_base.txt``.
122+
The requirements shared by both environments (everything except psycopg).
141123
``requirements.in``
142124
``-r requirements_base.txt`` and ``psycopg[c]``.
143125
``requirements_dev.in``
144126
``-r requirements_base.txt``, ``psycopg[binary]`` and the development-only tools.
145127

146-
Both downstream files reference the locked ``requirements_base.txt`` (not ``requirements_base.in``), mirroring how ``requirements_dev.in`` references the locked ``requirements.txt``.
147-
148-
Configure three ``pip-compile`` pre-commit hooks, base first, with ``files:`` patterns so that changing the base re-triggers the dependent locks:
128+
Compile the base first, so that the downstream locks pick up its versions. Order the ``pip-compile`` pre-commit hooks accordingly, and set ``files:`` patterns so that changing the base re-triggers the dependent locks:
149129

150130
.. code-block:: yaml
151131
:caption: .pre-commit-config.yaml
@@ -163,33 +143,21 @@ Configure three ``pip-compile`` pre-commit hooks, base first, with ``files:`` pa
163143
args: [requirements_dev.in, -o, requirements_dev.txt]
164144
files: ^requirements(_base|_dev)?\.(in|txt)$
165145
166-
Add the new file to the :ref:`requirements check<linting-ci>` in ``lint.yml``, so that it is not reported as missing:
146+
Add ``requirements_base.in`` to the :ref:`requirements check<linting-ci>` in ``lint.yml``:
167147

168148
.. code-block:: yaml
169149
:caption: .github/workflows/lint.yml
170150
171151
with:
172152
standard-maintenance-scripts-files: requirements_base.in
173153
174-
.. attention::
175-
176-
Introducing ``requirements_base.txt`` makes the resolver re-pin every shared dependency, unless it is anchored to the existing lock. Compile the base against the pre-change ``requirements.txt``, and do not pass ``-U``/``--upgrade``:
177-
178-
.. code-block:: bash
179-
180-
uv pip compile -c requirements.txt requirements_base.in -o requirements_base.txt
181-
uv pip compile requirements.in -o requirements.txt
182-
uv pip compile requirements_dev.in -o requirements_dev.txt
183-
184-
The committed diff for a psycopg upgrade should touch only psycopg. Reviewers should reject incidental version bumps to other packages.
185-
186154
.. note::
187155

188-
If a module that imports psycopg is loaded unconditionally at startup, the application needs a working psycopg just to run. Because ``requirements.txt`` pins ``psycopg[c]``, installing it requires a build toolchain even in development – and ``pip install "psycopg[binary]" -r requirements.txt`` does not help, because the pinned ``psycopg-c`` must still build. Instead, install the binary build against the base, which contains no psycopg:
156+
When first adding ``requirements_base.txt``, compile it against the existing lock, so that the diff touches only psycopg and not unrelated packages:
189157

190158
.. code-block:: bash
191159
192-
pip install "psycopg[binary]" -r requirements_base.txt
160+
uv pip compile -c requirements.txt requirements_base.in -o requirements_base.txt
193161
194162
Install requirements
195163
--------------------

docs/services/postgresql.rst

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Connect to a database
1010

1111
Connect to the database using a connection string stored in the ``DATABASE_URL`` environment variable.
1212

13-
In Python, connect to the database using `dj-database-url <https://github.com/kennethreitz/dj-database-url#readme>`__ if using :doc:`Django<../python/django>`, or `psycopg <https://www.psycopg.org/psycopg3/docs/api/connections.html#psycopg.Connection.connect>`__ otherwise. See :ref:`requirements<requirements-psycopg>` for how to declare and install psycopg.
13+
In Python, connect to the database using `dj-database-url <https://github.com/kennethreitz/dj-database-url#readme>`__ if using :doc:`Django<../python/django>`, or `psycopg <https://www.psycopg.org/psycopg3/docs/api/connections.html#psycopg.Connection.connect>`__ otherwise.
1414

1515
To set the search path for a PostgreSQL connection, append to the connection string:
1616

@@ -66,15 +66,6 @@ Load (or dump) data
6666

6767
Use the `\copy <https://www.postgresql.org/docs/current/app-psql.html#APP-PSQL-META-COMMANDS-COPY>`__ meta-command instead of the `COPY <https://www.postgresql.org/docs/current/sql-copy.html>`__ command, so that file accessibility and privileges are those of the user, not the server – such that no SQL superuser privileges are required.
6868

69-
In Python, psycopg 3 removed the ``copy_expert()``, ``copy_from()`` and ``copy_to()`` methods. Use the `cursor.copy() <https://www.psycopg.org/psycopg3/docs/basic/copy.html>`__ context manager instead, writing the file in blocks:
70-
71-
.. code-block:: python
72-
73-
with open(filename, "rb") as f:
74-
with cur.copy("COPY mytable (data) FROM stdin") as copy:
75-
while block := f.read(65536):
76-
copy.write(block)
77-
7869
.. _sql-statements:
7970

8071
Construct SQL statements

0 commit comments

Comments
 (0)