Add generic zodb-convert support for all storage backends

Generate storage-agnostic convert-import.conf and convert-export.conf
for all non-filestorage backends (RelStorage, PGJsonb, ZEO) using the
generic zodb-convert tool. Filestorage is always the portable format
on the other side of the conversion.

- Add tagmarker parameter to pgjsonb() and zeo() macros in db.conf
- Add db_convert_import/export_* settings to cookiecutter.json
- Create convert-import.conf and convert-export.conf templates
- Update post_gen_project.py for directory creation and file cleanup
- Deprecate RelStorage-specific zodbconvert configs in favor of zodb-convert
- Add migration how-to guide (docs/sources/how-to/migrate-storage.md)
- Update storage-backends explanation and backend how-to guides
- Keep existing relstorage-import/export.conf for backward compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: jensens

CHANGES.md

@@ -2,7 +2,12 @@
 
 ## 2.4.1 (unreleased)
 
-- No changes yet.
+- Feature: Generate generic `convert-import.conf` and `convert-export.conf`
+  for all non-filestorage backends (RelStorage, PGJsonb, ZEO) using the
+  storage-agnostic [zodb-convert](https://pypi.org/project/zodb-convert/)
+  tool. Deprecate RelStorage-specific `zodbconvert` configuration in favor
+  of `zodb-convert`. Add migration how-to guide.
+  [@jensens]
 
 ## 2.4.0 (2026-02-25)
 

cookiecutter.json

@@ -154,6 +154,11 @@
   "db_pgjsonb_blob_cache_dir": "",
   "db_pgjsonb_blob_cache_size": "",
 
+  "db_convert_import_filestorage_location": "",
+  "db_convert_import_blobs_location": "",
+  "db_convert_export_filestorage_location": "",
+  "db_convert_export_blobs_location": "",
+
   "db_z3blobs_enabled": false,
   "db_z3blobs_bucket_name": "",
   "db_z3blobs_s3_prefix": "",

docs/sources/explanation/storage-backends.md

@@ -161,5 +161,12 @@ For most teams, the decision comes down to:
 5. **Need S3 blob storage with direct, relstorage, or zeo?** Enable `db_z3blobs_enabled`.
 
 All four backends support the same ZODB API, so switching between them is a
-configuration change (plus data migration). Your application code does not
-need to change.
+configuration change plus a one-time data migration. The
+[zodb-convert](https://pypi.org/project/zodb-convert/) tool handles this
+migration generically -- it can copy data between any two ZODB-compatible
+storages. Because `cookiecutter-zope-instance` generates a complete
+`zope.conf` for each backend, `zodb-convert` can read the storage
+configuration directly from these files. See {doc}`/how-to/migrate-storage`
+for a step-by-step guide.
+
+Your application code does not need to change.

docs/sources/how-to/configure-pgjsonb.md

@@ -98,6 +98,8 @@ used** with PGJsonb. PGJsonb manages blob storage through its own settings
 
 ## Next steps
 
+- {doc}`migrate-storage` -- Migrate data from an existing storage backend
+  into PGJsonb using `zodb-convert`.
 - {doc}`/reference/database-pgjsonb` -- Full reference for all PGJsonb
   options including connection pool, cache, and S3 settings.
 - {doc}`/explanation/storage-backends` -- Comparison of all storage backends.

docs/sources/how-to/configure-relstorage.md

@@ -104,8 +104,18 @@ for RelStorage command-line utilities:
   settings are provided. Used with `zodbconvert` to import data from a
   filestorage.
 
+:::{tip}
+For migrating data between storage backends, the generic
+[zodb-convert](https://pypi.org/project/zodb-convert/) tool is now the
+recommended approach. It works with all storage backends and can read storage
+configuration directly from `zope.conf` files.
+See {doc}`migrate-storage` for a step-by-step guide.
+:::
+
 ## Next steps
 
+- {doc}`migrate-storage` -- Migrate data between any storage backends using
+  `zodb-convert`.
 - {doc}`/reference/database-relstorage` -- Full reference for all RelStorage
   options including caching, replication, MySQL, Oracle, and SQLite settings.
 - {doc}`/explanation/storage-backends` -- Comparison of all storage backends.

docs/sources/how-to/index.md

@@ -11,6 +11,10 @@ Goal-oriented guides for common tasks.
 - {doc}`configure-zeo`
 - {doc}`configure-z3blobs`
 
+## Storage Migration
+
+- {doc}`migrate-storage`
+
 ## Web Layer
 
 - {doc}`configure-cors`
@@ -31,6 +35,7 @@ configure-relstorage
 configure-pgjsonb
 configure-zeo
 configure-z3blobs
+migrate-storage
 configure-cors
 use-environment-variables
 upgrade-v1-to-v2

docs/sources/how-to/migrate-storage.md

@@ -0,0 +1,140 @@
+# Migrate Data Between Storage Backends
+
+<!-- diataxis: how-to -->
+
+This guide shows how to copy ZODB data from one storage backend to another
+using the generic [zodb-convert](https://pypi.org/project/zodb-convert/) tool.
+
+## Prerequisites
+
+- The `zodb-convert` package installed in your virtual environment
+- The storage-specific packages for both source and destination installed
+  (e.g. `ZODB` for filestorage, `RelStorage` for relstorage, `zodb-pgjsonb`
+  for pgjsonb)
+
+## Approach 1: Generated configuration files
+
+When your instance uses a non-filestorage backend (RelStorage, PGJsonb, or
+ZEO), the cookiecutter generates `convert-import.conf` and
+`convert-export.conf` files in the `etc/` directory. These are ready-made
+configuration files for `zodb-convert` that convert between your configured
+backend and a FileStorage.
+
+### Step 1: Set the conversion paths
+
+In your `instance.yaml`, provide the FileStorage paths for the "other side"
+of the conversion:
+
+```yaml
+default_context:
+    db_storage: pgjsonb
+    db_pgjsonb_dsn: "dbname='zodb' user='zodb' host='localhost' port='5433'"
+
+    # Conversion settings (FileStorage side)
+    db_convert_import_filestorage_location: var/import/Data.fs
+    db_convert_import_blobs_location: var/import/blobs
+    db_convert_export_filestorage_location: var/export/Data.fs
+    db_convert_export_blobs_location: var/export/blobs
+```
+
+These settings work with all non-filestorage backends.
+
+### Step 2: Import data from FileStorage
+
+Place your `Data.fs` and blobs at the import locations, then run:
+
+```bash
+zodb-convert etc/convert-import.conf
+```
+
+### Step 3: Export data to FileStorage
+
+To create a portable FileStorage backup:
+
+```bash
+zodb-convert etc/convert-export.conf
+```
+
+## Approach 2: Using existing zope.conf files
+
+If you have two Zope instances with different storage backends, `zodb-convert`
+can read storage configuration directly from their `zope.conf` files. No
+extra configuration files are needed.
+
+### Step 1: Generate the destination instance
+
+Create a second instance configured for the new storage backend:
+
+```yaml
+# instance-new.yaml
+default_context:
+    db_storage: pgjsonb
+    db_pgjsonb_dsn: "dbname='zodb' user='zodb' host='localhost' port='5433'"
+```
+
+Run cookiecutter to generate:
+
+```bash
+cookiecutter -f --no-input --config-file instance-new.yaml \
+    gh:plone/cookiecutter-zope-instance
+```
+
+### Step 2: Run the conversion
+
+```bash
+zodb-convert \
+    --source-zope-conf /old-instance/etc/zope.conf \
+    --dest-zope-conf /new-instance/etc/zope.conf
+```
+
+### Specifying a named database
+
+If your `zope.conf` defines multiple `<zodb_db>` sections:
+
+```bash
+zodb-convert \
+    --source-zope-conf old/etc/zope.conf --source-db main \
+    --dest-zope-conf new/etc/zope.conf --dest-db main
+```
+
+The default database name is `main`.
+
+## Useful options
+
+- `--dry-run` -- preview what would be copied without writing data
+- `--incremental` -- resume from the last transaction in the destination
+  (useful for large databases or after interruptions)
+- `-v` -- show INFO-level progress; `-vv` for DEBUG-level detail
+
+## Common migration scenarios
+
+| From | To | Notes |
+|---|---|---|
+| direct (filestorage) | relstorage | Classic scale-out migration |
+| direct (filestorage) | pgjsonb | Move to SQL-queryable storage |
+| relstorage | pgjsonb | Switch to JSONB representation |
+| zeo | relstorage | Remove ZEO server dependency |
+| any | direct (filestorage) | Create a portable backup |
+
+All combinations work -- `zodb-convert` is storage-agnostic.
+
+## Configuration reference
+
+| Setting | Default | Description |
+|---|---|---|
+| `db_convert_import_filestorage_location` | *(unset)* | Path to the source FileStorage for import |
+| `db_convert_import_blobs_location` | *(unset)* | Path to the source blob directory for import |
+| `db_convert_export_filestorage_location` | *(unset)* | Path for the destination FileStorage on export |
+| `db_convert_export_blobs_location` | *(unset)* | Path for the destination blob directory on export |
+
+The `convert-import.conf` file is generated when both `db_convert_import_*`
+settings are provided. The `convert-export.conf` file is generated when both
+`db_convert_export_*` settings are provided. Neither file is generated for
+the `direct` (filestorage) backend since it *is* the portable format.
+
+## Next steps
+
+- [zodb-convert on PyPI](https://pypi.org/project/zodb-convert/) -- Full
+  documentation and source code
+- {doc}`/explanation/storage-backends` -- Understanding the trade-offs
+  between backends

docs/sources/reference/database-relstorage.md

@@ -83,6 +83,20 @@ replications. For details read
 RelStorage provides helper scripts for packing (`zodbpack`) and
 import/export from filestorage (`zodbconvert`).
 
+:::{deprecated} 2.5
+The RelStorage-specific `zodbconvert` import/export configuration files
+(`relstorage-import.conf`, `relstorage-export.conf`) and their corresponding
+`db_relstorage_import_*` / `db_relstorage_export_*` settings are deprecated
+in favor of the generic
+[zodb-convert](https://pypi.org/project/zodb-convert/) tool, which works
+with any ZODB storage backend. Use the new `db_convert_*` settings and the
+generated `convert-import.conf` / `convert-export.conf` files instead.
+See {doc}`/how-to/migrate-storage` for details.
+
+The existing settings and generated files continue to work but may be
+removed in a future major version.
+:::
+
 The file `relstorage-pack.conf` for the command line utility `zodbpack` is
 always generated for all RelStorage configurations. For usage information
 read [Packing Or Reference Checking A ZODB Storage: zodbpack](https://relstorage.readthedocs.io/en/latest/zodbpack.html).

hooks/post_gen_project.py

@@ -74,6 +74,29 @@
         if db_relstorage_export_filestorage_location:
             filepath = Path(db_relstorage_export_filestorage_location)
             filepath.parent.mkdir(parents=True, exist_ok=True)
+    if "{{ cookiecutter.db_storage }}" != "direct":
+        # generic convert import directories
+        db_convert_import_blobs = (
+            "{{ cookiecutter.db_convert_import_blobs_location | abspath }}"
+        )
+        if db_convert_import_blobs:
+            Path(db_convert_import_blobs).mkdir(parents=True, exist_ok=True)
+        db_convert_import_fs = (
+            "{{ cookiecutter.db_convert_import_filestorage_location | abspath }}"
+        )
+        if db_convert_import_fs:
+            Path(db_convert_import_fs).parent.mkdir(parents=True, exist_ok=True)
+        # generic convert export directories
+        db_convert_export_blobs = (
+            "{{ cookiecutter.db_convert_export_blobs_location | abspath }}"
+        )
+        if db_convert_export_blobs:
+            Path(db_convert_export_blobs).mkdir(parents=True, exist_ok=True)
+        db_convert_export_fs = (
+            "{{ cookiecutter.db_convert_export_filestorage_location | abspath }}"
+        )
+        if db_convert_export_fs:
+            Path(db_convert_export_fs).parent.mkdir(parents=True, exist_ok=True)
     if "{{ cookiecutter.db_storage }}" == "pgjsonb":
         blob_temp_dir = "{{ cookiecutter.db_pgjsonb_blob_temp_dir }}"
         if blob_temp_dir:
@@ -85,6 +108,10 @@
         z3blobs_cache_dir = "{{ cookiecutter.db_z3blobs_cache_dir }}"
         if z3blobs_cache_dir:
             Path(z3blobs_cache_dir).mkdir(parents=True, exist_ok=True)
+    if "{{ cookiecutter.db_storage }}" == "direct":
+        # cleanup convert files — filestorage doesn't need conversion configs
+        (etc / "convert-import.conf").unlink()
+        (etc / "convert-export.conf").unlink()
     if "{{ cookiecutter.db_storage }}" != "relstorage":
         # cleanup relstorage files if no relstorage is configured
         (etc / "relstorage-export.conf").unlink()

templates/db.conf

@@ -192,9 +192,10 @@
     drop_cache_rather_verify,
     client_label,
     storage,
-    wait
+    wait,
+    tagmarker=""
 )-%}
-    <zeoclient>
+    <zeoclient{{ tagmarker }}>
 {%- if not z3blobs_active %}
         # blobs
         shared-blob-dir {{ "true" if blob_mode == "shared" else "false" }}
@@ -248,7 +249,7 @@
 {%- if wait %}
         wait {{ wait }}
 {%- endif %}
-    </zeoclient>
+    </zeoclient{{ tagmarker }}>
 {%- endmacro %}
 
 # =============================================================================
@@ -271,10 +272,11 @@
     s3_secret_key,
     s3_use_ssl,
     blob_cache_dir,
-    blob_cache_size
+    blob_cache_size,
+    tagmarker=""
 )-%}
     %import zodb_pgjsonb
-    <pgjsonb>
+    <pgjsonb{{ tagmarker }}>
         dsn {{ dsn }}
 {%- if name and name != "pgjsonb" %}
         name {{ name }}
@@ -326,7 +328,7 @@
         blob-cache-size {{ blob_cache_size }}
 {%- endif %}
 {%- endif %}
-    </pgjsonb>
+    </pgjsonb{{ tagmarker }}>
 {%- endmacro %}
 
 # =============================================================================