Skip to content

Commit 227ccd9

Browse files
aclark4lifeaclark4life
committed
feat: add dbx spec command for spec sync lifecycle management
Adds `dbx spec` with subcommands to automate the manual workflow from CONTRIBUTING.md. Auto-detects both the driver repo and the specifications repo from dbx config, eliminating the manual MDB_SPECS export and cd dance. Commands: - dbx spec sync [SPECS] [-b BLOCK] [--apply-patches] [--dry-run] - dbx spec list - dbx spec patch list [-v] - dbx spec patch create TICKET [FILES] [--dry-run] - dbx spec patch remove TICKET - dbx spec patch apply [--dry-run] Includes 24 tests and docs covering the full workflow including how to triage automated spec sync PRs from mongodb-drivers-pr-bot.
1 parent a334630 commit 227ccd9

5 files changed

Lines changed: 1264 additions & 0 deletions

File tree

‎docs/features/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,4 @@ Features
1212
just-commands
1313
documentation
1414
status
15+
spec-sync

‎docs/features/spec-sync.rst‎

Lines changed: 367 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,367 @@
1+
Spec Sync
2+
=========
3+
4+
The ``dbx spec`` command streamlines syncing spec tests from the `MongoDB specifications repository <https://github.com/mongodb/specifications>`_ into a driver repo, and managing the patch files that exclude tests for not-yet-implemented features.
5+
6+
Background
7+
----------
8+
9+
Driver repos like ``mongo-python-driver`` carry a copy of the spec tests defined in the central ``specifications`` repository. Keeping them in sync requires running a shell script (``resync-specs.sh``) that lives inside ``.evergreen/`` of the driver repo, with the ``MDB_SPECS`` environment variable pointing at a local clone of the specifications repo.
10+
11+
When a spec introduces tests for features that haven't been implemented yet, those tests are excluded via patch files in ``.evergreen/spec-patch/``. After each sync, ``git apply -R`` is run on every ``PYTHON-XXXX.patch`` file to reverse the unwanted changes.
12+
13+
The manual process documented in ``CONTRIBUTING.md`` looks like this:
14+
15+
.. code-block:: bash
16+
17+
# One-time clone of the specs repo (if not already present)
18+
git clone git@github.com:mongodb/specifications.git ~/specifications
19+
20+
# Every time you want to sync
21+
export MDB_SPECS=~/specifications
22+
cd ~/mongo-python-driver/.evergreen
23+
./resync-specs.sh -b "<regex>" spec1 spec2 ...
24+
25+
# Apply patches manually
26+
git apply -R --allow-empty --whitespace=fix .evergreen/spec-patch/*.patch
27+
28+
``dbx spec`` eliminates the manual steps by auto-detecting both repos from your existing dbx config and running the script for you, and ``dbx spec patch`` gives you full lifecycle management over the patch files.
29+
30+
How It Improves on the Manual Workflow
31+
---------------------------------------
32+
33+
+--------------------------------+-----------------------------------------------+------------------------------------------+
34+
| Task | Manual | With ``dbx spec`` |
35+
+================================+===============================================+==========================================+
36+
| Locate the specs repo | Remember/export ``MDB_SPECS`` path | Auto-detected from config |
37+
+--------------------------------+-----------------------------------------------+------------------------------------------+
38+
| Navigate to the script | ``cd ~/mongo-python-driver/.evergreen`` | Not required |
39+
+--------------------------------+-----------------------------------------------+------------------------------------------+
40+
| Sync all specs | ``./resync-specs.sh`` | ``dbx spec sync`` |
41+
+--------------------------------+-----------------------------------------------+------------------------------------------+
42+
| Sync specific specs | ``./resync-specs.sh crud sessions`` | ``dbx spec sync crud sessions`` |
43+
+--------------------------------+-----------------------------------------------+------------------------------------------+
44+
| Block files by pattern | ``./resync-specs.sh -b "unified" crud`` | ``dbx spec sync crud -b "unified"`` |
45+
+--------------------------------+-----------------------------------------------+------------------------------------------+
46+
| Sync and apply patches | Run script, then ``git apply -R ...`` | ``dbx spec sync --apply-patches`` |
47+
+--------------------------------+-----------------------------------------------+------------------------------------------+
48+
| Target a different driver repo | Repeat the ``cd``/``export`` dance | ``dbx spec sync -r django-mongodb-backend`` |
49+
+--------------------------------+-----------------------------------------------+------------------------------------------+
50+
| Preview without running | No built-in option | ``dbx spec sync --dry-run`` |
51+
+--------------------------------+-----------------------------------------------+------------------------------------------+
52+
| Discover available specs | Browse the ``specifications`` repo on disk | ``dbx spec list`` |
53+
+--------------------------------+-----------------------------------------------+------------------------------------------+
54+
| See active patches | ``ls .evergreen/spec-patch/`` | ``dbx spec patch list`` |
55+
+--------------------------------+-----------------------------------------------+------------------------------------------+
56+
| Create a patch file | Manually write/save a git diff | ``dbx spec patch create PYTHON-XXXX`` |
57+
+--------------------------------+-----------------------------------------------+------------------------------------------+
58+
| Remove a resolved patch | ``rm .evergreen/spec-patch/PYTHON-XXXX.patch``| ``dbx spec patch remove PYTHON-XXXX`` |
59+
+--------------------------------+-----------------------------------------------+------------------------------------------+
60+
| Apply all patches | ``git apply -R --allow-empty ...`` | ``dbx spec patch apply`` |
61+
+--------------------------------+-----------------------------------------------+------------------------------------------+
62+
63+
Prerequisites
64+
-------------
65+
66+
Both repos must be cloned locally. If you use the default config, the ``specifications`` repo is part of the ``pymongo`` group:
67+
68+
.. code-block:: bash
69+
70+
# Clone the pymongo group (includes specifications)
71+
dbx clone -g pymongo
72+
73+
Commands
74+
--------
75+
76+
dbx spec sync
77+
~~~~~~~~~~~~~
78+
79+
Runs ``.evergreen/resync-specs.sh`` in the driver repo with ``MDB_SPECS`` pointing at the specifications repo. After syncing, active patches are listed automatically so you know what will be excluded.
80+
81+
.. code-block:: bash
82+
83+
# Sync all specs
84+
dbx spec sync
85+
86+
# Sync specific specs by name
87+
dbx spec sync crud sessions change-streams
88+
89+
# Exclude files matching a regex (passed as -b to resync-specs.sh)
90+
dbx spec sync crud -b "unified"
91+
92+
# Sync and immediately apply all patches in one shot
93+
dbx spec sync crud --apply-patches
94+
95+
# Target a different driver repo
96+
dbx spec sync -r django-mongodb-backend
97+
98+
# Preview the exact command without running it
99+
dbx spec sync crud --dry-run
100+
101+
# Use a custom path for the specifications repo
102+
dbx spec sync --specs-dir ~/my-specs crud
103+
104+
**Options:**
105+
106+
.. code-block:: text
107+
108+
SPECS Spec names to sync (e.g. crud transactions). Syncs all if omitted.
109+
-r, --repo Driver repository to target [default: mongo-python-driver]
110+
-b, --block Regex pattern passed to resync-specs.sh -b to exclude matching files
111+
--specs-dir Path to the MongoDB specifications repo (overrides auto-detection)
112+
--apply-patches Apply all .evergreen/spec-patch files after syncing
113+
--dry-run Print the command that would be run without executing it
114+
115+
dbx spec list
116+
~~~~~~~~~~~~~
117+
118+
Lists available spec directories from the local ``specifications`` repository.
119+
120+
.. code-block:: bash
121+
122+
# List all available specs
123+
dbx spec list
124+
125+
# List specs from a custom path
126+
dbx spec list --specs-dir ~/my-specs
127+
128+
**Example output:**
129+
130+
.. code-block:: text
131+
132+
Specs in ~/Developer/mongodb/specifications:
133+
134+
├── auth
135+
├── change-streams
136+
├── client-side-encryption
137+
├── connection-monitoring-and-pooling
138+
├── crud
139+
├── gridfs
140+
├── load-balancers
141+
├── max-staleness
142+
├── read-write-concern
143+
├── retryable-reads
144+
├── retryable-writes
145+
├── server-discovery-and-monitoring
146+
├── sessions
147+
├── transactions
148+
└── ...
149+
150+
Use the spec names from this output as arguments to ``dbx spec sync``.
151+
152+
dbx spec patch list
153+
~~~~~~~~~~~~~~~~~~~
154+
155+
Lists all active patch files and the test files each one affects. Add ``-v`` to see the individual filenames.
156+
157+
.. code-block:: bash
158+
159+
dbx spec patch list
160+
dbx spec patch list -r django-mongodb-backend
161+
dbx -v spec patch list # shows individual affected files
162+
163+
**Example output:**
164+
165+
.. code-block:: text
166+
167+
Active patches in mongo-python-driver (3):
168+
169+
├── PYTHON-2673 (2 file(s))
170+
├── PYTHON-4261 (1 file(s))
171+
└── PYTHON-5759 (4 file(s))
172+
173+
dbx spec patch create
174+
~~~~~~~~~~~~~~~~~~~~~
175+
176+
Captures the current ``git diff`` into a new ``.evergreen/spec-patch/<ticket>.patch`` file. Run this after a spec sync brings in tests you don't want yet, then revert or edit those files so the diff captures only the unwanted changes.
177+
178+
.. code-block:: bash
179+
180+
# Capture all unstaged changes
181+
dbx spec patch create PYTHON-1234
182+
183+
# Capture changes to specific files only
184+
dbx spec patch create PYTHON-1234 test/crud/foo.json test/crud/bar.json
185+
186+
# Preview the diff that would be saved without writing the file
187+
dbx spec patch create PYTHON-1234 --dry-run
188+
189+
dbx spec patch remove
190+
~~~~~~~~~~~~~~~~~~~~~
191+
192+
Deletes a patch file once the corresponding ticket has been implemented and the tests should be re-enabled.
193+
194+
.. code-block:: bash
195+
196+
dbx spec patch remove PYTHON-1234
197+
dbx spec patch remove PYTHON-1234 -r django-mongodb-backend
198+
199+
dbx spec patch apply
200+
~~~~~~~~~~~~~~~~~~~~
201+
202+
Runs ``git apply -R --allow-empty --whitespace=fix`` on all ``.evergreen/spec-patch/*.patch`` files, matching what ``resync-all-specs.py`` does in CI.
203+
204+
.. code-block:: bash
205+
206+
dbx spec patch apply
207+
dbx spec patch apply -r django-mongodb-backend
208+
dbx spec patch apply --dry-run # list what would be applied
209+
210+
Reviewing Automated Spec Sync PRs
211+
----------------------------------
212+
213+
The ``mongodb-drivers-pr-bot`` opens a weekly pull request (e.g. *[Spec Resync] 04-13-2026*) that runs ``resync-all-specs.py`` against the latest ``specifications`` repo and submits the result. The PR body summarises three things you need to triage:
214+
215+
* **Changed specs** — spec test files that were updated upstream and need review
216+
* **Patch errors** — existing ``.evergreen/spec-patch/`` files that no longer apply cleanly
217+
* **New directories** — spec folders that exist upstream but have no corresponding test directory yet
218+
219+
**Example PR body:**
220+
221+
.. code-block:: text
222+
223+
The following specs were changed:
224+
-change_streams
225+
226+
The following spec syncs encountered errors:
227+
-applying patches
228+
error: patch failed: test/uri_options/client-backpressure-options.json:1
229+
error: test/uri_options/client-backpressure-options.json: patch does not apply
230+
...
231+
232+
The following directories are in the specification repository and not in our test directory:
233+
-client_backpressure
234+
-open_telemetry
235+
236+
Triaging each section with ``dbx spec``
237+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
238+
239+
**1. Reproduce the sync locally**
240+
241+
Check out the PR branch and replay the sync for the specs listed as changed:
242+
243+
.. code-block:: bash
244+
245+
git fetch origin && git checkout <pr-branch>
246+
247+
# Re-sync the changed spec(s) from the PR description
248+
dbx spec sync change-streams
249+
250+
**2. Investigate patch errors**
251+
252+
Patch errors mean a patch file references test content that no longer matches what the spec repo contains — the upstream file may have been renamed, removed, or changed in a way that makes the old diff inapplicable.
253+
254+
.. code-block:: bash
255+
256+
# See which patches exist and what files they cover
257+
dbx spec patch list
258+
259+
# Verbose mode shows the individual filenames — useful for spotting
260+
# stale paths that no longer exist after a rename/removal upstream
261+
dbx -v spec patch list
262+
263+
# Try applying patches to see which ones fail
264+
dbx spec patch apply
265+
266+
Once you know which patch is stale, the fix is one of:
267+
268+
* **File was deleted upstream** — the test the patch was protecting is gone; remove the patch:
269+
270+
.. code-block:: bash
271+
272+
dbx spec patch remove PYTHON-XXXX
273+
274+
* **File was renamed/modified** — re-sync, make the manual edits again, and recreate the patch:
275+
276+
.. code-block:: bash
277+
278+
dbx spec patch remove PYTHON-XXXX
279+
dbx spec sync <spec-name>
280+
# manually revert the unwanted lines
281+
dbx spec patch create PYTHON-XXXX
282+
283+
**3. Evaluate new spec directories**
284+
285+
New directories in the spec repo (``client_backpressure``, ``open_telemetry`` in the example above) mean the upstream team has added a new spec that pymongo doesn't implement yet. For each one, decide:
286+
287+
* **Not yet implemented** — create a JIRA ticket (``PYTHON-XXXX``), sync the spec, and immediately patch out the tests:
288+
289+
.. code-block:: bash
290+
291+
# Pull in the new spec tests
292+
dbx spec sync client-backpressure
293+
294+
# Patch them all out until the feature is implemented
295+
dbx spec patch create PYTHON-XXXX
296+
297+
* **Already implemented** — sync the spec and verify the tests pass without a patch:
298+
299+
.. code-block:: bash
300+
301+
dbx spec sync client-backpressure
302+
303+
**4. Full local repro in one shot**
304+
305+
Once you have the right patches in place, confirm the complete sync applies cleanly:
306+
307+
.. code-block:: bash
308+
309+
dbx spec sync --apply-patches
310+
311+
A clean run with no patch errors means the PR is safe to merge.
312+
313+
Typical Workflows
314+
-----------------
315+
316+
**Full sync with patches in one command**
317+
318+
.. code-block:: bash
319+
320+
dbx sync specifications # pull latest from upstream
321+
dbx spec sync crud --apply-patches # sync and apply patches
322+
323+
**Sync, then handle new unimplemented tests**
324+
325+
.. code-block:: bash
326+
327+
# 1. Sync the spec you're working on
328+
dbx spec sync crud
329+
330+
# 2. Active patches are shown automatically — apply them
331+
dbx spec patch apply
332+
333+
# 3. If new unimplemented tests appeared, revert/edit them and create a patch
334+
dbx spec patch create PYTHON-1234
335+
336+
**Implementing a ticket (removing a patch)**
337+
338+
.. code-block:: bash
339+
340+
# The feature is now implemented — remove its patch file
341+
dbx spec patch remove PYTHON-1234
342+
343+
# Re-sync to verify the tests now pass without the patch
344+
dbx spec sync crud
345+
346+
**See what's excluded before syncing**
347+
348+
.. code-block:: bash
349+
350+
dbx spec patch list # quick overview
351+
dbx -v spec patch list # with individual filenames
352+
353+
Configuration
354+
-------------
355+
356+
No extra configuration is required beyond having ``specifications`` in your repo groups. The spec command finds it automatically:
357+
358+
.. code-block:: toml
359+
360+
[repo.groups.pymongo]
361+
repos = [
362+
"git@github.com:mongodb/specifications.git",
363+
"git@github.com:mongodb/mongo-python-driver.git",
364+
# ...
365+
]
366+
367+
If you keep the specifications repo at a non-standard location, pass ``--specs-dir`` each time or symlink it into your ``base_dir``.

‎src/dbx_python_cli/cli.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
patch,
2121
project,
2222
remove,
23+
spec,
2324
status,
2425
switch,
2526
sync,
@@ -71,6 +72,7 @@ def get_help_text():
7172
app.add_typer(patch.app, name="patch")
7273
app.add_typer(project.app, name="project")
7374
app.add_typer(remove.app, name="remove")
75+
app.add_typer(spec.app, name="spec")
7476
app.add_typer(status.app, name="status")
7577
app.add_typer(switch.app, name="switch")
7678
app.add_typer(sync.app, name="sync")

0 commit comments

Comments
 (0)