Skip to content

Accommodate ORM schema reorg for search/browse queries#207

Draft
skearnes wants to merge 2 commits into
open-reaction-database:mainfrom
skearnes:accommodate-orm-schema-reorg
Draft

Accommodate ORM schema reorg for search/browse queries#207
skearnes wants to merge 2 commits into
open-reaction-database:mainfrom
skearnes:accommodate-orm-schema-reorg

Conversation

@skearnes

@skearnes skearnes commented Jun 29, 2026

Copy link
Copy Markdown
Member

Summary

ord-schema's upcoming 0.8 release reorganizes the ORM into role-based schemas (open-reaction-database/ord-schema#859) and splits dataset loading into separate ingest and derivation stages (open-reaction-database/ord-schema#860, open-reaction-database/ord-schema#861):

  • public.reactions — the served Reaction proto payload, keyed by reaction_id (moved off ord.reaction.proto).
  • public.datasets — per-dataset metadata num_reactions / submitted_at (moved off ord.dataset).
  • derived.* — generated SMILES + rdkit_*_id links: reaction_smiles, compound_smiles, product_compound_smiles (moved off ord.reaction / ord.compound / ord.product_compound).
  • ord.* — now a pure search index (decomposed message tables only).
  • Loading is now two stagesadd_dataset / add_parquet_dataset ingest ord.* + public.*, and update_derived_data populates the derived.* SMILES / RDKit links / reaction classes.

This PR updates the search/browse queries to read from the new locations and updates the test-fixture loader to the two-stage API.

Changes

  • ReactionSmartsQuery / ReactionComponentQuery (queries.py) — join through derived.reaction_smiles / derived.compound_smiles / derived.product_compound_smiles to reach the rdkit_*_id links, replacing the dropped ord.reaction.rdkit_reaction_id / compound.rdkit_mol_id columns.
  • fetch_reactions — reads proto from public.reactions.
  • _fetch_dataset_most_used_smiles — joins the derived SMILES tables for the smiles column (new smiles_table / smiles_foreign_key params).
  • get_datasets / get_dataset (search.py) — join public.datasets for num_reactions / submitted_at.
  • setup_test_postgres (testing.py) — the removed add_dataset(..., rdkit_cartridge=...) keyword is gone; ingest with add_dataset(dataset, session) then derive with update_derived_data(dataset.dataset_id, session, rdkit_cartridge=...), mirroring the ord-schema ORM conftest and preserving the rdkit-cartridge gating.
  • queries_test.py — the _max_similarities helper mirrors _mols_join, updated to the same derived-table joins.
  • pyproject.toml — bump the pin to ord-schema[orm]>=0.8,<0.9.

Reaction classes (ord-schema#854)

No changes needed: ord-interface does not read the reaction class/name fields anywhere (no query, search, or frontend reference), so #854 — and its move to derived.reaction_classes in #859 — is transparent here.

Correctness note

All joins to the derived.* tables are INNER, which preserves the prior NULL-column filtering: update_derived_tables inserts a derived row only when a SMILES can be generated, so un-derivable entities are dropped exactly as the old rdkit_*_id IS NULL / smiles IS NOT NULL paths did. The API response shapes (DatasetInfo, QueryResult) are unchanged, so no frontend changes are needed.

Blocked on release ⚠️

Draft until ord-schema 0.8.0 is published:

  • uv.lock still pins 0.7.1 and must be regenerated (uv lock) once 0.8.0 is on PyPI.
  • CI (and the local ty check pre-commit hook) cannot resolve >=0.8 until then; these commits were pushed skipping only the ty-check hook for that reason — ruff check, ruff format, and addlicense all pass.

🤖 Generated with Claude Code

skearnes and others added 2 commits June 28, 2026 20:32
…uted)

ord-schema's upcoming 0.8 release reorganizes the ORM into role-based schemas:
the served Reaction proto moves to public.reactions, per-dataset metadata
(num_reactions, submitted_at) to public.datasets, and the generated SMILES plus
rdkit_*_id links to the derived.* tables (reaction_smiles, compound_smiles,
product_compound_smiles). The ord.* schema is now a pure search index.

Update the search/browse queries to read from the new locations:
- ReactionSmartsQuery / ReactionComponentQuery join ord.reaction (and the
  compound tables) through the derived SMILES tables to reach the rdkit_*_id
  links instead of the dropped ord.reaction.rdkit_reaction_id /
  compound.rdkit_mol_id columns.
- fetch_reactions reads proto from public.reactions.
- The dataset-stats aggregation joins the derived SMILES tables for the SMILES.
- get_datasets / get_dataset join public.datasets for num_reactions /
  submitted_at.

All joins to the derived tables are INNER, mirroring the prior NULL-column
filtering: update_derived_tables inserts a derived row only when a SMILES can be
generated, so un-derivable entities are dropped exactly as before.

Bump the ord-schema pin to >=0.8,<0.9. uv.lock must be regenerated once 0.8.0
is published.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ord-schema 0.8 splits dataset loading into ingest (add_dataset) and derivation
(update_derived_data); update the test-postgres helper to call both, mirroring the
ord-schema ORM conftest and preserving the rdkit-cartridge gating.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant