Skip to content

fix: backend-agnostic improvements for multi-backend SQL support#2

Open
kushalbakshi wants to merge 1 commit intomasterfrom
fix/backend-agnostic-sql
Open

fix: backend-agnostic improvements for multi-backend SQL support#2
kushalbakshi wants to merge 1 commit intomasterfrom
fix/backend-agnostic-sql

Conversation

@kushalbakshi
Copy link
Copy Markdown
Owner

Improves DataJoint's adapter abstraction to properly support non-MySQL backends. These changes benefit PostgreSQL and enable third-party adapter registration.

Changes:

  • Add make_full_table_name() to adapter ABC — centralizes table name construction (was hardcoded in 5 files)
  • Add foreign_key_action_clause property — FK referential actions via adapter (was hardcoded in declare.py)
  • Use fetchone() instead of rowcount for existence checks — DBAPI2 does not guarantee rowcount for non-DML statements (table.py, schemas.py)
  • Guard transaction methods against empty SQL — supports backends without multi-table transactions (connection.py)
  • Use adapter.quote_identifier() for job metadata columns — fixes PostgreSQL which uses double-quotes (autopopulate.py)
  • Use adapter.split_full_table_name() for name parsing — backend- agnostic instead of manual split (declare.py)
  • Route lineage table check through adapter — catalog-qualified queries for backends with namespaced information_schema (lineage.py)
  • Add "bytes"/"binary" to blob type detection list — supports backends that use BINARY instead of longblob (codecs.py)
  • Extend backend Literal to accept third-party adapter names (settings.py)

@kushalbakshi kushalbakshi force-pushed the fix/backend-agnostic-sql branch from f848623 to 520e2b5 Compare March 30, 2026 16:58
Improves DataJoint's adapter abstraction to properly support non-MySQL
backends. These changes benefit PostgreSQL and enable third-party
adapter registration.

Changes:
- Add make_full_table_name() to adapter ABC — centralizes table name
  construction (was hardcoded in 5 files)
- Add foreign_key_action_clause property — FK referential actions via
  adapter (was hardcoded in declare.py)
- Use fetchone() instead of rowcount for existence checks — DBAPI2 does
  not guarantee rowcount for non-DML statements (table.py, schemas.py)
- Guard transaction methods against empty SQL — supports backends
  without multi-table transactions (connection.py)
- Use adapter.quote_identifier() for job metadata columns — fixes
  PostgreSQL which uses double-quotes (autopopulate.py)
- Use adapter.split_full_table_name() for name parsing — backend-
  agnostic instead of manual split (declare.py)
- Route lineage table check through adapter — catalog-qualified
  queries for backends with namespaced information_schema (lineage.py)
- Add "bytes"/"binary" to blob type detection list — supports backends
  that use BINARY instead of longblob (codecs.py)
- Extend backend Literal to accept third-party adapter names (settings.py)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kushalbakshi kushalbakshi force-pushed the fix/backend-agnostic-sql branch from 520e2b5 to 6dc5d4f Compare March 30, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant