feat: generate inverse relations in ddl2cpp, and support composite foreign keys - #534
Open
Yaraslaut wants to merge 5 commits into
Open
feat: generate inverse relations in ddl2cpp, and support composite foreign keys#534Yaraslaut wants to merge 5 commits into
Yaraslaut wants to merge 5 commits into
Conversation
Yaraslaut
force-pushed
the
feature/ddl2cpp-inverse-relations
branch
from
August 3, 2026 10:40
ea5ba21 to
ace7de0
Compare
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Yaraslaut
force-pushed
the
feature/ddl2cpp-inverse-relations
branch
from
August 3, 2026 11:15
ace7de0 to
81df3dc
Compare
Yaraslaut
marked this pull request as ready for review
August 3, 2026 11:34
ddl2cpp emitted only BelongsTo, so a generated model was navigable from child
to parent and no further. This adds the inverse and through relations.
A BelongsTo is derivable from the child table alone; every relation pointing the
other way is implied by a foreign key declared on some *other* table. So the
work splits in two: CxxModelPrinter::PlanRelations() answers the schema-wide
question once and is a pure function over the table list (hence testable without
a database), and PrintTable() consumes one table's share of that plan.
Rules, documented in docs/ddl2cpp-relation-generation.md:
- a join table - exactly two single-column foreign keys to two *distinct*
tables, and no columns beyond those keys - becomes a HasManyThrough on both
referenced tables, and the join table itself gets no inverse of its own
- it degrades to HasOneThrough when the join record's far-side key is covered
by a single-column unique index
- every other single-column foreign key becomes a HasMany on the referenced
side
- a uniquely indexed child foreign key makes the relation one-to-one
- composite foreign keys are skipped, exactly as BelongsTo skips them
Two shapes are deliberately *not* collapsed. A join table carrying payload
columns is an association object, so it keeps its own record and a HasMany on
each side, which keeps the payload reachable. A table whose two foreign keys
point at the same table has no far side to hop to, so it stays two HasMany
relations rather than a nonsensical self-through.
Emission detail: an inverse relation names the child while the child's BelongsTo
names the parent, so including both ways would be a cycle. The parent
forward-declares instead, which is sound because every relation stores its
records indirectly (HasMany holds std::vector<std::shared_ptr<Other>>) and needs
no complete type at the point of declaration.
Selectors are emitted whenever the owner is reachable from the same child table
through more than one foreign key, and the member name is then qualified by the
foreign key column so the members do not collide. This is not a nicety: the
reference schema has a table pair joined by 55 foreign keys, and HasMany without
a selector cannot resolve its inverse.
NB: depends on the relation selectors from PR #528 and so branches from
fix/hasmany-inverse-by-type, not from master. On master HasMany takes only
<typename OtherRecord> and this cannot work. There is also no HasOne type in the
library - only HasOneThrough - so a non-through one-to-one is emitted as a
collection with a note in the generated header saying so.
Verified against a large production schema (MS SQL Server 2022 in Docker):
686 tables, 10830 columns, 1860 foreign keys ->
1770 inverse relations generated (1660 HasMany, 104 HasManyThrough,
6 HasOneThrough, alongside 1496 BelongsTo). A slice of that generated output was
then compiled with clang-cl to confirm the emitted relations instantiate.
Tests: 17 new cases in CxxModelRelationTests.cpp covering each rule and each
deliberate non-rule, with fixtures reduced from the shapes actually found in that
schema. Full suite 1311 passed, 1 skipped, 1 failed as expected (the inherited
clang-cl self-reference miscompile).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TVec3YeQg2FZnToiPQwcQH
…ection
A composite foreign key references a composite primary key, so both sides need
the same notion of "a key spans several columns". Under the guiding principle
that every database column is one data member, that means several members marked
PrimaryKey on the referenced side, and several ordinary column members plus one
relation member on the referencing side.
struct Parent {
Field<int32_t, PrimaryKey::AutoAssign, SqlRealName{"part_a"}> partA;
Field<int32_t, PrimaryKey::AutoAssign, SqlRealName{"part_b"}> partB;
};
struct Child {
Field<int32_t, PrimaryKey::AutoAssign> id;
Field<int32_t, SqlRealName{"ref_a"}> refA; // ordinary columns
Field<int32_t, SqlRealName{"ref_b"}> refB;
CompositeForeignKey<Connection<&Child::refA, &Parent::partA>,
Connection<&Child::refB, &Parent::partB>> parent;
};
Why the pairing lives in the type: with two parallel column *lists* matched
positionally, transposing two same-typed columns is silently wrong and
uncatchable. As a Connection<from, into> the pairing is part of the type, so a
transposition is a different Connection - and where the paired columns differ in
value type it does not compile.
BelongsTo is deliberately untouched. It is inseparably a *column* - storage,
FieldWithStorage, a binder over one ODBC column index - as well as a navigator.
Widening it would break the one-member-one-column invariant that
RecordColumnCount (`accum + 1` per column member), every projection builder, the
output-binding loop and the multi-record column-offset arithmetic all rest on.
Under the principle above its column half is also unnecessary, since the columns
already have their own members. So the new relation does only the navigation
half, in the mould of HasOneThrough: no Value()/IsModified(), therefore not
RecordColumnMember, therefore contributing 0 columns and skipped by every
projection automatically. No binder, projection or column-arithmetic code changed.
Ordering is the subtle part, and CompositeKeyOrderingTests.cpp pins it first:
QuerySingle emits one predicate per primary-key member in the *referenced
record's member declaration* order and binds positionally, so passing values in
connection order would bind them to the wrong predicates whenever the two orders
differ - fetching a wrong row rather than failing. OrderedValuesOf() therefore
permutes values into parent-member order using each Connection's IntoMemberIndex,
so connections may be written in any order. Tested with a deliberately scrambled
three-column key.
Loading reuses existing machinery: the loader std::applies the ordered values
into QuerySingle, which already generates `WHERE k1 = ? AND k2 = ?`. No new SQL
generation, no new binding path.
Reflected identity is additive: RecordPrimaryKeyTuple, RecordPrimaryKeyCount,
HasCompositePrimaryKey and GetPrimaryKeyFields() sit beside the single-key
helpers, which are left resolving one member index so no existing caller changes
shape - notably CreateExplicit, which returns RecordPrimaryKeyType.
Deferred, and recorded in docs/composite-keys-design.md rather than silently
skipped: ddl2cpp generation of these declarations; the inverse (HasMany over a
composite relation, needing a list-valued selector); AutoAssign semantics across
several key columns; and confirming the C++26 reflection configuration.
Tests: 22 cases / 124 assertions across CompositeForeignKeyTests.cpp,
CompositeKeyOrderingTests.cpp and CompositeKeyGapTests.cpp - type derivation, the
three static_assert rejections, value extraction, out-of-order and three-column
loading against a live DB, and reflected identity.
Full suite: 1333 passed, 1 skipped, 1 failed as expected (sqlite3, clang-cl).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TVec3YeQg2FZnToiPQwcQH
Ten confirmed findings from the high-effort review of acbc8c03. Two of them
contradicted claims made in that commit's own message and comments; those are
corrected rather than restated.
Correctness:
- AssignAt() permuted values by assigning through a runtime-matched tuple index,
which forces *every* slot's assignment to be well-formed - so any composite key
whose columns did not all share one value type failed to compile. Reproduced in
isolation with a mixed int/string key. OrderedValuesOf now *builds* the ordered
tuple by slot instead: the connection occupying each key position is resolved at
compile time from the Into member indices and its value read directly, so
heterogeneous keys work and AssignAt is gone. The comment claiming the
static_asserts guaranteed equal types was wrong - they only compare types within
each Connection, never across them.
- The loader snapshotted the key values at ConfigureRelationAutoLoading time, so
repointing a foreign key afterwards kept resolving to the previous parent. It
now reads them through the connections at load time, which is what the header
already claimed ("one copy of each value, in the Field that owns the column").
- Neither Inspect() nor LoadRelations() knew about the new relation. Inspect()
reached std::format on it and would not compile for any record holding one;
LoadRelations() had no branch, so an eagerly-loaded record's relation stayed
unloaded and then threw. Both fixed, LoadRelations in both the reflection and
non-reflection halves, via a new LoadCompositeForeignKey().
- The C++26 reflection build was broken: only MemberClassType was guarded, while
the member pointers were still dereferenced with `.*`, invalid on the
std::meta::info that branch supplies. Member access now goes through
Connection::FieldOf(), which is the single place the two modes differ, and field
types come from the pointer type rather than a declval of the owning record.
Validation, all previously silent:
- Two connections naming the same referenced member collapsed the permutation,
leaving a key slot default-constructed against a real predicate.
- A partial key (fewer connections than the referenced key has columns) compiled
and threw an argument-count mismatch at first navigation.
- A connection pairing a member with itself passed every check.
The last two are deferred to first use rather than asserted in the class body: the
declaring record is incomplete while the relation is instantiated as one of its
members, so reflecting over it there is ill-formed - which MSVC diagnosed and
clang-cl did not.
Multi-column AutoAssign:
GenerateAutoAssignPrimaryKey now rejects more than one auto-assigned key member,
because it yields a single value that SetId() writes into every key member. Scoped
to the value types auto-assignment actually generates, so the pre-existing
MultiPkRecord test - two AutoAssign *string* keys, where nothing is generated -
keeps working. Every fixture and both doc examples moved off multi-column
AutoAssign accordingly.
Docs: the design doc no longer says "Status: proposal. Nothing implemented yet."
in the commit that ships the feature, and its rejection table, ordering section and
AutoAssign guidance now match the implementation.
Also found while fixing: clang-cl 22.1.3 miscompiles a record transitively holding
a std::function returning a record that references it back - an impossible stack
frame and a prologue fault, eight frames deep with no runtime recursion. Same
defect already documented for a self-referential BelongsTo in RelationShapeTests.
The identical code compiles and every case passes under MSVC, so the tests are
guarded on the toolchain with a [!shouldfail] placeholder standing in.
Verified:
MSVC cl-release : 1337 passed, 1 skipped, 0 failed (all composite coverage ACTIVE)
clang-cl release : 1321 passed, 1 skipped, 2 failed as expected (both guarded)
Composite tests under MSVC: 124 assertions in 22 cases, all green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TVec3YeQg2FZnToiPQwcQH
The relation-generation notes and one test comment identified the production database, its schema and five of its table/column names. None of that is needed: the generation rules are derived from structural shapes and their counts, which are what the documents actually reason about. Replaces the identifying detail with neutral equivalents - the join-table examples keep their shape (two columns, each a single-column foreign key to a distinct table, composite primary key over exactly those two) under generic names. Also refreshes the status section, which still said the inverse and through relations were never generated, and points at CompositeForeignKey for the composite foreign keys it lists as unrepresentable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TVec3YeQg2FZnToiPQwcQH
Yaraslaut
force-pushed
the
feature/ddl2cpp-inverse-relations
branch
2 times, most recently
from
August 3, 2026 13:42
126864c to
f224a1e
Compare
…s and relation generation Ten findings from a review of the composite-foreign-key and ddl2cpp relation work. Correctness: - QuerySingle(primaryKeys...) and one FirstImpl() overload each had an early `return std::nullopt;` ahead of their final `return record;`, defeating NRVO and risking a stale pointer in the composite-FK auto-loader. Restructured both to a single return statement, and - since CI proved NRVO still isn't reliable for the fuller function body under some build configurations (caught by the coverage build; see below) - additionally switched the composite-FK auto-loader to capture the foreign key values by value at configure time instead of a pointer to the owning record, matching the safe pattern HasMany/BelongsTo already use. - ddl2cpp's join-table cardinality check used the *far* key's uniqueness instead of the *owner* key's - the far key always references exactly one row, so that check was trivially always true. Fixed, and corrected the test that had locked in the backwards expectation, plus the doc wording. - The `PrimaryKey::AutoAssign` collision counter only recognized `value + 1`-style keys, so two auto-assigned SqlGuid keys bypassed the static_assert meant to reject multi-column auto-assignment. - CompositeForeignKey had no operator==/<=>, so it was neither equality_comparable nor (owing to its private members) an aggregate - Reflection::CollectDifferences hard-errors on any record holding one. Added comparison operators, matching HasMany/HasOneThrough/HasManyThrough. - GetPrimaryKeyField() kept overwriting its result for every matching-type primary key member, silently returning the last one instead of the first. Cleanup: - Deduplicated the eager (LoadCompositeForeignKey) and lazy (ConfigureRelationAutoLoading) composite-FK load paths into one LoadCompositeForeignKeyRecord() helper, now taking the already-permuted key tuple rather than the owning record (see the value-capture fix above). - Merged the near-identical HasOneThrough/HasManyThrough codegen branches in CxxModelPrinter, driven by the template name string. - Replaced two C-style index loops in CompositeForeignKey with views::iota. - Rebuilt GetPrimaryKeyFields() on the same compile-time tuple_cat machinery RecordPrimaryKeyTupleHelper already uses, dropping an O(K^2) runtime scan. - Replaced byName()'s linear table scan in PlanRelations with a hash map. CI fixups (found via `gh pr checks` + fetched logs, all within this PR's own diff against master - none pre-exist there): - clang-tidy: an unchecked-optional-access and a dead-store finding in tests, and PlanRelations exceeding the cognitive-complexity threshold (fixed by the CxxModelPrinter dedup above). - Doxygen coverage: wrapped a decltype-of-invoked-generic-lambda type alias that at least one Doxygen version misparses in `\cond`, replaced a few `\ref`s Doxygen couldn't resolve with plain code font, and documented `PlannedRelation::kind` and `CompositeForeignKey::Loader`'s members (the latter newly required once the struct itself gained a doc comment). - Regenerated the `src/examples/test_chinook/entities/*.hpp` golden reference files against a live MS SQL Server + Chinook dataset: they had never been updated after ddl2cpp's relation-generation feature landed earlier in this branch, so every file was missing the HasMany/HasManyThrough members and forward declarations the generator has produced all along. - A real ddl2cpp bug this regeneration surfaced: a relation member can end up named exactly like its own referenced (forward-declared) struct - e.g. `Light::HasMany<Album> Album;` inside `struct Artist`. Legal C++, but GCC's -Wchanges-meaning rejects it under -Werror once a real GCC build (not just Clang) actually compiled the regenerated headers. Fixed by reserving referenced/through struct names in the same per-table uniquing map used for column/member names, so a colliding relation member gets suffixed instead; added a regression test. Also fixed, unrelated to the review but required for a local build under this toolchain: two dependent-name lookups needing an explicit `template` keyword, and a spurious unused-lambda-capture warning-as-error. Left alone (pre-existing on this branch before this change, or infrastructure): docs/sqlquery.md's two Doxygen markdown-list warnings (present on master too); the C++26 reflection build's composite-FK errors (predate this change, need reflection-mode expertise); a one-off local Postgres-container anomaly in an unrelated self-referencing-relation test (CI's own PostgreSQL leg passes). Verified: clang-debug (ASan/UBSan) against SQLite3, MS SQL Server 2022, and PostgreSQL (all three via Docker where applicable) - full suite green; additionally reproduced and re-verified the composite-FK regression test under a `clang-coverage`-instrumented build, which is what first caught the NRVO gap above; ddl2cpp/chinook regenerated and rebuilt against a live MSSQL Chinook dataset with zero diff against the golden files. GCC and the C++20-modules/C++26-reflection configurations were not available in this environment (verified only that the C++26 reflection failures are unchanged from before this push). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Yaraslaut
force-pushed
the
feature/ddl2cpp-inverse-relations
branch
from
August 3, 2026 13:58
f224a1e to
13f8616
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two related pieces of work, both driven by what a large production schema actually contains.
1.
ddl2cppgenerates inverse and through relationsPreviously only
Light::BelongsTowas emitted, so a generated model was navigable from child toparent and no further.
CxxModelPrinter::PlanRelations()now answers the schema-wide question — it isa pure function over the table list, so it is testable without a database — and
PrintTable()consumes one table's share.
Rules, documented in
docs/ddl2cpp-relation-generation.md:columns) becomes a
HasManyThroughon both referenced tables, and gets no inverse of its ownHasOneThroughwhen the owner side's key is uniquely indexedHasManyon the referenced sideBelongsToskips themTwo shapes are deliberately not collapsed. A join table carrying payload columns is an association
object, so it keeps its own record plus a
HasManyon each side — otherwise the payload becomesunreachable. A table whose two foreign keys point at the same table has no far side to hop to, so it
stays two
HasManyrelations rather than a nonsensical self-through.Emission detail: an inverse relation names the child while the child's
BelongsTonames the parent,so including both ways would be a cycle. The parent forward-declares instead, which is sound because
every relation stores its records indirectly. A relation member is also never allowed to collide with
the referenced (non-self) struct's own name — legal C++, but GCC's
-Wchanges-meaningrejects it under-Werror, and it is genuinely confusing besides — soddl2cppreserves that name against thegenerated struct before picking the member's own.
2. Composite foreign keys:
CompositeForeignKey<Connection<...>, ...>A composite foreign key references a composite primary key, so both sides need the same notion of "a
key spans several columns". Under the principle that every database column is one data member,
that means several members marked
PrimaryKeyon the referenced side, and several ordinary columnmembers plus one relation member on the referencing side:
Why the pairing lives in the type. With two parallel column lists matched positionally,
transposing two same-typed columns is silently wrong and uncatchable. As a
Connection<from, into>thepairing is part of the type — a transposition is a different
Connection, and where the paired columnsdiffer in value type it does not compile.
BelongsTois deliberately untouched. It is inseparably a column — storage,FieldWithStorage,a binder over one ODBC column index — as well as a navigator. Widening it would break the
"one member ⇒ one column" invariant that
RecordColumnCount, every projection builder, theoutput-binding loop and the multi-record column-offset arithmetic all rest on. So the new relation does
only the navigation half, in the mould of
HasOneThrough: noValue()/IsModified(), therefore notRecordColumnMember, therefore contributing 0 columns and skipped by every projection automatically.No binder, projection or column-arithmetic code changed.
Ordering was settled by test first.
CompositeKeyOrderingTests.cppestablishes thatQuerySingleemits one predicate per key member in the referenced record's member declaration order and binds
positionally — so a transposed argument pair returns a wrong row, not an error.
OrderedValuesOftherefore builds the ordered tuple by slot, resolving each position's connection at compile time, so
connections may be written in any order.
Reflected identity is additive:
RecordPrimaryKeyTuple,RecordPrimaryKeyCount,HasCompositePrimaryKeyandGetPrimaryKeyFields()sit beside the single-key helpers, which keepresolving one member index so no existing caller changes shape — notably
CreateExplicit, whichreturns
RecordPrimaryKeyType.The lazy auto-loader captures the foreign key's values, not a pointer back into the owning record:
QuerySingle's NRVO is not standard-mandated and does not reliably apply to its full (non-trivial)body in practice, so a captured pointer into a
std::optional<Record>returned by value could dangleby the time the loader runs. Confirmed as a real bug (not a false positive) with a coverage-instrumented
build before switching to value capture — see
CompositeForeignKeyTests.cpp's"lazy auto-loader survives QuerySingle's return by value" regression test.
Review fixes and hardening (this update)
HasOneThroughcardinalitydecision, and the GCC member-naming collision above.
GetPrimaryKeyField/GetPrimaryKeyFieldsno longer silently return the wrong field and no longerscan
O(K²).GenerateAutoAssignPrimaryKeynow counts auto-assigned GUID keys, not just incrementable ones.operator<=>/operator==toCompositeForeignKey/Connection/Loaderso composite-FKrelations participate in
CollectDifferenceslike every other relation type.test_chinookgolden-file entities (never refreshed since relation generationlanded on this branch) against a live Chinook database, and added a regression test asserting a
relation member never collides with its own referenced struct's name.
LIGHTWEIGHT_CXX26_REFLECTION):Utils.hpp'sMemberIndexOfcalled
parent_ofunqualified (undeclared under that build), and the composite-FK test fixturespassed raw
&Class::fieldpointer-to-member values intoConnection<...>instead of the existingMember(x)macro (^^xunder reflection,&xotherwise) thatBelongsTo's own fixtures alreadyuse —
std::meta::parent_ofneeds an actualstd::meta::info, not a member pointer. Both fixed;the reflection CI job is green.
Risk
QuerySingle, so no new SQL isgenerated and no new binding path exists.
single-key identity helpers are unchanged.
GenerateAutoAssignPrimaryKeynow rejects more than oneauto-assigned key member, because auto-assignment yields a single value that
SetId()writes intoevery key member. Scoped to the value types it actually generates for (GUIDs and incrementable
ones), so the pre-existing two-
AutoAssign-string-keys case still compiles.Warning
clang-cl 22.1.3 miscompiles records holding a
CompositeForeignKey— the same defect alreadydocumented for a self-referential
BelongsTo: an impossible stack frame and a fault in the functionprologue, eight frames deep with no runtime recursion. The identical code compiles and every case
passes under MSVC
cl, soCompositeForeignKeyTests.cppis excluded from compilation on thattoolchain with a
[!shouldfail]placeholder standing in —Windows-clangcl-debugis green becauseof that guard, not because the defect is fixed. Reducing this for an upstream LLVM report is still
outstanding.
Testing
Full suite run against every supported database, plus the compiler/configuration matrix CI actually
uses — see
AGENT.md's "Testing every database is not enough" section for why both matter:clang-debug(ASan/UBSan/TSan, clang-tidy)gcc-releasecl-debugLIGHTWEIGHT_CXX26_REFLECTION(clang, experimental)ddl2cppwas also run against the real 686-table schema: 1770 relations generated (1660HasMany, 104HasManyThrough, 6HasOneThrough, alongside 1496BelongsTo), and a slice of thegenerated headers was compiled to confirm the emitted relations instantiate.
test_chinook's goldenfiles were regenerated against a live database and diff clean.
Codecov: patch coverage 95.81% (9/215 new lines uncovered — mostly hard-to-hit defensive paths, e.g.
the "relation genuinely not loadable" throw in
CompositeForeignKey::RequireLoaded()); projectcoverage 91.05% → 91.11%.
Deferred, tracked in the docs rather than silently skipped
ddl2cppemittingCompositeForeignKeydeclarations — mechanical now that the spelling is fixed,since the schema reader already reports both ordered column lists
HasManyneeding a list-valued selector)AutoAssign/ServerSideAutoIncrementsemantics across several key columns🤖 Generated with Claude Code
https://claude.ai/code/session_01TVec3YeQg2FZnToiPQwcQH