Skip to content

ENG-1475 Export relations to imported nodes#817

Merged
maparent merged 4 commits into
mainfrom
eng-1475-export-relations-to-imported-nodes
Mar 4, 2026
Merged

ENG-1475 Export relations to imported nodes#817
maparent merged 4 commits into
mainfrom
eng-1475-export-relations-to-imported-nodes

Conversation

@maparent

@maparent maparent commented Feb 23, 2026

Copy link
Copy Markdown
Collaborator

https://linear.app/discourse-graphs/issue/ENG-1475/export-relations-to-imported-nodes

https://www.loom.com/share/527b312dca4f441083f9ae25987e5f57

Code note: the main work was to send the original node's koi identifier in upsertConcept, and have that function resolve the identifier to the original node.


Open with Devin

@linear

linear Bot commented Feb 23, 2026

Copy link
Copy Markdown

@supabase

supabase Bot commented Feb 23, 2026

Copy link
Copy Markdown

Updates to Preview Branch (eng-1475-export-relations-to-imported-nodes) ↗︎

Deployments Status Updated
Database Wed, 04 Mar 2026 17:15:11 UTC
Services Wed, 04 Mar 2026 17:15:11 UTC
APIs Wed, 04 Mar 2026 17:15:11 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks Status Updated
Configurations Wed, 04 Mar 2026 17:15:11 UTC
Migrations Wed, 04 Mar 2026 17:15:11 UTC
Seeding Wed, 04 Mar 2026 17:15:12 UTC
Edge Functions Wed, 04 Mar 2026 17:15:15 UTC

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

@maparent maparent marked this pull request as draft February 23, 2026 15:07
@maparent maparent changed the base branch from main to eng-1344-upload-relation-and-rel-schema February 23, 2026 15:07

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

View 8 additional findings in Devin Review.

Open in Devin Review

@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from ad7af74 to ce78692 Compare February 23, 2026 16:29
@maparent maparent force-pushed the eng-1344-upload-relation-and-rel-schema branch from 30e40b2 to bb38514 Compare February 23, 2026 16:30
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from ce78692 to b87afad Compare February 23, 2026 17:17
@maparent maparent force-pushed the eng-1344-upload-relation-and-rel-schema branch 2 times, most recently from 6784bf7 to 91b4992 Compare February 23, 2026 19:02
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch 3 times, most recently from 9d6b14b to 1b31245 Compare February 24, 2026 20:25
@maparent maparent force-pushed the eng-1344-upload-relation-and-rel-schema branch from cc52d2f to a6f2d9e Compare February 24, 2026 20:26
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch 2 times, most recently from ce150f3 to 3169b1e Compare February 25, 2026 17:30
Base automatically changed from eng-1344-upload-relation-and-rel-schema to main February 25, 2026 18:26
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch 3 times, most recently from 7489026 to 2df080d Compare February 27, 2026 18:16
@maparent maparent changed the base branch from main to eng-1476-repair-lint-ci-task-for-database February 27, 2026 18:17
@maparent maparent marked this pull request as ready for review February 27, 2026 18:17

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 12 additional findings in Devin Review.

Open in Devin Review

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Missing legacy importedFromSpaceUri check in collectDiscourseNodesFromPaths allows imported files to be synced

Same root cause as in shouldSyncFile: collectDiscourseNodesFromPaths only checks frontmatter.importedFromRid but not the legacy frontmatter.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault at apps/obsidian/src/utils/getDiscourseNodes.ts:32-38 correctly checks both fields, but this function was not updated consistently.

Root Cause and Impact

This function is used by syncDiscourseNodeChanges (called from FileChangeListener). During the migration window or if migration fails, any file change event on a file that still has only the legacy importedFromSpaceUri will pass through both shouldSyncFile and collectDiscourseNodesFromPaths guards, causing the imported node to be synced to Supabase as a locally-created node. This is a second guard that should also check the legacy field for defense-in-depth.

(Refers to lines 717-720)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@maparent maparent force-pushed the eng-1476-repair-lint-ci-task-for-database branch from 9121ca2 to 6aa3c39 Compare February 28, 2026 00:12
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from 2df080d to 21b378d Compare February 28, 2026 00:16
devin-ai-integration[bot]

This comment was marked as resolved.

@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from 21b378d to 2df8ee0 Compare February 28, 2026 01:08
devin-ai-integration[bot]

This comment was marked as resolved.

@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from 2df8ee0 to de9dff5 Compare February 28, 2026 03:14

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 18 additional findings in Devin Review.

Open in Devin Review

@maparent maparent force-pushed the eng-1476-repair-lint-ci-task-for-database branch from 6aa3c39 to 44ac414 Compare March 2, 2026 15:02
Base automatically changed from eng-1476-repair-lint-ci-task-for-database to main March 2, 2026 16:09
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from de9dff5 to 0299cbf Compare March 2, 2026 16:34

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 18 additional findings in Devin Review.

Open in Devin Review

Comment thread apps/obsidian/src/utils/syncDgNodesToSupabase.ts
END;
$$;

CREATE OR REPLACE FUNCTION public.rid_or_local_id_to_concept_db_id(rid VARCHAR, default_space_id BIGINT)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i accidentally found a case where this might fail:

  • if we clean up the orphaned node, then the stale relation instance won't be able to resolve to any db id -> cause this function to return null -> cause concept.reference_content -> violate non-null constraints -> unable to upsert ANY concepts.

i discovered this bug after having ran supbase db reset, keeping the vault the same. this means that locally there will be old imported files with frontmatter field importFromRid, that we can't resolve in db. Hence, i received this error:
VM209 plugin:@discou…raph/obsidian:55074 Initial sync failed: Error: upsert_concepts failed: {
"code": "23502",
"details": null,
"hint": null,
"message": "null value in column "reference_content" of relation "Concept" violates not-null constraint"
}

this shouldn't be blocking any syncing operation

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this is a bit of a headache. I corrected the function to allow partial failures; but it means we cannot use the lastSync time anymore!
I think... we need to look at error values, i.e. negative Ids returned by upsertConcept, and possibly remove the offending node/relations. Let's discuss 1-1.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving a trail: we will think about error handling in another PR (ENG-1510.)
The issue of lastSync time may not be as pressing, since the nodes with an error probably cannot be inserted anyway.

@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from 0299cbf to b0251a3 Compare March 4, 2026 14:45

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 18 additional findings in Devin Review.

Open in Devin Review

devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 19 additional findings in Devin Review.

Open in Devin Review

Comment thread packages/database/supabase/migrations/20260221193625_rid_functions.sql Outdated
@maparent maparent merged commit cd30dd5 into main Mar 4, 2026
10 checks passed
@maparent maparent deleted the eng-1475-export-relations-to-imported-nodes branch March 4, 2026 17:17

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 18 additional findings in Devin Review.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants