[recipes] Preserve full frontmatter in obsidian-vault-import metadata

[dhanjit]

Contribution Type

• Recipe (/recipes)

This modifies an existing recipe (recipes/obsidian-vault-import); it does not introduce a new recipe folder, so there is no new README.md / metadata.json to ship.

What does this do?

This fixes a standard information-loss problem in the Obsidian vault import recipe: any user-defined frontmatter fields are silently dropped on ingestion.

import-obsidian.py parses YAML frontmatter into a Python meta dict but only persists 6 hardcoded keys (source, title, folder, tags, date, wikilinks) to the row's metadata JSONB column. Everything else — custom fields, plugin-generated metadata, structured properties, references — never makes it into the database. The information is in the source file, it's parsed correctly, and then it's thrown away.

This PR adds a frontmatter side-car key under metadata carrying a JSON-safe deep copy of every parsed frontmatter key/value. A new private helper _jsonify_frontmatter converts datetime.datetime / datetime.date values (which YAML auto-parses from date: YYYY-MM-DD) to ISO strings so they're safe for the JSONB column, and recurses into nested dicts and lists.

All existing top-level metadata keys remain unchanged. Existing readers that look at metadata.source, metadata.title, metadata.date, etc. are unaffected. The side-car simply preserves everything else under metadata.frontmatter for downstream queries like metadata->'frontmatter'->>'<custom_key>'.

Example

Input note with a mix of standard and custom frontmatter:

---
title: My Custom Title
date: 2024-05-13
tags: [tag-a, tag-b]
status: active
priority: 2
authors: [Alice, Bob]
external_id: abc-123
nested:
  key: value
  list: [1, 2, 3]
---

Before this PR, only tags and date would survive into metadata. After:

 {
   "source": "obsidian",
   "title": "note",
   "folder": "",
   "tags": ["tag-a", "tag-b"],
   "date": "2024-05-13",
-  "wikilinks": []
+  "wikilinks": [],
+  "frontmatter": {
+    "title": "My Custom Title",
+    "date": "2024-05-13",
+    "tags": ["tag-a", "tag-b"],
+    "status": "active",
+    "priority": 2,
+    "authors": ["Alice", "Bob"],
+    "external_id": "abc-123",
+    "nested": {"key": "value", "list": [1, 2, 3]}
+  }
 }

Note that metadata.frontmatter.date is the ISO string "2024-05-13" rather than a datetime.date object — the helper handles this so the row stays JSONB-insertable.

Requirements

No new dependencies. Uses stdlib datetime only.

No schema migration needed — thoughts.metadata is already jsonb.

Backward compatibility

• Existing top-level metadata keys are unchanged. No reader needs to update.
• Existing rows in any user's Supabase do not get a frontmatter key retroactively — only new inserts (and re-imports of modified notes) populate it.
• The recipe's content-hash dedup means re-running on the same unchanged vault skips rows; users who want to backfill can --force or modify notes.

Testing

Verified with a manual smoke test against a fixture vault containing a note with mixed-type frontmatter (strings, ints, lists, nested dicts, a YAML date):

• Custom frontmatter fields land verbatim under metadata.frontmatter.
• Standard top-level keys (source, title, folder, tags, date, wikilinks) are unchanged.
• A datetime.date value (date: 2024-05-13) becomes the ISO string "2024-05-13" in metadata.frontmatter.date.
• Nested dicts and lists are deep-copied (mutating the result would not affect note['meta']).
• json.dumps(metadata) round-trips without error — no non-JSON-native types remain.

End-to-end python import-obsidian.py /path/to/fixture-vault --dry-run --verbose runs to completion with the change applied.

Checklist

• I've read CONTRIBUTING.md
• I tested this on my own Open Brain instance (fixture vault, dry-run + harness)
• No credentials, API keys, or secrets are included
• No new dependencies introduced
• No schema migration required
• My contribution has a README.md — N/A; this modifies an existing recipe and is not a new contribution folder
• My metadata.json has all required fields — N/A; existing recipe's metadata is unchanged

[github-actions]

Hey @dhanjit — welcome to Open Brain Source! 👋

Thanks for submitting your first PR. The automated review will run shortly and check things like metadata, folder structure, and README completeness. If anything needs fixing, the review comment will tell you exactly what.

Once the automated checks pass, a human admin will review for quality and clarity. Expect a response within a few days.

If you have questions, check out CONTRIBUTING.md or open an issue.

[alanshurafa]

Thanks for the contribution. This is a real bug fix — YAML frontmatter with date: values parses into Python datetime / date objects, which Supabase's JSONB column rejects. The _jsonify_frontmatter helper converting them to ISO strings, recursing into nested dicts and lists, is the correct fix. Recommend maintainer review.

— Alan (community reviewer; non-binding)