feat(markdown): replace unified.js with custom markdown parser/serializer (#2624)

We've decided to move away from shipping the unified.js ecosystem for parsing and serializing to markdown and instead use our own custom parser/serializer. This should significantly reduce bundle-size, if you need to handle more complex edge cases with markdown, we would recommend preprocessing it beforehand and inserting the result into the editor as HTML (which we have better parsing support for).

Author: nperez0111

docs/content/docs/features/import/markdown.mdx

@@ -15,6 +15,12 @@ imageTitle: Markdown Import
 
 BlockNote can import Markdown content into Block objects. Note that this is considered "lossy", as not all Markdown structures can be entirely represented as BlockNote blocks.
 
+<Callout type={"warning"}>
+    **BlockNote ships a minimal Markdown parser.** It covers the common subset used by most users (CommonMark + GFM basics: headings, paragraphs, lists, task lists, tables, code, blockquotes, links, images, emphasis, strikethrough, hard breaks).
+
+    There are many Markdown specifications (CommonMark, GFM, MDX, Pandoc, and various dialect-specific extensions) and supporting all of them inside a rich text editor is not a goal of BlockNote. **If you need to handle Markdown beyond this minimal subset, parse it to HTML yourself with a parser of your choice (e.g. [`marked`](https://github.com/markedjs/marked), [`markdown-it`](https://github.com/markdown-it/markdown-it), or [`remark`](https://github.com/remarkjs/remark)) and pass the resulting HTML to [`tryParseHTMLToBlocks`](/docs/features/import) instead.** BlockNote's HTML interoperability is much broader, since HTML is the format the editor uses internally for arbitrary pastes.
+</Callout>
+
 ## Markdown to Blocks
 
 Use `tryParseMarkdownToBlocks` to try parsing a Markdown string into `Block` objects:

docs/content/docs/foundations/supported-formats.mdx

@@ -165,6 +165,20 @@ export default function App() {
 
 BlockNote also supports converting to and from Markdown. However, converting to and from Markdown is a **lossy** conversion.
 
+<Callout type="warning">
+  BlockNote ships a **minimal** Markdown parser/serializer that targets the
+  common CommonMark + GFM subset (headings, paragraphs, lists, task lists,
+  tables, code, blockquotes, links, images, emphasis, strikethrough, hard
+  breaks). Supporting every Markdown dialect (CommonMark, GFM, MDX, Pandoc,
+  and various extensions) is not a goal for the editor. If your use case
+  requires Markdown features beyond this subset, **parse the Markdown to
+  HTML yourself** (with a library like [`marked`](https://github.com/markedjs/marked),
+  [`markdown-it`](https://github.com/markdown-it/markdown-it), or
+  [`remark`](https://github.com/remarkjs/remark)) and feed the resulting
+  HTML to `editor.tryParseHTMLToBlocks` — HTML is the format BlockNote uses
+  for arbitrary pastes and has much broader interoperability.
+</Callout>
+
 ### Saving as Markdown
 
 To convert the document to a Markdown string, you can use `editor.blocksToMarkdownLossy()`:

packages/core/package.json

@@ -107,30 +107,18 @@
     "@tiptap/pm": "^3.13.0",
     "emoji-mart": "^5.6.0",
     "fast-deep-equal": "^3.1.3",
-    "hast-util-from-dom": "^5.0.1",
     "lib0": "^0.2.99",
     "prosemirror-highlight": "^0.15.1",
     "prosemirror-model": "^1.25.4",
     "prosemirror-state": "^1.4.4",
     "prosemirror-tables": "^1.8.3",
     "prosemirror-transform": "^1.11.0",
     "prosemirror-view": "^1.41.4",
-    "rehype-format": "^5.0.1",
-    "rehype-parse": "^9.0.1",
-    "rehype-remark": "^10.0.1",
-    "rehype-stringify": "^10.0.1",
-    "remark-gfm": "^4.0.1",
-    "remark-parse": "^11.0.0",
-    "remark-rehype": "^11.1.2",
-    "remark-stringify": "^11.0.0",
-    "unified": "^11.0.5",
-    "unist-util-visit": "^5.0.0",
     "y-prosemirror": "^1.3.7",
     "y-protocols": "^1.0.6",
     "yjs": "^13.6.27"
   },
   "devDependencies": {
-    "@types/hast": "^3.0.4",
     "eslint": "^8.57.1",
     "jsdom": "^29.0.2",
     "rimraf": "^5.0.10",