feat(web):Add web page to Markdown conversion tool#1048
Closed
wondywang wants to merge 1 commit intojackwener:mainfrom
Closed
feat(web):Add web page to Markdown conversion tool#1048wondywang wants to merge 1 commit intojackwener:mainfrom
wondywang wants to merge 1 commit intojackwener:mainfrom
Conversation
This file implements a CLI tool to convert web pages to Markdown format, utilizing the Readability library for content extraction and Turndown for HTML-to-Markdown conversion. It includes options for outputting to a file or stdout and handles various media URLs.
wondywang
changed the title
5 tasks
Owner
|
感谢你提交这个 PR！@wondywang 🙏 经过 review，我们决定不引入独立的
不保留的部分（已在
所以这个 PR 按当前形态是多余的，但你的思路完全对——功能已通过 #1146 合入。Close by superseded-by。 |
jackwener
added a commit
that referenced
this pull request
Apr 22, 2026
* feat(web,download): absorb #1048 media + --stdout into web read Distill the useful pieces of the abandoned PR #1048 (`web md`) into the existing shared pipeline instead of introducing a parallel command: - Turndown rules for <video> / <audio> / <iframe>. Video and audio are emitted as inline HTML so renderers that support it keep playback, and iframes degrade to markdown links (title + src) so embedded content (YouTube, CodePen, …) stays reachable. `iframe` moves out of STRIPPED_TAGS since it's now handled explicitly. - `stdout` option on ArticleDownloadOptions: writes the full markdown to process.stdout, skips image download + mkdir + file write, and reports saved='-'. Remote image URLs stay intact so piped output is self-contained. - `web read --stdout` wires the above through. - Lazy-load src rewrite: the extractor now promotes data-src / data-original / data-lazy-src / data-srcset onto `src` before the HTML is frozen, so the markdown body and the image-download list reference the same URL (previously a page with placeholder.gif + data-src produced broken image links in the output). Nothing in #1048 that overlapped with the already-merged #1143 hardening was kept — no new Readability wiring, no duplicate Turndown config, no new command. * fix(web): keep stdout streaming output clean * fix(tests): update iframe e2e assertion and drop relative src import - article-extract e2e fixture test: iframe now converts to a markdown link instead of being stripped, so assert the YouTube embed link survives rather than asserting its absence. - clis/web/read.test.js: replace vi.importActual('../../src/registry.js') with a direct __test__.command export from read.js; the relative import into src/ tripped the package-exports adapter guardrail.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This file implements a CLI tool to convert web pages to Markdown format, utilizing the Readability library for content extraction and Turndown for HTML-to-Markdown conversion. It includes options for outputting to a file or stdout and handles various media URLs.
Description
This PR introduces a new, dedicated CLI plugin for converting web pages into high-fidelity Markdown. It is designed to address the limitations of the existing web read command, which currently lacks robust support for rich content like embedded media and complex tables.
This plugin ensures a more complete conversion by strictly preserving the original URLs of images and videos and accurately rendering HTML tables into Markdown format, making it suitable for archiving or processing content-heavy web pages.
Motivation
The primary motivation for this feature is to overcome the shortcomings of the current web read functionality. While web read is useful for extracting plain text, it often fails to capture the full context of a webpage, specifically:
Multimedia Content: Images and videos are frequently omitted or their links are lost.
Tabular Data: HTML tables are not correctly converted, leading to a loss of structured information.
This plugin fills that gap by providing a specialized tool for users who need to preserve the integrity of rich media and structured data when converting HTML to Markdown.
Web MD — Convert any web page to Markdown with enhanced quality.
Uses @mozilla/readability for content extraction and Turndown + GFM
or HTML-to-Markdown conversion. Preserves image/video URLs.
Usage:
Related issue: None
Type of Change