Commit d988b5b
committed
fix(web_fetch): replace hand-written HTML parser with html2text/htmd
- Replace fragile hand-written html_to_text() and html_to_markdown() with
html2text (v0.16) and htmd (v0.5) library calls
- Fixes UTF-8 multi-byte character panic in html_to_text()
- Add test for multi-byte UTF-8 content (Diseño, café, résumé)
- Removes ~80 lines of brittle parsing code1 parent 849ca77 commit d988b5b
3 files changed
Lines changed: 255 additions & 112 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
71 | 75 | | |
72 | 76 | | |
73 | 77 | | |
| |||
0 commit comments