File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -22,10 +22,10 @@ pip install -i [https://test.pypi.org/simple/](https://test.pypi.org/simple/) sm
2222Run the chunk command to process a file and generate a JSONL output.
2323
2424smartchunk chunk docs/README.md \
25- --mode markdown \
26- --max-tokens 700 \
27- --dedupe \
28- --out out/ chunks.jsonl
25+ fetch Fetch, parse, and chunk content directly from a URL in one go. │
26+ │ chunk Chunk a local file and print/save chunks. │
27+ │ compare Compare SmartChunker with a naive fixed-size chunker. │
28+ │ stream Stream chunks from STDIN in near-real-time.
2929
30303 . Compare with a Naive Splitter
3131Use the compare command to generate an HTML report that visually shows the difference between SmartChunk and a standard token-based splitter.
@@ -64,7 +64,7 @@ Options:
6464License
6565This project is licensed under the MIT License. See the LICENSE file for details.
6666
67- (IN SIMPLE:-)
67+ # (IN SIMPLE:-)
6868SmartChunk 🧩
6969An intelligent, structure-aware, and semantic-aware chunking tool for RAG pipelines.
7070
You can’t perform that action at this time.
0 commit comments