Skip to content

Commit f54feb1

Browse files
committed
add markdown output
1 parent 0af3fbd commit f54feb1

3 files changed

Lines changed: 454 additions & 5 deletions

File tree

Readme.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ A simple, efficient Python client for [GROBID](https://github.com/kermitt2/grobi
3131
- **Coordinate Extraction**: Optional PDF coordinate extraction for precise element positioning
3232
- **Sentence Segmentation**: Layout-aware sentence segmentation capabilities
3333
- **JSON Output**: Convert TEI XML output to structured JSON format with CORD-19-like structure
34+
- **Markdown Output**: Convert TEI XML output to clean Markdown format with structured sections
3435

3536
## 📋 Prerequisites
3637

@@ -134,6 +135,7 @@ grobid_client [OPTIONS] SERVICE
134135
| `--segmentSentences` | Segment sentences with coordinates |
135136
| `--flavor` | Processing flavor for fulltext extraction |
136137
| `--json` | Convert TEI output to JSON format |
138+
| `--markdown` | Convert TEI output to Markdown format |
137139

138140
#### Examples
139141

@@ -147,6 +149,9 @@ grobid_client --input ~/pdfs --output ~/tei --n 20 --teiCoordinates processFullt
147149
# Process with JSON output
148150
grobid_client --input ~/pdfs --output ~/results --json processFulltextDocument
149151

152+
# Process with Markdown output
153+
grobid_client --input ~/pdfs --output ~/results --markdown processFulltextDocument
154+
150155
# Process citations with custom server
151156
grobid_client --server https://grobid.example.com --input ~/citations.txt processCitationList
152157

@@ -202,6 +207,14 @@ client.process(
202207
json_output=True
203208
)
204209

210+
# Process with Markdown output
211+
client.process(
212+
service="processFulltextDocument",
213+
input_path="/path/to/pdfs",
214+
output_path="/path/to/output",
215+
markdown_output=True
216+
)
217+
205218
# Process citation lists
206219
client.process(
207220
service="processCitationList",
@@ -329,6 +342,75 @@ client.process(
329342
> [!NOTE]
330343
> When using `--json`, the `--force` flag only checks for existing TEI files. If a TEI file is rewritten (due to `--force`), the corresponding JSON file is automatically rewritten as well.
331344
345+
### Markdown Output Format
346+
347+
When using the `--markdown` flag, the client converts TEI XML output to a clean, readable Markdown format. This provides:
348+
349+
- **Structured Sections**: Title, Authors, Affiliations, Publication Date, Fulltext, Annex, and References
350+
- **Clean Formatting**: Human-readable format suitable for documentation and sharing
351+
- **Preserved Content**: All text content with proper section organization
352+
- **Reference Formatting**: Bibliographic references in a readable format
353+
354+
#### Markdown Structure
355+
356+
The generated Markdown follows this structure:
357+
358+
```markdown
359+
# Document Title
360+
361+
## Authors
362+
- Author Name 1
363+
- Author Name 2
364+
365+
## Affiliations
366+
- Affiliation 1
367+
- Affiliation 2
368+
369+
## Publication Date
370+
January 1, 2023
371+
372+
## Fulltext
373+
### Introduction
374+
Content of the introduction section...
375+
376+
### Methods
377+
Content of the methods section...
378+
379+
## Annex
380+
### Acknowledgements
381+
Acknowledgement text...
382+
383+
### Competing Interests
384+
Competing interests statement...
385+
386+
## References
387+
**[1]** Paper Title. *Author Name*. *Journal Name* (2023).
388+
**[2]** Another Paper. *Author et al.*. *Conference* (2022).
389+
```
390+
391+
#### Usage Examples
392+
393+
```bash
394+
# Generate both TEI and Markdown outputs
395+
grobid_client --input pdfs/ --output results/ --markdown processFulltextDocument
396+
397+
# Markdown output with coordinates and sentence segmentation
398+
grobid_client --input pdfs/ --output results/ --markdown --teiCoordinates --segmentSentences processFulltextDocument
399+
```
400+
401+
```python
402+
# Python library usage
403+
client.process(
404+
service="processFulltextDocument",
405+
input_path="/path/to/pdfs",
406+
output_path="/path/to/output",
407+
markdown_output=True
408+
)
409+
```
410+
411+
> [!NOTE]
412+
> When using `--markdown`, the `--force` flag only checks for existing TEI files. If a TEI file is rewritten (due to `--force`), the corresponding Markdown file is automatically rewritten as well.
413+
332414
### Header Document Processing
333415
Extracts only document metadata (title, authors, abstract, etc.).
334416

0 commit comments

Comments
 (0)