Skip to content

Commit da51adf

Browse files
committed
refactor: The project was comprehensively refactored, with litellm introduced as the LLM client library, and input/output parameters were optimized.
1 parent f9e71dc commit da51adf

23 files changed

Lines changed: 2615 additions & 672 deletions

.env.sample

Lines changed: 55 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,55 @@
1-
OPENAI_API_KEY="sk-xxxxxx"
2-
OPENAI_API_BASE="https://api.openai.com/v1/"
3-
OPENAI_DEFAULT_MODEL="gpt-4o"
4-
OPENAI_REASONING_EFFORT="" # 'high' | 'medium' | 'low' | 'minimal'
1+
# MarkPDFDown Configuration Template
2+
# Copy this file to .env and configure your settings
3+
4+
# =============================================================================
5+
# Model Configuration
6+
# =============================================================================
7+
8+
# Model name - LiteLLM will automatically route to the correct provider
9+
# Examples:
10+
# OpenAI models: gpt-4o, gpt-4o-mini, gpt-4-vision-preview
11+
# OpenRouter models: openrouter/anthropic/claude-3.5-sonnet, openrouter/google/gemini-pro-vision
12+
# OpenAI-Compatible models: openai/hunyuan-turbo-vision, openai/doubao-1-5-vision-pro-32k-250115
13+
MODEL_NAME=gpt-4o
14+
15+
# =============================================================================
16+
# API Keys (LiteLLM automatically detects these environment variables)
17+
# =============================================================================
18+
19+
# OpenAI API Key (for OpenAI models)
20+
# OPENAI_API_KEY=your-openai-api-key-here
21+
# OPENAI_API_BASE=https://api.openai.com/v1 # [OPTIONAL]
22+
23+
# OpenRouter API Key (for OpenRouter models)
24+
# OPENROUTER_API_KEY=your-openrouter-api-key-here
25+
# OPENROUTER_API_BASE=https://openrouter.ai/api/v1/ # [OPTIONAL]
26+
27+
# =============================================================================
28+
# Generation Parameters (Optional)
29+
# =============================================================================
30+
31+
# Temperature for text generation (0.0 to 2.0)
32+
TEMPERATURE=0.3
33+
34+
# Maximum number of tokens for generated text
35+
MAX_TOKENS=8192
36+
37+
# Number of retries for failed API calls
38+
RETRY_TIMES=3
39+
40+
# =============================================================================
41+
# Usage Examples
42+
# =============================================================================
43+
44+
# For OpenAI GPT-4o:
45+
# MODEL_NAME=gpt-4o
46+
# OPENAI_API_KEY=sk-...
47+
48+
# For OpenRouter Claude 4 Sonnet:
49+
# MODEL_NAME=openrouter/anthropic/claude-sonnet-4
50+
# OPENROUTER_API_KEY=sk-or-v1-...
51+
52+
# For Doubao 1.5 Vision Pro:
53+
# MODEL_NAME=openai/doubao-1-5-vision-pro-32k-250115
54+
# OPENAI_API_KEY=sk-...
55+
# OPENAI_API_BASE=https://ark.cn-beijing.volces.com/api/v3/

Dockerfile

Lines changed: 28 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,29 @@
1-
from python:3.9-slim
1+
FROM python:3.9-slim
2+
3+
# Set working directory
24
WORKDIR /app
3-
COPY . /app
4-
RUN pip install -e .
5-
CMD ["python", "main.py"]
5+
6+
# Install system dependencies
7+
RUN apt-get update && apt-get install -y \
8+
curl \
9+
&& rm -rf /var/lib/apt/lists/*
10+
11+
# Install uv for faster package management
12+
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
13+
ENV PATH="/root/.cargo/bin:$PATH"
14+
15+
# Copy project files
16+
COPY pyproject.toml uv.lock ./
17+
COPY src/ ./src/
18+
COPY .env.sample ./
19+
20+
# Install dependencies and the package
21+
RUN uv sync --frozen
22+
RUN uv pip install -e .
23+
24+
# Set environment variables
25+
ENV PYTHONPATH=/app/src
26+
ENV PYTHONUNBUFFERED=1
27+
28+
# Use markpdfdown as the entry point
29+
ENTRYPOINT ["markpdfdown"]

README.md

Lines changed: 134 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,17 @@
1515

1616
## Overview
1717

18-
MarkPDFDown is designed to simplify the process of converting PDF documents into clean, editable Markdown text. By utilizing advanced multimodal AI models, it can accurately extract text, preserve formatting, and handle complex document structures including tables, formulas, and diagrams.
18+
MarkPDFDown is designed to simplify the process of converting PDF documents into clean, editable Markdown text. By utilizing advanced multimodal AI models through LiteLLM, it can accurately extract text, preserve formatting, and handle complex document structures including tables, formulas, and diagrams.
1919

2020
## Features
2121

2222
- **PDF to Markdown Conversion**: Transform any PDF document into well-formatted Markdown
2323
- **Image to Markdown Conversion**: Transform image into well-formatted Markdown
24-
- **Multimodal Understanding**: Leverages AI to comprehend document structure and content
24+
- **Multi-Provider Support**: Supports OpenAI and OpenRouter through LiteLLM
25+
- **Flexible CLI**: Both file-based and pipe-based usage modes
2526
- **Format Preservation**: Maintains headings, lists, tables, and other formatting elements
26-
- **Customizable Model**: Configure the model to suit your needs
27+
- **Page Range Selection**: Convert specific page ranges from PDF documents
28+
- **Modular Architecture**: Clean, maintainable codebase with separation of concerns
2729

2830
## Demo
2931
![](https://raw.githubusercontent.com/markpdfdown/markpdfdown/refs/heads/master/tests/demo_02.png)
@@ -43,6 +45,8 @@ cd markpdfdown
4345
# Install dependencies and create virtual environment
4446
uv sync
4547

48+
# Install the package in development mode
49+
uv pip install -e .
4650
```
4751

4852
### Using conda
@@ -58,29 +62,106 @@ cd markpdfdown
5862
# Install dependencies
5963
pip install -e .
6064
```
65+
66+
## Configuration
67+
68+
MarkPDFDown uses environment variables for configuration. Create a `.env` file in your project directory:
69+
70+
```bash
71+
# Copy the sample configuration
72+
cp .env.sample .env
73+
```
74+
75+
Edit the `.env` file with your settings:
76+
77+
```bash
78+
# Model Configuration
79+
MODEL_NAME=gpt-4o
80+
81+
# API Keys (LiteLLM automatically detects these)
82+
OPENAI_API_KEY=your-openai-api-key
83+
# or for OpenRouter
84+
OPENROUTER_API_KEY=your-openrouter-api-key
85+
86+
# Optional Parameters
87+
TEMPERATURE=0.3
88+
MAX_TOKENS=8192
89+
RETRY_TIMES=3
90+
```
91+
92+
### Supported Models
93+
94+
#### OpenAI Models
95+
```bash
96+
MODEL_NAME=gpt-4o
97+
MODEL_NAME=gpt-4o-mini
98+
MODEL_NAME=gpt-4-vision-preview
99+
```
100+
101+
#### OpenRouter Models
102+
```bash
103+
MODEL_NAME=openrouter/anthropic/claude-3.5-sonnet
104+
MODEL_NAME=openrouter/google/gemini-pro-vision
105+
MODEL_NAME=openrouter/meta-llama/llama-3.2-90b-vision
106+
```
107+
61108
## Usage
109+
110+
### File Mode (Recommended)
111+
62112
```bash
63-
# Set up your OpenAI API key
64-
export OPENAI_API_KEY="your-api-key"
65-
# Optionally, set up your OpenAI API base
66-
export OPENAI_API_BASE="your-api-base"
67-
# Optionally, set up your OpenAI API model
68-
export OPENAI_DEFAULT_MODEL="your-model"
69-
70-
# pdf to markdown
71-
python main.py < tests/input.pdf > output.md
72-
73-
# image to markdown
74-
python main.py < input_image.png > output.md
113+
# Basic conversion
114+
markpdfdown --input document.pdf --output output.md
115+
116+
# Convert specific page range
117+
markpdfdown --input document.pdf --output output.md --start 1 --end 10
118+
119+
# Convert image to markdown
120+
markpdfdown --input image.png --output output.md
121+
122+
# Using python module
123+
python -m markpdfdown --input document.pdf --output output.md
124+
```
125+
126+
### Pipe Mode (Docker-friendly)
127+
128+
```bash
129+
# PDF to markdown via pipe
130+
markpdfdown < document.pdf > output.md
131+
132+
# Using python module
133+
python -m markpdfdown < document.pdf > output.md
75134
```
76-
## Advanced Usage
135+
136+
### Advanced Usage
137+
77138
```bash
78-
python main.py page_start page_end < tests/input.pdf > output.md
139+
# Convert pages 5-15 of a PDF
140+
markpdfdown --input large_document.pdf --output chapter.md --start 5 --end 15
141+
142+
# Process multiple files
143+
for file in *.pdf; do
144+
markpdfdown --input "$file" --output "${file%.pdf}.md"
145+
done
79146
```
80147

81148
## Docker Usage
149+
82150
```bash
83-
docker run -i -e OPENAI_API_KEY=your-api-key -e OPENAI_API_BASE=your-api-base -e OPENAI_DEFAULT_MODEL=your-model jorbenzhu/markpdfdown < input.pdf > output.md
151+
# Build the image (if needed)
152+
docker build -t markpdfdown .
153+
154+
# Run with environment variables
155+
docker run -i \
156+
-e MODEL_NAME=gpt-4o \
157+
-e OPENAI_API_KEY=your-api-key \
158+
markpdfdown < input.pdf > output.md
159+
160+
# Using OpenRouter
161+
docker run -i \
162+
-e MODEL_NAME=openrouter/anthropic/claude-3.5-sonnet \
163+
-e OPENROUTER_API_KEY=your-openrouter-key \
164+
markpdfdown < input.pdf > output.md
84165
```
85166

86167
## Development Setup
@@ -126,7 +207,24 @@ ruff check --fix
126207
- Python 3.9+
127208
- [uv](https://astral.sh/uv/) (recommended for package management) or conda/pip
128209
- Dependencies specified in `pyproject.toml`
129-
- Access to the specified multimodal AI model
210+
- Access to supported LLM providers (OpenAI or OpenRouter)
211+
212+
## Architecture
213+
214+
The project follows a modular architecture:
215+
216+
```
217+
src/markpdfdown/
218+
├── __init__.py # Package initialization
219+
├── __main__.py # Entry point for python -m
220+
├── cli.py # Command line interface
221+
├── main.py # Core conversion logic
222+
├── config.py # Configuration management
223+
└── core/ # Core modules
224+
├── llm_client.py # LiteLLM integration
225+
├── file_worker.py # File processing
226+
└── utils.py # Utility functions
227+
```
130228

131229
## Contributing
132230
Contributions are welcome! Please feel free to submit a Pull Request.
@@ -150,10 +248,26 @@ Contributions are welcome! Please feel free to submit a Pull Request.
150248

151249
Please ensure your code follows the project's coding standards by running the linting and formatting tools before submitting.
152250

251+
## Changelog
252+
253+
### v0.2.0 (Latest)
254+
- **Breaking Changes**: Complete architecture refactor
255+
- **New**: LiteLLM integration for multi-provider support
256+
- **New**: Unified CLI with `--input`/`--output` parameters
257+
- **New**: Support for `python -m markpdfdown` execution
258+
- **New**: Environment-based configuration with `.env` support
259+
- **New**: OpenRouter support alongside OpenAI
260+
- **Improved**: Modular codebase with better separation of concerns
261+
- **Improved**: Enhanced error handling and logging
262+
263+
### v0.1.0
264+
- Initial release with basic PDF to Markdown conversion
265+
153266
## License
154267
This project is licensed under the Apache License 2.0. See the LICENSE file for details.
155268

156269
## Acknowledgments
270+
- Thanks to the developers of LiteLLM for providing unified LLM access
157271
- Thanks to the developers of the multimodal AI models that power this tool
158272
- Inspired by the need for better PDF to Markdown conversion tools
159273

@@ -164,4 +278,4 @@ This project is licensed under the Apache License 2.0. See the LICENSE file for
164278
[Size]: https://img.shields.io/docker/image-size/jorbenzhu/markpdfdown/latest?color=066da5&label=size
165279
[Pulls]: https://img.shields.io/docker/pulls/jorbenzhu/markpdfdown.svg?style=flat&label=pulls&logo=docker
166280
[Tag]: https://img.shields.io/github/release/markpdfdown/markpdfdown.svg
167-
[License]: https://img.shields.io/github/license/markpdfdown/markpdfdown
281+
[License]: https://img.shields.io/github/license/markpdfdown/markpdfdown

0 commit comments

Comments
 (0)