Is your feature request related to a problem? Please describe.
Currently, when converting PDF documents using PDFMinerToDocument or PyPDFToDocument, embedded hyperlinks are discarded during text extraction. Only the display text is extracted, resulting in the loss of valuable URL information. This creates a feature gap in the Haystack ecosystem, as other document converters (DOCXToDocument, PPTXToDocument, and XLSXToDocument) already support hyperlink extraction.
Describe the solution you'd like
Add a link_format parameter to the PDFMinerToDocument and PyPDFToDocument components to enable the extraction of hyperlink addresses from PDF files.
The supported formats should mirror the existing implementation in other document converters:
"markdown": [text](url)
"plain": text (url)
"none" (default): Only text is extracted, and link addresses are ignored.
Describe alternatives you've considered
-
Utilizing libraries like PyMuPDF in a custom component. However, this breaks the out-of-the-box pipeline experience and introduces complex dependency management.
-
Parsing the PDF twice (once for text, once for links) and attempting to map the URLs to the correct text snippets post-extraction using string matching.
Additional context
Components requiring this update:
Is your feature request related to a problem? Please describe.
Currently, when converting PDF documents using
PDFMinerToDocumentorPyPDFToDocument, embedded hyperlinks are discarded during text extraction. Only the display text is extracted, resulting in the loss of valuable URL information. This creates a feature gap in the Haystack ecosystem, as other document converters (DOCXToDocument,PPTXToDocument, andXLSXToDocument) already support hyperlink extraction.Describe the solution you'd like
Add a link_format parameter to the PDFMinerToDocument and PyPDFToDocument components to enable the extraction of hyperlink addresses from PDF files.
The supported formats should mirror the existing implementation in other document converters:
Describe alternatives you've considered
Utilizing libraries like PyMuPDF in a custom component. However, this breaks the out-of-the-box pipeline experience and introduces complex dependency management.
Parsing the PDF twice (once for text, once for links) and attempting to map the URLs to the correct text snippets post-extraction using string matching.
Additional context
Components requiring this update: