fix: handle KeyError: 'N' for certain pdfs (#2072)

Closes #2059.

We've found some pdfs that throw an error in pdfminer. These files use a
ICCBased color profile but do not include an expected value `N`. As a
workaround, we can wrap pdfminer and drop any colorspace info, since we
don't need to render the document.

To verify, try to partition the document in the linked issue.

```
elements = partition(filename="google-2023-environmental-report_condensed.pdf", strategy="fast")
```

---------

Co-authored-by: cragwolfe <crag@unstructured.io>

Author: awalker4

CHANGELOG.md

@@ -1,4 +1,4 @@
-## 0.10.31-dev3
+## 0.10.31-dev4
 
 ### Enhancements
 * **Temporary Support for paddle language parameter** User can specify default langage code for paddle with ENV `DEFAULT_PADDLE_LANG` before we have the language mapping for paddle.
@@ -11,6 +11,7 @@
 * **Remove default user ./ssh folder** The default notebook user during image build would create the known_hosts file with incorrect ownership, this is legacy and no longer needed so it was removed.
 * **Include `languages` in metadata when partitioning strategy='hi_res' or 'fast'** User defined `languages` was previously used for text detection, but not included in the resulting element metadata for some strategies. `languages` will now be included in the metadata regardless of partition strategy for pdfs and images.
 * **Handle a case where Paddle returns a list item in ocr_data as None** In partition, while parsing PaddleOCR data, it was assumed that PaddleOCR does not return None for any list item in ocr_data. Removed the assumption by skipping the text region whenever this happens.
+* **Fix some pdfs returning `KeyError: 'N'`** Certain pdfs were throwing this error when being opened by pdfminer. Added a wrapper function for pdfminer that allows these documents to be partitioned.
 
 ## 0.10.30
 

example-docs/pdf-bad-color-space.pdf

@@ -15,3 +15,4 @@ numpy
 rapidfuzz
 backoff
 typing-extensions
+wrapt

requirements/base.in

@@ -973,3 +973,10 @@ def test_partition_pdf_with_all_number_table_and_ocr_only_strategy():
     # numerical values with `strategy="ocr_only"`
     filename = example_doc_path("all-number-table.pdf")
     assert pdf.partition_pdf(filename, strategy="ocr_only")
+
+
+# As of pdfminer 221105, this pdf throws an error and requires a workaround
+# See #2059
+def test_partition_pdf_with_bad_color_profile():
+    filename = example_doc_path("pdf-bad-color-space.pdf")
+    assert pdf.partition_pdf(filename, strategy="fast")

test_unstructured/partition/pdf_image/test_pdf.py

@@ -1 +1 @@
-__version__ = "0.10.31-dev3"  # pragma: no cover
+__version__ = "0.10.31-dev4"  # pragma: no cover

unstructured/__version__.py

@@ -20,6 +20,7 @@
 
 import numpy as np
 import pdf2image
+import wrapt
 from pdfminer.converter import PDFPageAggregator
 from pdfminer.layout import (
     LAParams,
@@ -521,6 +522,19 @@ def _extract_text(item: LTItem) -> str:
     return "\n"
 
 
+# Some pages with a ICC color space do not follow the pdf spec
+# They throw an error when we call interpreter.process_page
+# Since we don't need color info, we can just drop it in the pdfminer code
+# See #2059
+@wrapt.patch_function_wrapper("pdfminer.pdfinterp", "PDFPageInterpreter.init_resources")
+def pdfminer_interpreter_init_resources(wrapped, instance, args, kwargs):
+    resources = args[0]
+    if "ColorSpace" in resources:
+        del resources["ColorSpace"]
+
+    return wrapped(resources)
+
+
 def _process_pdfminer_pages(
     fp: BinaryIO,
     filename: str,