feat(ocr): initialize form rendering for fillable PDFs in OcrService (#241)

- **Fillable PDF form fields missing from rendered page images** — Fixed bug where fillable PDF form fields (text inputs, checkboxes, radio buttons, dropdowns) were not rendered in page images, causing OCR and extraction to miss user-entered data. Root cause: pypdfium2's `render(may_draw_forms=True)` requires `PdfDocument.init_forms()` to be called first to initialize the form rendering engine. Added `init_forms()` call in both Pattern 2 (`OcrService`) and Pattern 1 (`create_pdf_page_images`) PDF rendering pipelines. ([#240](#240))

Author: rstrahan

CHANGELOG.md

@@ -56,6 +56,8 @@ SPDX-License-Identifier: MIT-0
 
 ### Fixed
 
+- **Fillable PDF form fields missing from rendered page images** — Fixed bug where fillable PDF form fields (text inputs, checkboxes, radio buttons, dropdowns) were not rendered in page images, causing OCR and extraction to miss user-entered data. Root cause: pypdfium2's `render(may_draw_forms=True)` requires `PdfDocument.init_forms()` to be called first to initialize the form rendering engine. Added `init_forms()` call in both Pattern 2 (`OcrService`) and Pattern 1 (`create_pdf_page_images`) PDF rendering pipelines. ([#240](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/issues/240))
+
 - **Discovery subscription handler dropping errorMessage and other fields** — Fixed bug where the UI subscription handler did `{ ...oldJob, status: updatedJob.status }`, discarding all fields except status from real-time subscription updates. Error messages, discovered class names, and status messages were being sent by the backend but silently dropped by the UI. Now spreads all fields: `{ ...oldJob, ...updatedJob }`.
 
 - **Discovery processor S3 race condition causing NoSuchKey failures** — The discovery upload resolver sends the SQS message before the browser finishes uploading the file to S3 via presigned POST. Previously worked around with a hardcoded `time.sleep(30)`. Replaced with `_wait_for_s3_object()` that polls S3 with exponential backoff (2s initial, 10s max, 60s timeout), proceeding as soon as the file appears.

lib/idp_common_pkg/idp_common/ocr/service.py

@@ -388,6 +388,10 @@ def process_document(self, document: Document) -> Document:
                 if is_pdf:
                     # Determine which pages need processing (retry-safe: skip completed pages)
                     pdf_document = pdfium.PdfDocument(file_content)
+                    # Initialize form rendering engine so fillable PDF form fields
+                    # (text inputs, checkboxes, etc.) appear in rendered page images.
+                    # Without this, may_draw_forms=True in render() has no effect.
+                    pdf_document.init_forms()
                     num_pages = len(pdf_document)
                     document.num_pages = num_pages
 

lib/idp_common_pkg/tests/unit/ocr/test_ocr_service.py

@@ -302,6 +302,57 @@ def test_init_with_preprocessing_config(self):
 
             assert service.preprocessing_config == preprocessing_config
 
+    @patch("boto3.client")
+    @patch("idp_common.ocr.service.pdfium.PdfDocument")
+    def test_process_document_calls_init_forms_for_fillable_pdfs(
+        self, mock_pdfium_doc, mock_boto_client, mock_document, mock_pdf_content
+    ):
+        """Test that init_forms() is called on PDF documents to enable fillable form field rendering.
+
+        Fillable PDFs (AcroForm) have form fields like text inputs, checkboxes, and
+        radio buttons stored as separate overlay layers. Without calling init_forms(),
+        pypdfium2's render() will not include these form field values in the output
+        image even when may_draw_forms=True (the default). This test ensures we
+        always initialize the form rendering engine before rendering pages.
+
+        Regression test for: https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/issues/240
+        """
+        # Mock S3 client
+        mock_s3_client = MagicMock()
+        mock_s3_client.get_object.return_value = {"Body": BytesIO(mock_pdf_content)}
+        mock_boto_client.return_value = mock_s3_client
+
+        # Mock PDF document
+        mock_pdf_doc = MagicMock()
+        mock_pdf_doc.__len__.return_value = 1
+        mock_pdf_doc.__iter__.return_value = iter(range(1))
+        mock_pdfium_doc.return_value = mock_pdf_doc
+
+        with (
+            patch(
+                "idp_common.ocr.service.OcrService._extract_page_image"
+            ) as mock_extract,
+            patch(
+                "idp_common.ocr.service.OcrService._process_page_with_image"
+            ) as mock_process,
+        ):
+            mock_extract.return_value = b"image_data"
+            mock_process.return_value = (
+                {
+                    "raw_text_uri": "s3://output/raw.json",
+                    "parsed_text_uri": "s3://output/parsed.json",
+                    "text_confidence_uri": "s3://output/confidence.json",
+                    "image_uri": "s3://output/image.jpg",
+                },
+                {"OCR/textract/detect_document_text": {"pages": 1}},
+            )
+
+            service = OcrService()
+            service.process_document(mock_document)
+
+            # Verify init_forms() was called to enable fillable PDF form rendering
+            mock_pdf_doc.init_forms.assert_called_once()
+
     @patch("boto3.client")
     @patch("idp_common.ocr.service.pdfium.PdfDocument")
     def test_process_document_success(

patterns/unified/src/bda_processresults_function/index.py

@@ -156,6 +156,10 @@ def create_pdf_page_images(bda_result_bucket, output_bucket, object_key):
 
         # Open the PDF using pypdfium2
         pdf_document = pdfium.PdfDocument(pdf_content)
+        # Initialize form rendering engine so fillable PDF form fields
+        # (text inputs, checkboxes, etc.) appear in rendered page images.
+        # Without this, may_draw_forms=True in render() has no effect.
+        pdf_document.init_forms()
 
         # Process each page
         for page_num in range(len(pdf_document)):