Production hardening: local PDF receipt OCR for Coins itemized split

[ucguy4u]

Production hardening: local PDF receipt OCR for Coins itemized split

Problem

Coins now has a local-first PDF receipt parsing path:

PDF -> rendered page images -> ML Kit OCR -> ordered OCR merge -> deterministic parser -> itemized split UI

The feature has been implemented and device-validated, but it still needs production hardening before release. The remaining work is mainly around build-system cleanup, reproducible fixtures, CI coverage, and UX/QA polish.

Current State / Evidence

Implemented locally:

• pdfx dependency renders PDF pages to image bytes on-device.
• ReceiptPdfRenderer abstraction plus PdfxReceiptPdfRenderer.
• MLKitReceiptParserService.parseReceiptPdf(...) and parseReceiptPdfBytes(...).
• Rendered PDF pages are OCR'd page-by-page.
• OCR text is merged in page order before the parser runs once.
• Invoice parser handles multi-invoice OCR text without cross-page total bleed.
• Itemized split UI accepts image/PDF upload.
• Fallback hook is still disabled by default; no cloud path is used.
• Clear local failure copy exists: Couldn't parse this PDF locally. Add items manually.

Validation already performed:

• flutter drive --driver=test_driver/integration_test.dart --target=integration_test/receipt_on_device_parser_test.dart -d emulator-5554 --dart-define=RECEIPT_PDF_PATH=/data/local/tmp/uc_invoice.pdf
  • Passed on Android emulator using a generated two-page image-only Urban Company PDF fixture.
• Focused unit tests passed:
  • test/features/bill_split/receipt_pdf_pipeline_test.dart
  • test/features/bill_split/receipt_invoice_layout_parser_test.dart
  • test/features/bill_split/receipt_parser_fallback_hook_test.dart
  • test/features/coins/coins_split_workflow_test.dart
• Scoped analyzer passed on bill split implementation/tests.
• git diff --check passed on touched files.

Known Gaps

1. Replace fragile Android registrant workaround

During normal flutter drive, Flutter regenerated GeneratedPluginRegistrant.java with:

new io.flutter.plugins.sharedpreferences.SharedPreferencesPlugin()

That failed Java compilation because the current shared_preferences_android package exposes SharedPreferencesPlugin as Kotlin, while LegacySharedPreferencesPlugin is Java-compatible.

A temporary Gradle pre-compile patch was added in:

• app/android/app/build.gradle.kts

It rewrites the generated registrant to use:

new io.flutter.plugins.sharedpreferences.LegacySharedPreferencesPlugin()

This made the normal Android integration test pass, but it is a workaround. Find a stable fix before production.

Investigate:

• Whether shared_preferences_android should be upgraded/downgraded/pinned.
• Whether Flutter tooling/version is generating a bad Java registrant for Kotlin-only plugin classes.
• Whether the app should avoid checking/generated registrant assumptions.
• Whether a cleaner Gradle/task dependency fix exists.

2. Add reproducible image-only PDF fixture

The original user PDF in ~/Downloads became unreadable from automation due macOS privacy. Device validation used a generated image-only Urban Company PDF fixture under /private/tmp.

Production handoff needs a reproducible fixture strategy:

• Preferred: add a deterministic test fixture generator script.
• Acceptable: add a small checked-in synthetic image-only PDF fixture if repo policy allows binary test fixtures.
• The fixture must have no extractable embedded receipt text for the PDF path to be meaningful.

3. Add/verify CI path

The feature currently has focused local tests and device validation. CI should include at least the unit parser tests. Device integration can be gated/manual if emulator setup is too expensive.

4. UI smoke test

The parser path was device-verified. The actual tap-through file picker flow still needs a manual QA pass or Patrol/integration coverage:

• Open Coins itemized split.
• Tap File.
• Select a PDF.
• Confirm parsed items appear.
• Confirm manual add flow remains available on failure.

5. LLM classification is intentionally deferred

Do not add a cloud path in this issue. After PDF render/OCR is production-safe, a follow-up can route messy OCR text through on-device LiteRT/LLM classification.

Scope

In Scope

• Stabilize Android build for normal flutter drive / CI.
• Add reproducible fixture or fixture generator.
• Tighten tests around deterministic PDF OCR parsing.
• Confirm no cloud fallback is used by default.
• Document how to run the Android PDF OCR integration test.

Out of Scope

• Cloud Gemini fallback.
• Server-side RAG/OCR.
• Full Splitwise export.
• New LLM extraction behavior beyond preserving existing hooks/tests.
• Large UI redesign.

Acceptance Criteria

• Normal Android build/test path passes without manual generated-file edits or --no-pub.
• The GeneratedPluginRegistrant workaround is removed or documented with a strong reason if no better fix exists.
• A reproducible image-only Urban Company PDF fixture or generator exists in the repo.
• Unit tests cover:
  • PDF renderer route for bytes/file.
  • Multi-page OCR ordering.
  • Repeated parse returns the same vendor, total, item names, and item prices.
  • Multi-invoice text does not leak totals across invoice sections.
  • Fallback hook remains unused by default.
• Android integration test passes on emulator against the image-only PDF fixture.
• Itemized split UI clearly handles local PDF parse failure and allows manual item entry.
• No cloud/API fallback is introduced.
• flutter analyze is clean for touched files.
• git diff --check is clean.
• PR notes include exact commands run and results.

Suggested Verification Commands

cd app

flutter test \
  test/features/bill_split/receipt_pdf_pipeline_test.dart \
  test/features/bill_split/receipt_invoice_layout_parser_test.dart \
  test/features/bill_split/receipt_parser_fallback_hook_test.dart \
  test/features/coins/coins_split_workflow_test.dart

flutter analyze \
  lib/features/bill_split/domain/services/receipt_parser_service.dart \
  lib/features/bill_split/domain/services/receipt_pdf_renderer_service.dart \
  lib/features/bill_split/presentation/screens/itemized_split_screen.dart \
  test/features/bill_split/receipt_pdf_pipeline_test.dart \
  test/features/bill_split/receipt_invoice_layout_parser_test.dart \
  test/features/bill_split/receipt_parser_fallback_hook_test.dart \
  integration_test/receipt_on_device_parser_test.dart \
  test_driver/integration_test.dart

flutter drive \
  --driver=test_driver/integration_test.dart \
  --target=integration_test/receipt_on_device_parser_test.dart \
  -d emulator-5554 \
  --dart-define=RECEIPT_PDF_PATH=/data/local/tmp/uc_invoice.pdf

From repo root:

git diff --check

Ownership

• Primary agent: agent/finance
• Supporting agents: agent/mobile-ui, agent/qa-testing, agent/devex

Priority

priority/P1

Estimate

1-2 days:

• 0.5 day fixture/generator and docs.
• 0.5 day Android build-system cleanup.
• 0.5 day CI/QA polish.
• Buffer for emulator/build quirks.

Risks

• Android build workaround hides a deeper dependency/tooling mismatch.
• OCR output may vary by ML Kit version/device locale; deterministic parser tests should use controlled OCR text and device tests should assert stable output from a known fixture.
• Checked-in binary PDFs can bloat repo if not kept small.
• User receipts may contain PII; do not log real receipt OCR in production.

Hermes Handoff Prompt

You are Hermes. Pick up this issue and production-harden the local PDF receipt OCR flow for Coins.

Do not add cloud fallback. Keep the feature local-first.

Start by auditing:
- app/lib/features/bill_split/domain/services/receipt_pdf_renderer_service.dart
- app/lib/features/bill_split/domain/services/receipt_parser_service.dart
- app/lib/features/bill_split/presentation/screens/itemized_split_screen.dart
- app/integration_test/receipt_on_device_parser_test.dart
- app/android/app/build.gradle.kts
- app/pubspec.yaml / app/pubspec.lock

Main tasks:
1. Replace or justify the GeneratedPluginRegistrant shared_preferences Gradle workaround.
2. Add a reproducible image-only Urban Company PDF fixture or generator.
3. Ensure normal flutter drive passes without --no-pub/manual edits.
4. Keep OCR page text merged in page order before one parser pass.
5. Keep fallback unused by default.
6. Run and report the verification commands in this issue.

Avoid unrelated dirty files. Do not revert user changes outside this scope.