|
| 1 | +# Document Preview Guidance |
| 2 | + |
| 3 | +Use this reference whenever Kognitos application work renders an extracted |
| 4 | +document (invoice, ID, form, contract) with the per-field overlay produced by |
| 5 | +Kognitos IDP — bounding boxes, confidence, and a side panel of extracted |
| 6 | +values. The same patterns apply to any single-document preview built on a |
| 7 | +Kognitos run output, not just invoices. |
| 8 | + |
| 9 | +The IDP payload is a free-form `Struct` under |
| 10 | +`state.completed.outputs.idp_extraction_results`. The OpenAPI spec does not |
| 11 | +schematize its inner shape, so the application owns the contract end-to-end: |
| 12 | +parser, overlay geometry, and panel UX must all agree on one canonical |
| 13 | +field model. |
| 14 | + |
| 15 | +## Default Expectations |
| 16 | + |
| 17 | +- Treat the run payload as canonical. If the parser yields zero highlights, |
| 18 | + log a one-line diagnostic at the fetch boundary and surface a |
| 19 | + user-visible empty state — never render an empty overlay silently. |
| 20 | +- Normalize the payload in an adapter, not in the UI. The viewer consumes a |
| 21 | + flat `FieldHighlight[]`; it does not walk protobuf wrappers. |
| 22 | +- Drive canvas, mask, and overlay layers from a single `layout` object so |
| 23 | + they pixel-align across zoom and resize. |
| 24 | +- Keep all interactive UI (overlay boxes, panel rows, toolbar) keyboard- and |
| 25 | + screen-reader-accessible — overlay buttons get `aria-label`, panel rows |
| 26 | + use real `<button>` elements with focus styling, tooltips have text. |
| 27 | +- Open the viewer in a modal dialog. Do not navigate away from the dashboard |
| 28 | + context to view a document. |
| 29 | + |
| 30 | +## Window Chrome and Color Scheme |
| 31 | + |
| 32 | +The preview is a centered modal with a viewport-relative size, not a |
| 33 | +full-screen takeover. Use a three-band dark palette so the document itself |
| 34 | +is the only light surface — the eye lands on the page automatically. |
| 35 | + |
| 36 | +| Surface | Role | Intent | |
| 37 | +|---|---|---| |
| 38 | +| Dialog shell | Frame around the entire preview | Near-black with a thin neutral border; close button in the dialog header | |
| 39 | +| Page rail (left) | Per-page thumbnails for multi-page docs | Slightly lighter than the dialog; thumbnails use a subtle outline when selected | |
| 40 | +| Document workspace (center) | The PDF canvas, overlay, dim mask, and bottom toolbar | Mid-dark neutral so a white page reads as "centered on a desk" | |
| 41 | +| Right panel | Extracted-fields list and filters | Same family as the page rail; collapsible; never floats over the document | |
| 42 | +| Document surface | The PDF page itself | The only white surface in the entire dialog | |
| 43 | + |
| 44 | +Other rules: |
| 45 | + |
| 46 | +- Borders are 1px and neutral. Do not stack drop shadows inside the dialog; |
| 47 | + the modal already has its own elevation. |
| 48 | +- Avoid colored backgrounds on the rails. Color is reserved for state |
| 49 | + (focused box, hovered row, active page). |
| 50 | +- Dialog header height stays small and shows only the document title and |
| 51 | + the close button. The toolbar lives in the workspace footer (see below). |
| 52 | + |
| 53 | +## Document Positioning |
| 54 | + |
| 55 | +The document fits the workspace width on first render and stays centered |
| 56 | +through every zoom and panel-toggle. Three rules drive this: |
| 57 | + |
| 58 | +1. **Single source of truth for layout.** Read PDF.js base viewport once |
| 59 | + (`{ baseW, baseH }` at scale 1), then derive `cssW`/`cssH` for the |
| 60 | + active zoom in one helper. Use the same numbers for the canvas size, |
| 61 | + the SVG mask viewBox, and the overlay container. Do not let PDF.js |
| 62 | + compute a viewport per layer — floating-point drift will desync them. |
| 63 | +2. **Reserve the side-panel width even while the panel is collapsed.** |
| 64 | + Subtract a fixed panel-width constant (mirroring the panel's max width) |
| 65 | + from the workspace measurement. This prevents the document from |
| 66 | + reflowing when the user opens the panel for the first time. |
| 67 | +3. **Lock the fit cap when the panel is open.** Once measured with the |
| 68 | + panel open, stash that cap in a ref. When the panel is later closed, |
| 69 | + `min(cap, currentRaw)` keeps the document at the narrower size and |
| 70 | + only re-centers — no surprise enlargement. |
| 71 | + |
| 72 | +Render the page in a vertically scrollable container above a sticky bottom |
| 73 | +toolbar (see next section). Re-fit on `ResizeObserver` callbacks so window |
| 74 | +resize and dialog resize behave the same. |
| 75 | + |
| 76 | +## Bottom Toolbar (Document Controls) |
| 77 | + |
| 78 | +A pill-shaped, floating toolbar pinned to the bottom of the workspace. |
| 79 | + |
| 80 | +| Order | Control | Notes | |
| 81 | +|---|---|---| |
| 82 | +| 1 | Zoom out | Disabled at min zoom | |
| 83 | +| 2 | Zoom in | Disabled at max zoom | |
| 84 | +| 3 | Fit to width | Resets to fit-cap | |
| 85 | +| 4 | Toggle field highlights | Pressed-state visually distinct (subtle accent fill) | |
| 86 | +| 5 | Download PDF | Streams from the document fetch endpoint | |
| 87 | +| — | Divider | Visually separates document controls from panel control | |
| 88 | +| 6 | Toggle right panel | Distinct visual treatment (outlined) so it reads as "panel" not "document" | |
| 89 | + |
| 90 | +Rules: |
| 91 | + |
| 92 | +- Buttons are square (~31×31 px), neutral hover, tooltip-on-top. |
| 93 | +- The container row is `pointer-events-none` so it does not block document |
| 94 | + clicks; the pill itself re-enables pointer events. This matters when the |
| 95 | + toolbar overlaps the bottom of the document during zoom. |
| 96 | +- Disabled-state for zoom limits is required — do not let the operator |
| 97 | + click into a no-op. |
| 98 | +- Min/max zoom and step factor are constants near the component. Pick a |
| 99 | + step that yields ~12% change per click so two clicks roughly double or |
| 100 | + halve the perceived size. |
| 101 | +- The toolbar re-centers itself on workspace resize via the same fit |
| 102 | + measurement used for the document. |
| 103 | + |
| 104 | +## Bounding Box Overlays |
| 105 | + |
| 106 | +Three layers, all sized in CSS pixels matching the canvas exactly: |
| 107 | + |
| 108 | +1. **SVG `<defs><mask>`** sized `cssW × cssH`. Inside the mask, draw a |
| 109 | + white rect over the whole page, then a black rect per field bbox. This |
| 110 | + is a luminance mask — black areas become transparent in the dim layer. |
| 111 | +2. **Dim layer** (`pointer-events-none`, z-index 10). A solid |
| 112 | + semi-transparent dark fill (~52% opacity) covers the entire page; the |
| 113 | + mask cuts holes only where field boxes sit, producing a "spotlight" |
| 114 | + effect. Do not use `backdrop-filter` — it doubles the cost without |
| 115 | + improving legibility. |
| 116 | +3. **Overlay button layer** (z-index 20). One transparent `<button>` per |
| 117 | + field, positioned in either CSS percentages (when the bbox is |
| 118 | + normalized) or scaled PDF-point units. Three visual states stacked by |
| 119 | + z-index inside the layer: |
| 120 | + - Idle (z-21): neutral white border, no fill. |
| 121 | + - Linked-hover from panel (z-22): cool accent border (e.g. sky), still |
| 122 | + no fill. |
| 123 | + - Focused (z-23): warm accent border + outer ring (e.g. amber). Only |
| 124 | + one box is focused at a time. |
| 125 | + |
| 126 | +Hover state is bidirectional: pointer enter on either the box or the |
| 127 | +panel row sets a `linkedHoverFieldId`; both sides observe it. Activation |
| 128 | +(click) sets a `focusedFieldId`, jumps to the field's page, scrolls the |
| 129 | +panel row into view, and scrolls the box into the document viewport. |
| 130 | + |
| 131 | +Coordinate mode is per-field, inferred from the decoded bbox magnitudes |
| 132 | +(see "IDP Payload Contract" below). Both modes resolve into the same |
| 133 | +percentage layout against the PDF base viewport, so the overlay code does |
| 134 | +not branch on units. |
| 135 | + |
| 136 | +## Right Panel — Extracted Values + Confidence |
| 137 | + |
| 138 | +A collapsible side panel rendered to the right of the document, never |
| 139 | +floating over it. Width matches the constant reserved by the document |
| 140 | +fit measurement (typically ~320 px, capped at viewport width on small |
| 141 | +screens). |
| 142 | + |
| 143 | +Header: |
| 144 | + |
| 145 | +- Title (e.g. "All extracted fields"). |
| 146 | +- Pill counter ("13 Fields", with the singular spelled out at 1). |
| 147 | +- Close button. |
| 148 | + |
| 149 | +Toolbar row beneath the header: |
| 150 | + |
| 151 | +- Page filter dropdown — "All fields" plus one entry per page present |
| 152 | + in the parsed highlights ("Page 1 only", "Page 2 only", …). |
| 153 | +- Search toggle — opens a single-line filter input below the toolbar. |
| 154 | + Input filters by label or value, case-insensitive, trimmed. |
| 155 | +- Sort cycle — one icon-button that cycles through three modes: page + |
| 156 | + name, name A–Z, confidence high-first. |
| 157 | + |
| 158 | +Field rows: |
| 159 | + |
| 160 | +- Type icon (monospace text icon for plain fields). |
| 161 | +- Monospace label (e.g. `vendor_invoice_number`). |
| 162 | +- Page badge (`p1`, `p2`). |
| 163 | +- Three-bar signal-style confidence meter: zero bars when confidence is |
| 164 | + null, one bar < 55, two bars < 85, three bars otherwise. Tooltip text: |
| 165 | + `Confidence: 98%` for fractional inputs (`0–1`), the bare number for |
| 166 | + larger inputs, and `No confidence score` for null. |
| 167 | +- The extracted value rendered in a read-only-styled chip on its own |
| 168 | + row, with overflow scrolling for long values. |
| 169 | + |
| 170 | +Interactions: |
| 171 | + |
| 172 | +- Hover sets `linkedHoverFieldId` (two-way with overlay). |
| 173 | +- Activate (click) sets `focusedFieldId`, switches the document to the |
| 174 | + field's page, scrolls the box into view in the document, scrolls the |
| 175 | + row into view inside the panel. |
| 176 | +- Empty filter result shows a quiet inline message; the empty-payload |
| 177 | + state shows a different message that tells the operator the run had |
| 178 | + no extracted fields. |
| 179 | + |
| 180 | +## IDP Payload Contract |
| 181 | + |
| 182 | +Tool-agnostic spec — the parser must accept all of this: |
| 183 | + |
| 184 | +- **Source path:** `payload.state.completed.outputs.idp_extraction_results` |
| 185 | + (or `idpExtractionResults`). |
| 186 | +- **Tree shape:** root is a protobuf `Struct`, either |
| 187 | + `{ dictionary: { entries: [...] } }` or a top-level `entries` array of |
| 188 | + `{ key, value }` rows. |
| 189 | +- **Fields list:** under the root, find the entry whose key text is |
| 190 | + `"fields"`. The value holds a list — resolve `value.list.items`, |
| 191 | + falling back to `list.items`, then `items`. |
| 192 | +- **Field row:** each item is `{ dictionary: { entries } }` and contains |
| 193 | + `element_type`, `name`, `values` (list of value objects), `page_number`, |
| 194 | + `confidence`, and `bounding_box` (a nested dictionary with `x`, `y`, |
| 195 | + `width`, `height`). |
| 196 | +- **Element type alias:** accept both `extracted_field` (legacy) and |
| 197 | + `document_field` (current `book-idp` shape), case-insensitive. Use a |
| 198 | + single `isExtractedFieldElementType` helper — never inline the literal |
| 199 | + comparison. New aliases must extend the helper, not the call sites. |
| 200 | + Bumblebee's `normalizeDocumentField` is the upstream canonical mapping. |
| 201 | +- **Number decoding:** numbers may arrive as primitives, as protobuf |
| 202 | + `Value` wrappers (`{ number: ... }`, nested `{ value: ... }` layers), |
| 203 | + or as C# `Decimal`-style bits `{ lo, hi, mid?, flags? }`. Decode the |
| 204 | + `Decimal` shape as `Decimal.GetBits` (96-bit unsigned magnitude |
| 205 | + composed of `lo | mid << 32 | hi << 64`, scale in `flags` bits 16–23 |
| 206 | + capped at 28, sign in bit 31). Do **not** use `lo / 2^32` — that |
| 207 | + decoder produced wrong bbox fractions on real payloads. |
| 208 | +- **Bbox coordinate mode:** if `max(x + width, y + height) ≤ 1.0005`, |
| 209 | + treat the bbox as normalized 0–1 fractions. Otherwise treat it as PDF |
| 210 | + user-space points relative to the PDF.js base viewport at scale 1. |
| 211 | + Both modes resolve into the same percentage layout downstream. |
| 212 | +- **Name blocklist:** skip rows whose name is in a small blocklist |
| 213 | + (`payment_recommendation`, `result_type`, `document_count`, `document`, |
| 214 | + `page_count`, `confidence`, empty), or contains `markdown_report`, or |
| 215 | + matches `summary` / starts with `summary_`. These are aggregate keys, |
| 216 | + not extracted fields, and they pollute the panel if rendered. |
| 217 | + |
| 218 | +## Document Fetch and Payload Fetch |
| 219 | + |
| 220 | +Two server endpoints owned by the application's adapter layer. The viewer |
| 221 | +consumes URLs only — it does not see Kognitos credentials or file ids. |
| 222 | + |
| 223 | +- **PDF bytes:** stream the document through a server route that resolves |
| 224 | + the file id from the run inputs and downloads via the Files API. See |
| 225 | + [`kognitos-api-client/references/runs-api.md`](../../kognitos-api-client/references/runs-api.md) |
| 226 | + for the run shape and the Files API for the download endpoint. |
| 227 | +- **Run payload JSON:** a server route that returns the raw payload object |
| 228 | + (the same JSON used to build `KognitosRun`, but unmapped). Log a single |
| 229 | + one-line diagnostic on every fetch: |
| 230 | + |
| 231 | + ``` |
| 232 | + payloadIsObject, hasIdpExtractionResults, fieldsListItemsLength, |
| 233 | + extractedFieldItemsCount, normalizedHighlightsCount |
| 234 | + ``` |
| 235 | + |
| 236 | + When Kognitos renames an `element_type` (it has happened), this log is |
| 237 | + the fastest signal — `extractedFieldItemsCount` will go to zero while |
| 238 | + `fieldsListItemsLength` stays positive. |
| 239 | + |
| 240 | +The viewer fetches both URLs once on mount, keyed by `runId` so changing |
| 241 | +the row resets the in-flight requests cleanly. |
| 242 | + |
| 243 | +## App Review Questions |
| 244 | + |
| 245 | +- If Kognitos adds a new element-type alias tomorrow, is there exactly |
| 246 | + one place in the parser to add it? |
| 247 | +- Does the payload-fetch endpoint log `extractedFieldItemsCount` so a |
| 248 | + silent regression is visible at the next on-call check? |
| 249 | +- Does toggling the right panel reflow the document, or does the fit cap |
| 250 | + stay locked? |
| 251 | +- Does activating a panel row both jump the document to the field's page |
| 252 | + and scroll the row into view inside the panel? |
| 253 | +- Are canvas, dim mask, and overlay layers driven by the same `layout` |
| 254 | + object, so they stay pixel-aligned across zoom and resize? |
| 255 | +- Does the toolbar's `pointer-events-none` container leave the document |
| 256 | + clickable beneath it, with the pill itself re-enabling pointer events? |
| 257 | +- When the parser yields zero highlights, does the UI render an explicit |
| 258 | + empty-state message instead of a blank overlay? |
0 commit comments