Skip to content

Redesign the extraction contract for grouped, parallel LLM calls #617

Description

@chetanr25

Redesign the extraction contract for grouped, parallel LLM calls

Background

Once the incident contract grew to cover a full incident report, it became too large to hand to the LLM in one shot and expect a clean result. This came up directly in discussion #609, structured extraction quality drops, so a single big pass is not viable for a schema this size.

The extraction endpoints also still assumed the old "extraction produces a standalone blob" model, with no link to a stored incident, and no way for the UI to tell which forms are ready to generate before the user tries and gets a 422.

What needs to change

  • Describe the extraction as multiple small grouped calls run in parallel against Ollama (bounded by OLLAMA_NUM_PARALLEL), each validated against its slice of the schema with retry, then stitched into one document. This replaces the single-pass description.
  • Extraction now creates a draft incident row the moment it completes. That row holds the contract document and becomes the single store of incident data. The extraction record itself keeps only job metadata and the corrections audit trail, nothing duplicated.
  • Add a defaults block to the extraction request (country, timezone, currency) so relative dates and money fields resolve correctly without the LLM guessing.
  • Add GET /extract/{extract_id}/readiness, a pure lookup endpoint that compares the extracted contract against every registered template's fields and reports which templates are ready to generate and what is blocking the rest. This is what drives the form-selection screen instead of the user finding out via a failed generate call.
  • Add FieldGap to describe a missing field with enough context for the UI to explain it (which template field, where the value should come from, whether it needs a manual entry or a contract correction).
  • Switch validate from a form_type enum to a template_id, since templates are user-registered and form_type is no longer a closed set.
  • Support "open" template fields, fields the incident contract does not cover, extracted as extra grouped calls in the same run and stored under the contract's custom_fields.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions