diff --git a/contracts/path/extraction.yaml b/contracts/path/extraction.yaml index 865e20c..9b6d8f9 100644 --- a/contracts/path/extraction.yaml +++ b/contracts/path/extraction.yaml @@ -2,6 +2,7 @@ # POST /api/v1/extract/{input_id} # GET /api/v1/extract/{extract_id} # PATCH /api/v1/extract/{extract_id} +# GET /api/v1/extract/{extract_id}/readiness # POST /api/v1/extract/{extract_id}/validate extract_by_input: @@ -9,10 +10,31 @@ extract_by_input: operationId: createExtraction summary: Start AI extraction from input narrative description: | - Sends the narrative (from a previously submitted input) to the local Ollama - LLM with a structured prompt to extract all incident fields into the canonical - FireForm JSON schema. This is an asynchronous operation the LLM may take - 30–120 seconds. Returns an extract_id and job_id for polling. + Extracts the narrative (from a previously submitted input) into the + incident contract. The contract is far too large for one LLM + call, so the extractor splits it into small field groups and runs them + as parallel calls against the local Ollama instance (bounded by + OLLAMA_NUM_PARALLEL and available RAM), validates each group against + the schema with per-group retry, and stitches the validated pieces + into one document. + + Open fields from registered templates (source=open) are extracted in + the same run as extra groups, so their values reach the review screen + together with everything else; they are stored under the contract's + custom_fields keyed "{form_type}.{field_name}". + + Deterministic context is applied without the LLM where possible: + relative dates resolve against the configured timezone, and country + and currency defaults come from the request or server config. + + The moment extraction completes, the server creates a draft incident + row holding the stitched contract document. That row is the single + store of incident data: review corrections write into it, and form + generation reads from it by incident_id. The extraction row itself + keeps only job metadata and the corrections audit trail. + + This is an asynchronous operation and may take 30-120 seconds. + Returns an extract_id and job_id for polling. tags: - extraction parameters: @@ -34,6 +56,10 @@ extract_by_input: extraction_hints: incident_type: "wildland_fire" state: "CA" + defaults: + country: "US" + timezone: "America/Los_Angeles" + currency: "USD" responses: "202": description: Extraction job queued successfully @@ -98,9 +124,9 @@ extract_by_id: operationId: getExtraction summary: Get extraction result by ID description: | - Returns the full canonical FireForm JSON when extraction is complete, or + Returns the full incident contract when extraction is complete, or the current job status while still processing. When status is "completed", - the response body contains the entire canonical incident schema. When still + the response body contains the entire incident contract. When still processing, includes a retry_after_seconds hint for polling. tags: - extraction @@ -123,10 +149,11 @@ extract_by_id: - $ref: "../schemas/extraction-record.yaml#/ExtractionProcessing" examples: completed: - summary: Extraction completed with canonical JSON + summary: Extraction completed with the incident contract value: extract_id: "550e8400-e29b-41d4-a716-446655440020" input_id: "550e8400-e29b-41d4-a716-446655440001" + incident_id: "550e8400-e29b-41d4-a716-446655440050" status: "completed" completed_at: "2024-07-15T14:31:05Z" incident_contract: @@ -159,10 +186,20 @@ extract_by_id: operationId: updateExtraction summary: Manually correct extracted fields description: | - Allows a responder to correct any field in the canonical JSON after LLM - extraction. Uses JSON Merge Patch (RFC 7396) only send the fields that - changed. The server records an audit trail of all changes vs the original - LLM output and recalculates completeness scores and applicable_forms. + The review-screen write path. Lets a responder correct any field in the + contract after extraction: fix a wrong value, add a missing one, or + remove a hallucinated one (RFC 7396: sending null deletes the field). + Manual template fields are entered the same way, and open-field values + corrected the same way, both under custom_fields keyed + "{form_type}.{field_name}". Only send the fields that changed. + + Corrections are applied to the contract document on the linked + incident row (the single store of incident data), so form generation + and analytics always see the reviewed values with no copy to sync. + The server records an audit trail of all changes vs the original LLM + output on the extraction and recalculates completeness, analytics + columns and readiness, so a corrected field immediately flips + dependent templates to ready in the readiness matrix. tags: - extraction parameters: @@ -181,13 +218,15 @@ extract_by_id: schema: $ref: "../schemas/incident-contract.yaml#/IncidentContract" example: - fire: - estimated_damage_usd: 250000 + losses: + property_loss: + amount: 250000 + currency: "USD" casualties: total_responder_injuries: 2 responses: "200": - description: Extraction updated returns full updated canonical JSON + description: Extraction updated returns the full updated incident contract content: application/json: schema: @@ -224,15 +263,90 @@ extract_by_id: issue: "Must be one of: confirmed, probable, suspected, undetermined" value: "maybe" +readiness: + get: + operationId: getExtractionReadiness + summary: Fill readiness of every registered template for this extraction + description: | + The form-selection matrix. Compares the extracted contract against the + field list of every active template and reports, per template, whether + it can be generated right now and which fields block it. Pure lookup + over stored data, no LLM, so it is cheap to refetch after every correction. + + The frontend renders ready templates green and selectable, the rest + greyed out; hovering or clicking a grey one lists its gaps from + missing_required. Typing a manual value or correcting the contract via + PATCH and refetching flips templates to ready. + tags: + - extraction + parameters: + - name: extract_id + in: path + required: true + description: Unique identifier of the extraction + schema: + type: string + format: uuid + responses: + "200": + description: Readiness of every active template + content: + application/json: + schema: + $ref: "../schemas/extraction-record.yaml#/ReadinessMatrix" + example: + extract_id: "550e8400-e29b-41d4-a716-446655440020" + computed_at: "2026-07-15T14:35:00Z" + templates: + - template_id: "550e8400-e29b-41d4-a716-446655440070" + form_type: "neris" + display_name: "NERIS Incident Report" + ready: true + missing_required: [] + missing_recommended: + - field_name: "smoke_alarm_presence" + source: "schema" + incident_mapping: "risk_reduction.smoke_alarm.presence" + field_coverage_percent: 94 + - template_id: "550e8400-e29b-41d4-a716-446655440073" + form_type: "state_texas" + display_name: "Texas State Fire Marshal Incident Report" + ready: false + missing_required: + - field_name: "marshal_signature_name" + source: "manual" + incident_mapping: "custom_fields.state_texas.marshal_signature_name" + description: "Reviewing marshal's printed name, entered per incident" + - field_name: "fire_cause" + source: "schema" + incident_mapping: "fire.cause_category" + missing_recommended: [] + field_coverage_percent: 78 + "404": + description: Extraction not found + content: + application/json: + schema: + $ref: "../schemas/common.yaml#/ErrorResponse" + "409": + description: Extraction not yet completed + content: + application/json: + schema: + $ref: "../schemas/common.yaml#/ErrorResponse" + example: + error_code: "EXTRACT_NOT_COMPLETED" + message: "Extraction is still processing. Wait until status is 'completed'." + validate: post: operationId: validateExtraction - summary: Validate extraction against a form's requirements + summary: Validate extraction against one template's requirements description: | - Validates the canonical JSON against a specific form type's field requirements. - Returns whether the extraction has all required fields, which recommended - fields are missing, and any warnings. Useful for checking "can I generate - a NERIS report with what I have?" before triggering form generation. + Single-template version of the readiness matrix: checks the contract + against one registered template and returns the gaps in detail. Useful + for re-checking just the template the user is looking at instead of + recomputing the whole matrix. tags: - extraction parameters: @@ -250,12 +364,13 @@ validate: schema: type: object required: - - form_type + - template_id properties: - form_type: - $ref: "../schemas/enums.yaml#/FormType" + template_id: + type: string + format: uuid example: - form_type: "neris" + template_id: "550e8400-e29b-41d4-a716-446655440070" responses: "200": description: Validation result @@ -265,49 +380,26 @@ validate: $ref: "../schemas/extraction-record.yaml#/ValidationResult" example: valid: true + template_id: "550e8400-e29b-41d4-a716-446655440070" form_type: "neris" extract_id: "550e8400-e29b-41d4-a716-446655440020" missing_required: [] missing_recommended: - - "fire.detector_present" - - "fire.detector_operated" + - field_name: "smoke_alarm_presence" + source: "schema" + incident_mapping: "risk_reduction.smoke_alarm.presence" + - field_name: "smoke_alarm_operation" + source: "schema" + incident_mapping: "risk_reduction.smoke_alarm.operation" warnings: - - "fire.estimated_damage_usd is null NERIS recommends providing damage estimates" + - "losses.property_loss is null. NERIS recommends providing damage estimates" field_coverage_percent: 94 "404": - description: Extraction not found - content: - application/json: - schema: - $ref: "../schemas/common.yaml#/ErrorResponse" - "422": - description: Unknown form type + description: Extraction or template not found content: application/json: schema: $ref: "../schemas/common.yaml#/ErrorResponse" example: - error_code: "UNKNOWN_FORM_TYPE" - message: "Form type 'xyz' is not recognized" - detail: - valid_form_types: - - neris - - nemsis_epcr - - nibrs - - nfirs_basic - - nfirs_fire - - nfirs_structure - - nfirs_wildland - - nfirs_ems - - nfirs_hazmat - - nfirs_apparatus - - nfirs_personnel - - nfirs_arson - - nfirs_casualty_civilian - - nfirs_casualty_responder - - cal_fire_ics209 - - osha_301 - - un_ssirs - - state_georgia - - state_california - - state_new_york + error_code: "TEMPLATE_NOT_FOUND" + message: "Template with ID 550e8400-e29b-41d4-a716-446655440099 not found" diff --git a/contracts/path/incidents.yaml b/contracts/path/incidents.yaml index 810a311..08d97de 100644 --- a/contracts/path/incidents.yaml +++ b/contracts/path/incidents.yaml @@ -8,11 +8,14 @@ incidents: post: operationId: createIncident - summary: Create a full incident record + summary: Finalize the incident record for an extraction description: | - Creates a permanent incident record that links input, extraction, and - generated forms into a single coherent unit. Assigns a permanent incident_id - and stores the complete incident for future retrieval and reporting. + A draft incident row is created automatically the moment extraction + completes; it owns the contract document (the single store of incident + data) and its incident_id is returned by the extraction endpoints. + This endpoint finalizes that draft: assigns the department's incident + number and tags, and promotes the record for retrieval and reporting. + It never creates a second row for the same extraction. tags: - incidents requestBody: @@ -40,6 +43,7 @@ incidents: extract_id: "550e8400-e29b-41d4-a716-446655440020" incident_number: "CA-SQF-2024-0421" status: "draft" + incident_datetime: "2024-07-10T13:52:00-07:00" forms_generated: - form_id: "550e8400-e29b-41d4-a716-446655440040" form_type: "neris" @@ -79,13 +83,15 @@ incidents: parameters: - name: date_from in: query - description: Start date filter (inclusive, ISO 8601) + description: Start date filter (inclusive, ISO 8601), applied to + incident_datetime schema: type: string format: date - name: date_to in: query - description: End date filter (inclusive, ISO 8601) + description: End date filter (inclusive, ISO 8601), applied to + incident_datetime schema: type: string format: date @@ -137,7 +143,7 @@ incidents: status: "draft" incident_name: "Bear Creek Wildfire" incident_type: "fire" - incident_date: "2024-07-10" + incident_datetime: "2024-07-10T13:52:00-07:00" forms_count: 3 created_at: "2024-07-15T14:35:00Z" pagination: @@ -166,7 +172,7 @@ incident_by_id: operationId: getIncident summary: Get full incident record description: | - Returns the complete incident record including the linked canonical + Returns the complete incident record including the linked incident extraction, all generated forms and their statuses, submission log, and audit trail. tags: diff --git a/contracts/schemas/extraction-record.yaml b/contracts/schemas/extraction-record.yaml index ae0fd48..c175b51 100644 --- a/contracts/schemas/extraction-record.yaml +++ b/contracts/schemas/extraction-record.yaml @@ -15,17 +15,41 @@ ExtractionRequest: description: Hint about the incident type (e.g. "wildland_fire", "structure_fire") state: type: string - description: US state code to apply state-specific extraction rules + description: State or region code to apply region-specific extraction rules agency_type: type: string description: Agency type hint for form selection additionalProperties: true + defaults: + type: object + description: | + Deployment context the extractor uses when the narrative does not say + otherwise: resolving relative dates ("yesterday evening") against the + local timezone, defaulting country, and picking the currency for + Money amounts. Server config supplies these when omitted. + properties: + country: + type: string + description: ISO 3166-1 alpha-2 + timezone: + type: string + description: IANA timezone name (e.g. "Asia/Kolkata") + currency: + type: string + description: ISO 4217 currency for extracted Money amounts ExtractionCompleted: type: object + description: | + A completed extraction. The contract document itself is stored once, on + the incident row this extraction links to (a draft incident is created + automatically the moment extraction completes). The extraction row keeps + only job metadata and the corrections audit trail; this response embeds + the contract read from the incident so the review screen needs one call. required: - extract_id - input_id + - incident_id - status - incident_contract properties: @@ -35,6 +59,11 @@ ExtractionCompleted: input_id: type: string format: uuid + incident_id: + type: string + format: uuid + description: The draft incident row created when extraction completed. + It holds the contract document and is what form generation targets. status: type: string enum: @@ -48,7 +77,10 @@ ExtractionCompleted: processing_time_seconds: type: number incident_contract: - $ref: "incident-contract.yaml#/IncidentContract" + description: The contract document, read from the linked incident row + (the single store; extractions hold no copy of it) + allOf: + - $ref: "incident-contract.yaml#/IncidentContract" corrections: type: array description: Audit trail of manual corrections applied via PATCH @@ -103,12 +135,15 @@ ValidationResult: type: object required: - valid - - form_type + - template_id - extract_id properties: valid: type: boolean - description: Whether all required fields for this form type are present + description: Whether all required fields for this template are present + template_id: + type: string + format: uuid form_type: $ref: "enums.yaml#/FormType" extract_id: @@ -117,13 +152,12 @@ ValidationResult: missing_required: type: array items: - type: string - description: JSON paths of required fields that are missing + $ref: "#/FieldGap" missing_recommended: type: array items: - type: string - description: JSON paths of recommended fields that are missing + $ref: "#/FieldGap" + description: Optional template fields that have no value yet warnings: type: array items: @@ -132,3 +166,82 @@ ValidationResult: field_coverage_percent: type: number description: Percentage of form fields that have values + +FieldGap: + type: object + description: One template field that has no value yet, with enough context + for the UI to explain it and offer the right fix (type it in for manual + fields, correct the contract for schema fields). + required: + - field_name + - source + properties: + field_name: + type: string + description: The template's own field name + source: + $ref: "enums.yaml#/FieldSource" + incident_mapping: + type: string + nullable: true + description: Contract path the value would come from (source=schema), + or the custom_fields key (source=manual/open) + description: + type: string + nullable: true + +ReadinessMatrix: + type: object + description: | + Per-template fill readiness for one extraction, computed by comparing the + contract (including custom_fields) against every registered template's + field list. Pure lookup, no LLM. Drives the form-selection screen: ready + templates show green and selectable, the rest grey with their gaps + listed. Filling a gap (typing a manual value, correcting the contract) + and refetching flips the template to ready. + required: + - extract_id + - templates + properties: + extract_id: + type: string + format: uuid + computed_at: + type: string + format: date-time + templates: + type: array + items: + $ref: "#/TemplateReadiness" + +TemplateReadiness: + type: object + required: + - template_id + - form_type + - display_name + - ready + properties: + template_id: + type: string + format: uuid + form_type: + $ref: "enums.yaml#/FormType" + display_name: + type: string + ready: + type: boolean + description: True when every required field resolves to a value + missing_required: + type: array + items: + $ref: "#/FieldGap" + description: Required fields with no value. Empty when ready. + missing_recommended: + type: array + items: + $ref: "#/FieldGap" + description: Optional fields with no value; the form can still generate + with these left blank + field_coverage_percent: + type: number