Skip to content

Single-Stage Extraction + Confidence Estimation Levers #341

Description

@sromoam

Is your feature request related to a problem? Please describe.
I want a way to extract both the value and the confidence value for each extracted key in a single-shot prompt in the extraction stage, instead of using assessment.

Describe the solution you'd like
I want a way to configure our extraction prompt to also include the instruction to extract confidence inline with the extraction value.
i.e.

# current Extraction
{"key": "value"}

# requested capability to prompt against
{"key": {"_value": "value", "_confidence": 0.5}

I want to have the ability to define per field instructions or global instructions in the Extraction config to define the behavior of extracting the confidence value per extraction key. Also, I want to have the accelerator magically handle the data plumbing downstream to make sure my extraction gets into the right place, and the confidence estimation goes to the right place as well.

Describe alternatives you've considered
It's possible to change the instructions, but what is unclear is if changing the output format will mess up downstream processing.

Additional context
We are willing to support with sample prompting, and inclsuions to stickler to accomodate any changes required.
This will enable users to reduce token consumption and to simplify extraction. We have a strong belief that this will work very well for documents < 10 pages, and with < 200 entities to be extracted based on our research.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions