Skip to content

fix: tag LLM summary descriptionType as Other (not Abstract); drop from prompt#6

Merged
jimnoneill merged 1 commit into
mainfrom
fix/prompt-remove-descriptiontype
Jun 10, 2026
Merged

fix: tag LLM summary descriptionType as Other (not Abstract); drop from prompt#6
jimnoneill merged 1 commit into
mainfrom
fix/prompt-remove-descriptiontype

Conversation

@jimnoneill

@jimnoneill jimnoneill commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

The fix

poster2json's descriptions[] entry is a machine-generated summary, so its descriptionType must be "Other". "Abstract" is reserved for the author's own formal poster abstract, which the platform attaches downstream (the submitter's abstract), never poster2json's summary. The field-coverage tracker says exactly this: summary -> Other, user abstract -> Abstract.

Post-processing had hardcoded descriptionType = "Abstract" since v0.9.3 (2026-06-05), a regression from the "Other" set on 2026-05-04 (3945d81) and reverted on 2026-05-19 (74536e8). So for roughly three weeks every extracted summary was mislabeled as an abstract. _postprocess_json now sets "Other".

Also here

  • Remove descriptionType from the prompt. The type is set deterministically in post-processing, so asking the model for it was redundant and its value was overwritten anyway. Dropped from both EXTRACTION_PROMPT and FALLBACK_PROMPT; output stays schema-valid (postprocess fills "Other" when the model omits it).

Scope

Code-only fix. The prompt-design doc update and the open discussion live in the companion PR (#7), not here.

Tests

Non-GPU suite passes (239 passed, 1 skipped), including the _postprocess_json tests.

Version

0.9.15 off main (0.9.14). PosterSentry PR (#4) is rebased to 0.9.16; merge #6 before #4 to keep versions ordered.

@fairdataihub-bot

Copy link
Copy Markdown

Thank you for submitting this pull request! We appreciate your contribution to the project. Before we can merge it, we need to review the changes you've made to ensure they align with our code standards and meet the requirements of the project. We'll get back to you as soon as we can with feedback. Thanks again!

@sourcery-ai

sourcery-ai Bot commented Jun 10, 2026

Copy link
Copy Markdown

Reviewer's Guide

Removes the redundant descriptionType field from extraction prompt JSON examples (relying on deterministic post-processing instead), updates documentation and changelog to reflect current field ownership and history (publicationYear and descriptionType), and bumps the package version to 0.9.15.

Sequence diagram for descriptionType handling in extraction pipeline

sequenceDiagram
    participant Extract as _generate
    participant LLM
    participant Postprocess as _postprocess_json
    participant Client

    Extract->>LLM: generate(prompt_without_descriptionType)
    LLM-->>Extract: JSON(descriptions[].description)
    Extract-->>Postprocess: result_json
    Postprocess->>Postprocess: set descriptionType="Abstract" on each descriptions[]
    Postprocess-->>Client: finalized_json
Loading

File-Level Changes

Change Details Files
Stop requesting descriptionType in the extraction and fallback prompts while keeping it deterministically set in post-processing.
  • Removed descriptionType from the descriptions JSON examples in the primary extraction prompt template.
  • Removed descriptionType from the descriptions JSON examples in the fallback prompt template.
  • Clarified post-processing comments to state that descriptionType is always set to "Abstract" and is no longer requested from the model.
poster2json/extract.py
Sync prompt-design documentation with current behavior for publicationYear, descriptionType, and funding-related fields, and trim resolved open questions.
  • Updated the description of which required schema fields are requested from the model versus filled by the platform, explicitly moving publicationYear to platform-owned.
  • Adjusted the field-ownership tables to show publicationYear as platform-sourced and explicitly not in the prompt.
  • Reworded the descriptions section to note that only description text is model-generated and descriptionType is set in post-processing.
  • Expanded the historical changelog in the doc to cover grounding and eventual removal of publicationYear and to add the new descriptionType removal step.
  • Rewrote the Open Questions section to remove resolved items and focus on fundingReferences coverage and the high-level rule for model vs. platform ownership.
docs/prompt-design.md
Record the descriptionType prompt removal and doc sync in the project changelog and bump the package version.
  • Added a 0.9.15 changelog entry describing removal of descriptionType from prompts, deterministic setting in post-processing, and doc refresh.
  • Updated the Poetry project version from 0.9.14 to 0.9.15.
CHANGELOG.md
pyproject.toml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@fairdataihub-bot

Copy link
Copy Markdown

Thanks for making updates to your pull request. Our team will take a look and provide feedback as soon as possible. Please wait for any GitHub Actions to complete before editing your pull request. If you have any additional questions or concerns, feel free to let us know. Thank you for your contributions!

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Since fundingReferences is now explicitly called out as a remaining gap in the docs, consider adding a small in-code TODO (near the prompt JSON shape) or linking a tracking issue so the eventual prompt update for fundingReferences doesn’t drift out of sync with the documentation again.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Since `fundingReferences` is now explicitly called out as a remaining gap in the docs, consider adding a small in-code TODO (near the prompt JSON shape) or linking a tracking issue so the eventual prompt update for `fundingReferences` doesn’t drift out of sync with the documentation again.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@jimnoneill
jimnoneill force-pushed the fix/prompt-remove-descriptiontype branch from 7872e7d to b7e16d5 Compare June 10, 2026 17:33
@jimnoneill jimnoneill changed the title fix: remove descriptionType from extraction prompt; sync prompt-design doc fix: tag LLM summary descriptionType as Other (not Abstract); drop from prompt Jun 10, 2026
…m prompt (0.9.15)

poster2json's descriptions[] entry is a machine-generated summary, so its
descriptionType is "Other". "Abstract" is reserved for the author's own formal
poster abstract, which the platform attaches downstream (the submitter's
abstract), never poster2json's summary.

Post-processing had hardcoded descriptionType = "Abstract" since v0.9.3
(2026-06-05) -- a regression from the "Other" set on 2026-05-04 (3945d81) and
reverted on 2026-05-19 (74536e8). For about three weeks every extracted summary
was mislabeled as an abstract. _postprocess_json now sets "Other".

Also remove descriptionType from EXTRACTION_PROMPT and FALLBACK_PROMPT: the type
is set deterministically in post-processing, so asking the model for it was
redundant and its value was overwritten anyway. Output stays schema-valid.

The prompt-design doc update lives in the companion discussion PR, not here, so
this PR is a focused code fix.
@jimnoneill
jimnoneill force-pushed the fix/prompt-remove-descriptiontype branch from b7e16d5 to 6237384 Compare June 10, 2026 18:55
@jimnoneill
jimnoneill merged commit e7571d5 into main Jun 10, 2026
2 checks passed
@fairdataihub-bot

Copy link
Copy Markdown

Thank you for getting this pull request merged. We appreciate your contribution and look forward to your next one!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant