Skip to content

Move prompt_engineered_annotator to modelbench.#1160

Merged
superdosh merged 3 commits into
mainfrom
dynamic-annotator-refactor
Jul 30, 2025
Merged

Move prompt_engineered_annotator to modelbench.#1160
superdosh merged 3 commits into
mainfrom
dynamic-annotator-refactor

Conversation

@superdosh

@superdosh superdosh commented Jul 30, 2025

Copy link
Copy Markdown
Contributor

This is copy/pasted from private, with the exceptions:

  • We use SUT_FACTORY instead of SUTS on line 72.
  • Also copies in flatten_conversation_to_str.
  • Some typing changes to pass mypy

There will be a separate PR in private which removes prompt_engineered_annotator from there.

@superdosh superdosh requested a review from a team as a code owner July 30, 2025 18:29
@github-actions

github-actions Bot commented Jul 30, 2025

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@bkorycki

Copy link
Copy Markdown
Contributor

Can you put this in src/modelgauge/annotators?

@superdosh superdosh temporarily deployed to Scheduled Testing July 30, 2025 18:46 — with GitHub Actions Inactive
@superdosh superdosh temporarily deployed to Scheduled Testing July 30, 2025 18:46 — with GitHub Actions Inactive
@superdosh superdosh temporarily deployed to Scheduled Testing July 30, 2025 18:46 — with GitHub Actions Inactive
@superdosh

Copy link
Copy Markdown
Contributor Author

Can you put this in src/modelgauge/annotators?

Good idea; will minimize changes with modelbench-private! Done! @bkorycki

@superdosh superdosh temporarily deployed to Scheduled Testing July 30, 2025 18:53 — with GitHub Actions Inactive
@superdosh superdosh temporarily deployed to Scheduled Testing July 30, 2025 18:53 — with GitHub Actions Inactive
@superdosh superdosh temporarily deployed to Scheduled Testing July 30, 2025 18:53 — with GitHub Actions Inactive
@superdosh superdosh merged commit ba582ed into main Jul 30, 2025
4 checks passed
@superdosh superdosh deleted the dynamic-annotator-refactor branch July 30, 2025 19:39
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 30, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants