Commit 85d38fb
authored
Update TaskCompletion Evaluator to support multi-turn evaluation (#4922)
* Update TaskCompletion to support multi-turn eval
* run code health
* Update multi-turn validation, change session term to conversation, add tests and comments
* Add optional evaluation_mode parameter
* update tests
* code health
* update conversation parameter to messages
* add supported_evaluation_level
* pop messages in single-turn path
* add messages to the optional params
* Add preview tag, update test
* Rename supported evaluation level to evaluation level
* add reformatting tests
* remove preview tag, update spec
* update tests
* Update supported_evaluation_level, use constants for roles1 parent 5b205c9 commit 85d38fb
6 files changed
Lines changed: 1525 additions & 15 deletions
File tree
- assets/evaluators
- builtin/task_completion
- evaluator
- tests
- test_evaluators_behavior
- test_evaluators_quality
0 commit comments