Commit eb284ff
authored
[feat][evaluation] Coze Coding Evaluation Target Support (#461)
* get all target fields for AgentEvaluator EvaluateTargetOutputFields
* test(evaluation): simplify test data setup for agent evaluation
* style: fix indentation in test data structure
* feat(evaluation): add support for custom RPC evaluator type
Add case for EvaluatorTypeCustomRPC in convertEntityEvaluatorTypeToOpenAPI function and refactor evaluator version ID retrieval to use GetEvaluatorVersionID method. Also add test case for agent evaluator in SubmitExperimentOApi test.
* test(evaluation): update test cases for OpenAPIColumnEvaluatorsDO2DTOs
Add more comprehensive test cases to verify conversion of different evaluator types
* feat(evaluator): add agent type support for evaluator
Add EvaluatorTypeAgent constant and handle conversion between entity and openapi types. Also add validation to reject agent type in evaluator openapi conversion.
* test(evaluator): add test case for agent evaluator rejection
* fix(evaluation): handle builtin evaluator version check properly
Skip workspace validation for builtin evaluators to allow cross-workspace execution. Add test cases for evaluator version not found and builtin success scenarios.
* feat(evaluation): add extra field to thrift structs and implement validation
Add optional Extra field to ImportEvaluationSetOApiRequest and GetEvaluationSetIOJobOApiRequest thrift structs
Implement validation, serialization and deserialization for the new field in generated code
* feat(evaluation): add builtin evaluator execution API
implement API to run builtin evaluators by ID or name, including:
- add new endpoint /v1/loop/evaluation/builtin_evaluators/run
- add service method to resolve visible version ID
- add repo method to get evaluator by space ID and name
- update thrift IDL and generate code
- add tests for new functionality
* feat(evaluation): restructure builtin evaluator endpoints and add tests
- Move builtin evaluator endpoint from `/builtin_evaluators/run` to `/evaluators/builtin/run`
- Add new middleware `_builtinMw` for builtin evaluator routes
- Implement `GetEvaluatorMetaBySpaceIDAndName` repo method and tests
- Add `ResolveBuiltinEvaluatorVisibleVersionID` service method and tests
* docs(thrift): update comment for builtin evaluator requirements
Clarify that either builtin_evaluator_id or builtin_evaluator_name must be provided, and if both are provided, they must match
* test: improve coverage for builtin evaluator domain service methods
* run code gen
* recover ai edit1 parent 88fdec6 commit eb284ff
36 files changed
Lines changed: 5575 additions & 1450 deletions
File tree
- backend
- api
- handler/coze/loop/apis
- router/coze/loop/apis
- kitex_gen/coze/loop
- apis/evalopenapiservice
- evaluation
- domain_openapi/evaluator
- evalopenapiservice
- openapi
- evaluationopenapiservice
- loop_gen/coze/loop/evaluation/loopenapi
- modules/evaluation
- application
- convertor
- evaluator
- experiment
- domain
- repo
- mocks
- service
- mocks
- infra/repo/evaluator
- mysql
- mocks
- idl/thrift/coze/loop/evaluation
- domain_openapi
Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments