Skip to content

Commit eb284ff

Browse files
authored
[feat][evaluation] Coze Coding Evaluation Target Support (#461)
* get all target fields for AgentEvaluator EvaluateTargetOutputFields * test(evaluation): simplify test data setup for agent evaluation * style: fix indentation in test data structure * feat(evaluation): add support for custom RPC evaluator type Add case for EvaluatorTypeCustomRPC in convertEntityEvaluatorTypeToOpenAPI function and refactor evaluator version ID retrieval to use GetEvaluatorVersionID method. Also add test case for agent evaluator in SubmitExperimentOApi test. * test(evaluation): update test cases for OpenAPIColumnEvaluatorsDO2DTOs Add more comprehensive test cases to verify conversion of different evaluator types * feat(evaluator): add agent type support for evaluator Add EvaluatorTypeAgent constant and handle conversion between entity and openapi types. Also add validation to reject agent type in evaluator openapi conversion. * test(evaluator): add test case for agent evaluator rejection * fix(evaluation): handle builtin evaluator version check properly Skip workspace validation for builtin evaluators to allow cross-workspace execution. Add test cases for evaluator version not found and builtin success scenarios. * feat(evaluation): add extra field to thrift structs and implement validation Add optional Extra field to ImportEvaluationSetOApiRequest and GetEvaluationSetIOJobOApiRequest thrift structs Implement validation, serialization and deserialization for the new field in generated code * feat(evaluation): add builtin evaluator execution API implement API to run builtin evaluators by ID or name, including: - add new endpoint /v1/loop/evaluation/builtin_evaluators/run - add service method to resolve visible version ID - add repo method to get evaluator by space ID and name - update thrift IDL and generate code - add tests for new functionality * feat(evaluation): restructure builtin evaluator endpoints and add tests - Move builtin evaluator endpoint from `/builtin_evaluators/run` to `/evaluators/builtin/run` - Add new middleware `_builtinMw` for builtin evaluator routes - Implement `GetEvaluatorMetaBySpaceIDAndName` repo method and tests - Add `ResolveBuiltinEvaluatorVisibleVersionID` service method and tests * docs(thrift): update comment for builtin evaluator requirements Clarify that either builtin_evaluator_id or builtin_evaluator_name must be provided, and if both are provided, they must match * test: improve coverage for builtin evaluator domain service methods * run code gen * recover ai edit
1 parent 88fdec6 commit eb284ff

36 files changed

Lines changed: 5575 additions & 1450 deletions

backend/api/handler/coze/loop/apis/eval_open_apiservice.go

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

backend/api/router/coze/loop/apis/coze.loop.apis.go

Lines changed: 4 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

backend/api/router/coze/loop/apis/middleware.go

Lines changed: 15 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

backend/kitex_gen/coze/loop/apis/evalopenapiservice/client.go

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

backend/kitex_gen/coze/loop/apis/evalopenapiservice/evalopenapiservice.go

Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

backend/kitex_gen/coze/loop/evaluation/domain_openapi/evaluator/evaluator.go

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

backend/kitex_gen/coze/loop/evaluation/evalopenapiservice/client.go

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

backend/kitex_gen/coze/loop/evaluation/evalopenapiservice/evalopenapiservice.go

Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)