You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Examples: feat: add new SDK method · feat(storage): support file uploads · feat!: breaking API change
Description
Removed custom scorers feature since it was unused.
Motivation
Users are advised to use bash scorers or python scorers instead.
Changes
Testing
Unit tests added
Integration tests added
Smoke Tests added/updated
Tested locally
Breaking Changes
Checklist
PR title follows Conventional Commits format (feat: or feat(scope):)
Documentation updated (if needed)
Breaking changes documented (if applicable)
CodeAnt-AI Description
Remove custom scorer validation and support for custom scorer type
What Changed
The scorer "validate" API, its request/response types, and the SDK validate method were removed; callers can no longer run a validation endpoint for a scorer.
The "custom_scorer" scoring function type and related parameters were removed from the ScoringFunction definitions; only bash, command, python, and test-based scorers remain.
Examples, SDK docs, and tests were updated to remove references to custom scorers and validation; listing, creation, retrieval, and update of scorers remain supported.
Impact
✅ No scorer validation endpoint available ✅ Cannot register or use "custom_scorer" scoring type ✅ SDK examples and tests no longer reference scorer validation
💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
@codeant-ai ask: Your question here
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
@codeant-ai ask: Can you suggest a safer alternative to storing this secret?
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
@codeant-ai: Your feedback here
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
@codeant-ai: Do not flag unused imports.
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
@codeant-ai: review
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.
Exposed removed feature The PR description says "removed custom scorers", but the SDK still exposes ScorerOps and Scorer across the public API (RunloopSDK property, exported types, and factory methods). That will confuse users and keep an unsupported code path reachable. Verify whether scorers should be fully removed, deprecated, or blocked at runtime.
Backwards compatibility risk The code still exports Scorer and re-exports it from the SDK namespace. If the feature is removed server-side, leaving client creation/listing in place may surface confusing runtime errors for existing callers. Decide whether to add a runtime guard/deprecation or fully remove the surface.
Test Coverage Reduction The PR removes the validate-related tests and mock behavior for scorer validation. Confirm this removal was intentional and that the production Scorer implementation no longer relies on a .validate method or endpoints. If any code still expects validate to exist, tests will now miss regressions and runtime code could break.
Documentation mismatch Some JSDoc comments still refer to "custom" scorers and ScorerOps in examples. Since custom scorers feature was removed, verify that docs, examples and cross-references remain accurate and do not suggest removed functionality or references that don't exist.
Loose Mock Typing mockClient is declared as any, which can mask client API shape mismatches in tests. Mistyped calls or incorrect mock signatures may not be caught by the test suite. Consider narrowing the mock's shape or using jest helpers to better reflect the real API surface to catch regressions early.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
codeant-ai
Bot
added
the
size:XS
User description
Format:
feat[optional scope]: <description>Examples:
feat: add new SDK method·feat(storage): support file uploads·feat!: breaking API changeDescription
Removed custom scorers feature since it was unused.
Motivation
Users are advised to use bash scorers or python scorers instead.
Changes
Testing
Breaking Changes
Checklist
feat:orfeat(scope):)CodeAnt-AI Description
Remove custom scorer validation and support for custom scorer type
What Changed
Impact
✅ No scorer validation endpoint available✅ Cannot register or use "custom_scorer" scoring type✅ SDK examples and tests no longer reference scorer validation💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.