Bump azure-ai-evaluation from 1.8.0 to 1.12.0

[dependabot]

Bumps azure-ai-evaluation from 1.8.0 to 1.12.0.

Release notes

Sourced from azure-ai-evaluation's releases.

azure-ai-evaluation_1.12.0

1.12.0 (2025-10-02)

Features Added

• AOAI Graders now accept a "credential" parameter that can be used for authentication with an AzureOpenAIModelConfiguration
• Added is_reasoning_model parameter support to CoherenceEvaluator, FluencyEvaluator, SimilarityEvaluator, GroundednessEvaluator, RetrievalEvaluator, and RelevanceEvaluator to enable reasoning model configuration for o1/o3 models.

Bugs Fixed

• Support for multi-level nesting in OpenAI grader (experimental)

azure-ai-evaluation_1.11.1

1.11.1 (2025-09-17)

Bugs Fixed

• Pinning duckdb version to 1.3.2 for redteam extra to fix error TypeError: unhashable type: '_duckdb.typing.DuckDBPyType'

azure-ai-evaluation_1.11.0

1.11.0 (2025-09-02)

Features Added

• Added support for user-supplied tags in the evaluate function. Tags are key-value pairs that can be used for experiment tracking, A/B testing, filtering, and organizing evaluation runs. The function accepts a tags parameter.
• Added support for user-supplied TokenCredentials with LLM based evaluators.
• Enhanced GroundednessEvaluator to support AI agent evaluation with tool calls. The evaluator now accepts agent response data containing tool calls and can extract context from file_search tool results for groundedness assessment. This enables evaluation of AI agents that use tools to retrieve information and generate responses. Note: Agent groundedness evaluation is currently supported only when the file_search tool is used.
• Added language parameter to RedTeam class for multilingual red team scanning support. The parameter accepts values from SupportedLanguages enum including English, Spanish, French, German, Italian, Portuguese, Japanese, Korean, and Simplified Chinese, enabling red team attacks to be generated and conducted in multiple languages.
• Added support for IndirectAttack and UngroundedAttributes risk categories in RedTeam scanning. These new risk categories expand red team capabilities to detect cross-platform indirect attacks and evaluate ungrounded inferences about human attributes including emotional state and protected class information.

Bugs Fixed

• Fixed issue where evaluation results were not properly aligned with input data, leading to incorrect metrics being reported.

Other Changes

• Deprecating AdversarialSimulator in favor of the AI Red Teaming Agent. AdversarialSimulator will be removed in the next minor release.
• Moved retry configuration constants (MAX_RETRY_ATTEMPTS, MAX_RETRY_WAIT_SECONDS, MIN_RETRY_WAIT_SECONDS) from RedTeam class to new RetryManager class for better code organization and configurability.

azure-ai-evaluation_1.10.0

1.10.0 (2025-07-31)

Breaking Changes

• Added evaluate_query parameter to all RAI service evaluators that can be passed as a keyword argument. This parameter controls whether queries are included in evaluation data when evaluating query-response pairs. Previously, queries were always included in evaluations. When set to True, both query and response will be evaluated; when set to False (default), only the response will be evaluated. This parameter is available across all RAI service evaluators including ContentSafetyEvaluator, ViolenceEvaluator, SexualEvaluator, SelfHarmEvaluator, HateUnfairnessEvaluator, ProtectedMaterialEvaluator, IndirectAttackEvaluator, CodeVulnerabilityEvaluator, UngroundedAttributesEvaluator, GroundednessProEvaluator, and EciEvaluator. Existing code that relies on queries being evaluated will need to explicitly set evaluate_query=True to maintain the previous behavior.

Features Added

• Added support for Azure OpenAI Python grader via AzureOpenAIPythonGrader class, which serves as a wrapper around Azure Open AI Python grader configurations. This new grader object can be supplied to the main evaluate method as if it were a normal callable evaluator.
• Added attack_success_thresholds parameter to RedTeam class for configuring custom thresholds that determine attack success. This allows users to set specific threshold values for each risk category, with scores greater than the threshold considered successful attacks (i.e. higher threshold means higher tolerance for harmful responses).
• Enhanced threshold reporting in RedTeam results to include default threshold values when custom thresholds aren't specified, providing better transparency about the evaluation criteria used.

Bugs Fixed

... (truncated)

Commits

• c0d0a7c [VoiceLive] Release initial GA version (#43219)
• 133f5f4 [VoiceLive] Update models for GA version (#43202)
• d07fc33 azure-ai-evaluation 1.12.0 release (#43217)
• d3e28d0 [evaluation] refactor: Make AzureOpenAIGrader._get_token_provider private (#4...
• 8d03e35 Performance Counters (#43024)
• 6097521 Bring back the 1.11.1 changes (#43212)
• 406233d [AutoRelease] t2-servicefabricmanagedclusters-2025-08-12-53724(can only be me...
• d6d03e5 Remove Azure AD reference from Identity README (#43210)
• 6536e58 bump the used proxy version to one that includes the optimizations and fixes ...
• bd69674 Add nested data handling for AOAI graders (#43204)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot]

Superseded by #151.