[Evaluation] Fix App Insights emission silently dropping all events when evaluator definition has no "metrics"

[slister1001]

Summary

Fixes a silent App Insights emission bug where rubric evaluators registered without metric metadata produced zero gen_ai.evaluation.result events for the entire run, even though the eval itself completed successfully.

Root cause

In _build_internal_log_attributes (_evaluate.py:1138):

if evaluator_definition := testing_criteria_config.get("_evaluator_definition"):
    metric_config_detail = evaluator_definition.get("metrics").get(metric_name)

RAISvc's RubricBasedEvaluatorDefinition.Metrics defaults to empty and is not required by Validate(), so a rubric registered without metric metadata is sent as {"type": "rubric"} — no metrics key at all. evaluator_definition.get("metrics") returns None, then None.get(...) raises:

AttributeError: 'NoneType' object has no attribute 'get'

That exception is caught by the per-event try/except at _log_events_to_app_insights line 1287 and silently swallowed before event_logger.emit() is reached. Because every event in the run hits the same code path with the same evaluator definition, every event is dropped — App Insights receives nothing.

Observed in production: rubric eval rubric-manual-260526043804-e45a09 in the westus2 bug bash project had zero AppInsights events while other rubric evals in the same workspace (e.g., aprilk-repro) that had populated metric metadata emitted normally.

Fix

Guard the metrics lookup with isinstance(metrics_section, dict) so a single misshapen definition does not abort emission for the entire run. The event is still emitted; only the optional min_value / max_value / desirable_direction / type attributes are absent for evaluators that did not register them.

Tests

Added TestBuildInternalLogAttributesEvaluatorDefinition with 5 regression cases:

• test_definition_without_metrics_key_does_not_raise — the exact failing payload ({"type": "rubric"})
• test_definition_with_metrics_none_does_not_raise
• test_definition_with_metrics_empty_dict_does_not_raise
• test_definition_with_metrics_list_does_not_raise — defensive against future malformed types
• test_definition_with_metric_metadata_still_populates_attributes — happy path unchanged

All 10 tests in TestBuildInternalLogAttributesThreshold + TestBuildInternalLogAttributesEvaluatorDefinition pass.

Validation

• tox run -e black -c ../../../eng/tox/tox.ini --root . — clean
• pytest tests/unittests/test_evaluate.py::TestBuildInternalLogAttributesEvaluatorDefinition tests/unittests/test_evaluate.py::TestBuildInternalLogAttributesThreshold -v — 10 passed
• Reproduced the original crash with a 2-line repro before the fix; confirmed the fix prevents it.