fix(eval): wire App plugins through eval paths

[doughayden]

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

• Closes: # cli_eval bypasses App.plugins, breaking observability plugins (e.g., BigQueryAgentAnalyticsPlugin) during eval runs #5503

Problem:

Eval inference builds its Runner from the bare root_agent, so the wrapping App's plugins never apply. EvaluationGenerator._generate_inferences_from_root_agent constructs Runner(app_name=..., agent=root_agent, plugins=[<internal eval plugins>]), which silently drops app.plugins for the duration of an eval run.

Three non-live surfaces reach that leaf, and all three build LocalEvalService without the App:

• the adk eval CLI (cli_eval / cli_tools_click), which is what # cli_eval bypasses App.plugins, breaking observability plugins (e.g., BigQueryAgentAnalyticsPlugin) during eval runs #5503 originally reported
• AgentEvaluator (the evaluate / evaluate_eval_set path), the programmatic and pytest entry point, so a scripted or CI eval bypasses observability and guardrail plugins
• the adk web eval tab (dev_server.run_eval)

The result is that an eval scores a different agent than adk web chat and the deployed server, both of which run the full App. It also degrades the LLM-judge metrics that read app_details (developer instructions plus tool declarations), since those are shaped by plugins the eval never applied.

Solution:

Thread app= from all three callers into the shared inference leaf, which then builds the eval Runner from app.model_copy(update={"plugins": list(app.plugins) + internal_eval_plugins, "root_agent": root_agent}). The copy leaves the caller's App untouched, merges the internal eval plugins with the user's rather than replacing them, and overrides root_agent so the Runner still targets whatever agent the caller asked to evaluate, a sub-agent included. The App's context_cache_config and resumability_config come along with it. When no App is present, the bare-agent construction is unchanged.

This also matches where the Runner API is already heading, since Runner(plugins=...) is deprecated in favor of app=.

Runner construction is shared between the non-live and live leaves through _build_eval_runner_kwargs, so use_live=True carries the App the same way. Agent resolution in cli_eval gains get_app_or_root_agent, which surfaces the App alongside the root agent and keeps get_root_agent as a thin wrapper for existing callers.

Testing Plan

Unit Tests:

• I have added or updated unit tests for my change.
• All unit tests pass locally.

New tests cover each surface: the CLI resolver (App present, App absent, app attribute that is not an App, get_agent_async modules, and the back-compat wrapper), AgentEvaluator._get_agent_for_eval returning the pair, LocalEvalService forwarding app on both the live and non-live branches, and the Runner construction itself asserting that agent=/plugins= are not passed when app= is and that the user's plugin survives the merge.

$ pytest tests/unittests/evaluation tests/unittests/cli/utils -q
851 passed, 96 warnings in 25.49s

Manual End-to-End (E2E) Tests:

A reproduction lives at doughayden/adk-issue-examples/06-eval_drops_app_plugins. An LlmAgent is wrapped in an App whose plugins include a SentinelPlugin that appends to a module-level list every time it runs inside a Runner, so "did the App's plugins apply during eval" is directly observable. The script runs one real eval through the public AgentEvaluator.evaluate entry point. Nothing is patched or stubbed in either run, so the second one exercises the full caller chain (AgentEvaluator to LocalEvalService to EvaluationGenerator).

Released google-adk[eval]==2.5.0, from the example directory:

$ uv run main.py
google-adk 2.5.0
App-aware eval inference: False
App-aware live eval inference: False

🐞 Running an ADK build without the fix.

App-level plugin invocations during eval inference: []
✅ Bug reproduced: the App's plugins were dropped — none ran during eval inference, so the eval scored the bare root_agent.

The same script against this branch, with only the ADK build swapped:

$ uv run --isolated --no-project \
    --with "google-adk[eval] @ git+https://github.com/doughayden/adk-python@fix/eval-app-aware-inference" \
    --with python-dotenv \
    main.py
google-adk 2.5.0
App-aware eval inference: True
App-aware live eval inference: True

🔧 Running an ADK build that carries the App through eval.

App-level plugin invocations during eval inference: ['sentinel']
✅ Fix verified end to end: the App's plugins ran during eval inference, so the eval scores the same agent production runs.

(An unrelated vertexai.preview.rag deprecation warning is elided from both outputs.)

The live path is covered by unit tests rather than here, since exercising it needs a live-API session.

Checklist

• I have read the CONTRIBUTING.md document.
• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.
• I have manually tested my changes end-to-end.
• Any dependent changes have been merged and published in downstream modules.

Additional context

This builds on #5534, which threads the App through cli_eval and evaluation_generator and fixes the CLI path. The work here keeps that approach and extends it to the two surfaces #5534 does not reach, AgentEvaluator and dev_server, plus the live inference leaf that landed after it was opened.

get_root_agent recently became async and gained get_agent_async support, so get_app_or_root_agent mirrors that shape and handles both entrypoints. Existing get_root_agent callers are unaffected.