test(plugins): assert error propagation in plugin error fallback tests

Merge #6309

Three plugin "fallback to runner" tests are meant to verify that when the
`on_*_error` callback is disabled, the underlying error is not swallowed and
propagates out through the runner. As written, they do not actually enforce that:

- `test_async_on_tool_error_fallback_to_runner` and
  `test_live_on_tool_error_fallback_to_runner`
  (`tests/unittests/flows/llm_flows/test_plugin_tool_callbacks.py`) are shaped as:

  ```python
  try:
      await invoke_tool_with_plugin(mock_error_tool, mock_plugin)
  except Exception as e:
      assert e == mock_error
  ```

  The only assertion lives inside the `except`. If the production code ever stops
  raising (exactly the regression these tests exist to catch), the `try` body
  completes normally, the `except` never runs, the assertion is skipped, and the
  test still passes green.

- `test_on_model_error_callback_fallback_to_runner`
  (`tests/unittests/flows/llm_flows/test_plugin_model_callbacks.py`) is worse: its
  `try` only constructs `InMemoryRunner(agent, plugins=[...])`, which never runs the
  agent, so no error path is exercised at all and the test can never fail.

All three pass today over correct code, so this is latent false safety, not a live
bug: they would not catch the swallow-the-error regression they are named for.

**Solution:**

Rewrite the three test bodies so a no-raise regression fails them, following
conventions already used in this repo. No production code changes.

- Tool tests: wrap the call in `pytest.raises(ClientError)` and assert the captured
  error, so the test fails if nothing is raised:

  ```python
  with pytest.raises(ClientError) as exc_info:
      await invoke_tool_with_plugin(mock_error_tool, mock_plugin)
  assert exc_info.value == mock_error
  ```

- Model test: actually run the agent, then assert the unhandled model error surfaces
  as a terminal error event. On the synchronous runner an unhandled model error is
  converted into an error event rather than raised to the caller, so `pytest.raises`
  is not the right shape here; this mirrors the existing error-event assertions in
  `tests/unittests/flows/llm_flows/test_base_llm_flow.py`:

  ```python
  runner = testing_utils.InMemoryRunner(agent, plugins=[mock_plugin])
  events = runner.run('test')
  error_events = [e for e in events if e.error_code]
  assert len(error_events) == 1
  assert error_events[0].error_code == 'ClientError'
  ```

Closes #6309

Co-authored-by: Jason Zhang <jasoncz@google.com>
PiperOrigin-RevId: 947861608

Author: 2 people

tests/unittests/flows/llm_flows/test_plugin_model_callbacks.py

@@ -171,18 +171,20 @@ def test_on_model_error_callback_with_plugin(mock_plugin):
 
 
 def test_on_model_error_callback_fallback_to_runner(mock_plugin):
-  """Tests that the model error is not handled and falls back to raise from runner."""
+  """Tests that the model error is not handled and surfaces from the runner."""
   mock_model = testing_utils.MockModel.create(error=mock_error, responses=[])
   mock_plugin.enable_on_model_error_callback = False
   agent = Agent(
       name='root_agent',
       model=mock_model,
   )
 
-  try:
-    testing_utils.InMemoryRunner(agent, plugins=[mock_plugin])
-  except Exception as e:
-    assert e == mock_error
+  runner = testing_utils.InMemoryRunner(agent, plugins=[mock_plugin])
+
+  events = runner.run('test')
+  error_events = [e for e in events if e.error_code]
+  assert len(error_events) == 1
+  assert error_events[0].error_code == 'ClientError'
 
 
 if __name__ == '__main__':

tests/unittests/flows/llm_flows/test_plugin_tool_callbacks.py

@@ -180,10 +180,9 @@ async def test_async_on_tool_error_fallback_to_runner(
 ):
   mock_plugin.enable_on_tool_error_callback = False
 
-  try:
+  with pytest.raises(ClientError) as exc_info:
     await invoke_tool_with_plugin(mock_error_tool, mock_plugin)
-  except Exception as e:
-    assert e == mock_error
+  assert exc_info.value == mock_error
 
 
 async def invoke_tool_with_plugin_live(
@@ -258,10 +257,9 @@ async def test_live_on_tool_error_fallback_to_runner(
 ):
   mock_plugin.enable_on_tool_error_callback = False
 
-  try:
+  with pytest.raises(ClientError) as exc_info:
     await invoke_tool_with_plugin_live(mock_error_tool, mock_plugin)
-  except Exception as e:
-    assert e == mock_error
+  assert exc_info.value == mock_error
 
 
 @pytest.mark.asyncio