Skip to content

Commit 544a6b9

Browse files
jsonbaileyclaude
andcommitted
docs: Warn about Task lifecycle in Evaluator.evaluate
The returned Task starts executing immediately, but asyncio only holds a weak reference to it. If the caller drops the Task before it completes, garbage collection can cancel it mid-execution and the judge evaluations silently never run. Document the strong- reference contract so callers know to retain it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 677e790 commit 544a6b9

1 file changed

Lines changed: 3 additions & 3 deletions

File tree

packages/sdk/server-ai/src/ldai/evaluator.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,9 +40,9 @@ def evaluate(
4040
"""
4141
Run all configured judges against the given input/output pair.
4242
43-
Schedules the judge evaluations as an asyncio Task and returns it
44-
immediately. The caller can await the task to get results or pass it
45-
to tracking helpers.
43+
The returned Task starts judge evaluations immediately. The caller
44+
must retain a reference to the results until it resolves to ensure
45+
Judge evaluations are completed.
4646
4747
:param input_text: The input that was provided to the AI model
4848
:param output_text: The AI-generated output to evaluate

0 commit comments

Comments
 (0)