Skip to content

Commit c984097

Browse files
authored
fix: evaluation docstring examples to use value=0 instead of value=None (#1520)
docs: fix Evaluation docstring examples to use value=0 instead of value=None Update docstring examples in EvaluatorFunction and RunEvaluatorFunction protocols to use value=0 instead of value=None, matching the type definition which requires Union[int, float, str, bool] and does not allow None. This change aligns with commit d11155e which established that Evaluation score values cannot be None. Changes: - Update accuracy_evaluator example in EvaluatorFunction docstring - Update llm_judge_evaluator error handling example - Update average_accuracy example in RunEvaluatorFunction docstring - Update accuracy_evaluator example in DatasetClient.run_experiment docstring
1 parent 0246093 commit c984097

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

langfuse/_client/datasets.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -286,7 +286,7 @@ def answer_questions(*, item, **kwargs):
286286
287287
def accuracy_evaluator(*, input, output, expected_output=None, **kwargs):
288288
if not expected_output:
289-
return {"name": "accuracy", "value": None, "comment": "No expected output"}
289+
return {"name": "accuracy", "value": 0, "comment": "No expected output"}
290290
291291
is_correct = output.strip().lower() == expected_output.strip().lower()
292292
return {

langfuse/experiment.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -719,7 +719,7 @@ def __call__(
719719
```python
720720
def accuracy_evaluator(*, input, output, expected_output=None, **kwargs):
721721
if expected_output is None:
722-
return {"name": "accuracy", "value": None, "comment": "No expected output"}
722+
return {"name": "accuracy", "value": 0, "comment": "No expected output"}
723723
724724
is_correct = output.strip().lower() == expected_output.strip().lower()
725725
return {
@@ -773,7 +773,7 @@ async def llm_judge_evaluator(*, input, output, expected_output=None, **kwargs):
773773
except ValueError:
774774
return {
775775
"name": "llm_judge_quality",
776-
"value": None,
776+
"value": 0,
777777
"comment": "Could not parse LLM judge score"
778778
}
779779
```
@@ -867,7 +867,7 @@ def average_accuracy(*, item_results, **kwargs):
867867
accuracy_values.append(evaluation.value)
868868
869869
if not accuracy_values:
870-
return {"name": "avg_accuracy", "value": None, "comment": "No accuracy evaluations found"}
870+
return {"name": "avg_accuracy", "value": 0, "comment": "No accuracy evaluations found"}
871871
872872
avg = sum(accuracy_values) / len(accuracy_values)
873873
return {

0 commit comments

Comments
 (0)