Skip to content

fix: add missing output sockes to LLM evaluators - #12049

Merged
anakin87 merged 2 commits into
mainfrom
fix-llm-evaluators-sockets
Jul 17, 2026
Merged

fix: add missing output sockes to LLM evaluators#12049
anakin87 merged 2 commits into
mainfrom
fix-llm-evaluators-sockets

Conversation

@anakin87

@anakin87 anakin87 commented Jul 17, 2026

Copy link
Copy Markdown
Member

Related Issues

I noticed that these evaluators were missing output sockets, so for example the following Pipeline would have failed at connection:

from haystack import Pipeline, component
from haystack.components.evaluators import ContextRelevanceEvaluator


@component
class ScoreReporter:
    @component.output_types(report=str)
    def run(self, individual_scores: list[float]):
        return {"report": f"received scores: {individual_scores}"}


pipeline = Pipeline()
pipeline.add_component("evaluator", ContextRelevanceEvaluator())
pipeline.add_component("reporter", ScoreReporter())

pipeline.connect("evaluator.individual_scores", "reporter.individual_scores")
# PipelineConnectError, 'evaluator.individual_scores' does not exist.

Proposed Changes:

  • add the missing output sockets and minimally test them

How did you test it?

CI

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

@vercel

vercel Bot commented Jul 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
haystack-docs Ready Ready Preview, Comment Jul 17, 2026 1:28pm

Request Review

@github-actions github-actions Bot added topic:tests type:documentation Improvements on the docs labels Jul 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/evaluators
  context_relevance.py
  faithfulness.py
  llm_evaluator.py
  haystack/core/pipeline
  pipeline.py
Project Total  

This report was generated by python-coverage-comment-action

@anakin87
anakin87 marked this pull request as ready for review July 17, 2026 13:28
@anakin87
anakin87 requested a review from a team as a code owner July 17, 2026 13:28
@anakin87
anakin87 requested review from sjrl and removed request for a team July 17, 2026 13:28

@sjrl sjrl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@anakin87
anakin87 merged commit e8f8897 into main Jul 17, 2026
27 checks passed
@anakin87
anakin87 deleted the fix-llm-evaluators-sockets branch July 17, 2026 13:43
@anakin87 anakin87 self-assigned this Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants