SerializationError when using RagasEvaluator with `ragas_metrics`

[michelepangrazi]

Describe the bug

When using the RagasEvaluator component with the ragas_metrics parameter, serializing the pipeline (e.g., using pipeline.dumps()) raises a SerializationError. This occurs because the class does not assign ragas_metrics to a public instance variable, nor implements a custom to_dict() method.

Error Trace

haystack.core.errors.SerializationError: Cannot determine the value of the init parameter 'ragas_metrics' for the class RagasEvaluator. You can fix this error by assigning 'self.ragas_metrics = ragas_metrics' or adding a custom serialization method 'to_dict' to the class.

---

To Reproduce

from haystack import Pipeline

import os
os.environ["AZURE_OPENAI_API_KEY"] = "your-api-key"
os.environ["AZURE_OPENAI_ENDPOINT"] = "your-endpoint"

from ragas.llms import HaystackLLMWrapper
from ragas.metrics import AnswerRelevancy, ContextPrecision, Faithfulness
from haystack_integrations.components.evaluators.ragas import RagasEvaluator
from haystack.components.generators.azure import AzureOpenAIGenerator

llm = AzureOpenAIGenerator()
evaluator_llm = HaystackLLMWrapper(llm)

ragas_evaluator = RagasEvaluator(
    ragas_metrics=[AnswerRelevancy(), ContextPrecision(), Faithfulness()],
    evaluator_llm=evaluator_llm,
)

pipeline = Pipeline()
pipeline.add_component("ragas_evaluator", ragas_evaluator)

print(pipeline.dumps())

Expected behavior

The pipeline should serialize successfully, including the RagasEvaluator component.

Proposed Fix

in the RagasEvaluator component, for better naming consistency, the parameter itself could be renamed from ragas_metrics to metrics, matching the internal attribute:

def __init__(
    self,
    metrics: List[Metric],
    evaluator_llm: Optional[BaseRagasLLM] = None,
    evaluator_embedding: Optional[BaseRagasEmbeddings] = None,
):
    self.metrics = metrics

[michelepangrazi]

Follow-up

I tried the suggested fix by modifying the RagasEvaluator class locally and assigning:

self.ragas_metrics = ragas_metrics

This allows the pipeline to serialize correctly up to that point. However, I'm now hitting a new serialization error, related to the actual metric instances:

haystack.core.errors.SerializationError: Component 'ragas_evaluator' of type 'RagasEvaluator' has an unsupported value of type 'Faithfulness' in the serialized data.

It seems the Faithfulness object from the ragas.metrics module are not natively serializable within the current Haystack component serialization system.

Is there currently a workaround to serialize/deserialize a pipeline that includes Ragas metrics like Faithfulness?