Skip to content

Updates to the ChatGenerator protocol #10992

@sjrl

Description

@sjrl

Two ideas for updating the ChatGenerator protocol:

  1. Update the return type of the run method to at least require replies: list[ChatMessage] since we rely on that key being present when we use the protocol in the codebase. This should be possible through using TypedDict, total=False and Required (still needs to be tested). So something like
from typing import Protocol
from typing_extensions import Required, TypedDict
from haystack.dataclasses import ChatMessage


class ChatGeneratorOutput(TypedDict, total=False):
    """
    Return type for Chat Generators.

    ``replies`` is always required. Implementations may include additional keys.
    """
    replies: Required[list[ChatMessage]]


class ChatGenerator(Protocol):
    def run(self, messages: list[ChatMessage]) -> ChatGeneratorOutput:
  1. Consider adding a run_async to the protocol since we have finished adding run_async to all of our chat generators in core and in core-integrations Add async run method for ChatGenerators in haystack core #8858 and Add async run method for all ChatGenerators haystack-core-integrations#1379. Only a consideration since it's not guaranteed that community made components follow the protocol, but we are starting to add run_async methods to components that use the ChatGenerator protocol so I wonder if creating a second protocol (e.g. ChatGeneratorWithAsync) would be an alternative.

👋 Hello there! This issue will be handled internally and isn't open for external contributions. If you'd like to contribute, please take a look at issues labeled contributions welcome or good first issue. We'd really appreciate it!

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priority, add to the next sprint if no P1 available

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions