fix: OpenSearch - do not serialize string Secrets + authentication refactoring

[anakin87]

Related Issues

• fixes https://github.com/deepset-ai/haystack-private/issues/268

Proposed Changes:

• avoid serializing plain string Secrets
• simplify authentication handling, similarly to Elasticsearch:
  • do not resolve Secrets in __init__ but in _ensure_initialized
  • fail if only one of username and password is provided
  • properly serialize env vars Secrets

How did you test it?

CI, new unit tests

Notes for the reviewer

To be on the safe side, I'd release a major version.
However, I saw this comment: https://github.com/deepset-ai/haystack-private/issues/268#issuecomment-4055392995

WDYT?

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[sjrl]

@anakin87 looking good! But I think we are missing something. I believe we also need to support dict types coming in --> basically the AWSAuth version but I'm not 100% on what this looks like.

@tstadel could you help out here and check that these changes still work with the platform?

[anakin87]

I believe we also need to support dict types coming in --> basically the AWSAuth version but I'm not 100% on what this looks like.

Could you better explain?

[sjrl]

I believe we also need to support dict types coming in --> basically the AWSAuth version but I'm not 100% on what this looks like.

Could you better explain?

Never mind took a look again at the PR and everything looks good. I think having the explicit types definitely helped me better understand the code.

[sjrl]

So I'd say this looks all good from my point of view. It would be great if @tstadel could give it a final look over.

[tstadel]

I guess that's fine to streamline, to be 100% accurrate better would be to also allow a callable/prototype with this signature (same as AWSAuth's call)

    def __call__(
        self,
        method: str,
        url: str,
        body: Any = None,
        headers: dict[str, str] | None = None,
    ) -> dict[str, str]:

[anakin87]

Makes sense, but we only support the explicitly handled http_auth types via _resolve_http_auth().

Adding a generic callable would be misleading unless we also pass it through at runtime (which we don't do in the current implementation), so I'd keep the type strict for now

[tstadel]

Left one small nit, to allow more custom auth (not relevant for platform).