Investigate better typing configuration for core integrations

[anakin87]

Most of our core integrations have a [[tool.mypy.overrides]] section in pyproject, containing several libraries.

This is usually done to avoid mypy errors like
error: Cannot find implementation or library stub for module named "..." [import-not-found].

The error above appears because we are running mypy with the --explicit-package-bases flag. This was likely done to support namespace packages. More info in mypy docs: https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-paths-to-modules.

My impression is that using these workarounds does not guarantee type safety: we may treat the types coming from haystack and other libraries as Any.
Now that we are adding explicit support for types (deepset-ai/haystack#9329), we should evaluate how to better configure typing in core integrations.

[anakin87]

I investigated briefly. Here are some notes.

The lint environment needs the dependencies, so it should not be detached.

• Removing the detached attribute removes several import errors and converts the [import-not-found] error to [import-untyped] if the library has no stubs/py.typed
• At the same time, other proper type errors start appearing if we test with real dependencies as we should have done from the beginning

ignore_missing_imports should only be applied to libraries with no stubs/py.typed.

• TODO: investigate if we can avoid including "haystack_integrations.*" in ignore_missing_imports (it seems not to play well with namespace packages)

If we end up with Haystack being explicitly typed, it does not need additional config in the pyproject of core integrations; otherwise, we can add it to follow_untyped_imports.

Example changes for Bedrock pyproject

[tool.hatch.envs.lint]
# REMOVED LINE detached = true
installer = "uv"
dependencies = ["pip", "black>=23.1.0", "mypy>=1.0.0", "ruff>=0.0.243"]

[[tool.mypy.overrides]]
module = ["haystack.*"]
follow_untyped_imports = true # to use type hinting even if currently we don't provide stubs/py.typed

[[tool.mypy.overrides]]
module = ["botocore.*", "haystack_integrations.*"]
ignore_missing_imports = true

UPDATE: Haystack now provides types support.

[anakin87]

Once types are properly checked in a specific integration, we may also add a py.typed file to that integration, to make types usable in downstream applications (as we did for Haystack in deepset-ai/haystack#9329).

We should understand how py.typed can be applied to namespace packages...

[anakin87]

In short, due to wrong mypy configurations, we were skipping most mypy checks on external libraries (the main point of core integrations).

We would have to refactor each integration a bit.
A complete example can be found in the following Bedrock PR: #1912.