feat: Add Cohere instrumentation package scaffolding#4418
feat: Add Cohere instrumentation package scaffolding#4418Nik-Reddy wants to merge 5 commits intoopen-telemetry:mainfrom
Conversation
|
Hi, Is there any way we could split this PR into two smaller PRs, one PR for chat completions and another for streaming. This is something, I believe, previous new instrumentations have followed, and it would make the PR easier to review. Thank you |
449c86a to
30d10ec
Compare
|
Hi @JWinermaSplunk, thanks for the feedback. Done! I've split it as requested:
This should make each PR easier to review. Let me know if you'd like any further changes! |
30d10ec to
ddb7243
Compare
|
Hi, this PR adds Cohere chat completions instrumentation following the same patterns as the existing OpenAI instrumentation. It was split from the original submission per @JWinermaSplunk's review feedback (streaming support is in #4421). Would appreciate a review from the GenAI instrumentation maintainers. Thanks! |
ddb7243 to
02af7c8
Compare
|
Rebased on latest main. This PR adds Cohere chat completions instrumentation following the same patterns as the OpenAI instrumentation. Streaming support is in the companion PR #4421. Would appreciate a review when the GenAI maintainers have bandwidth. Happy to address any feedback. |
|
Can you please create a base folder in this format #4179 before adding instrumentation to this library? Alongside do you have bandwidth to be a maintainer for cohere instrumentation? If yes, please feel free to add your name as a maintainer in the component owner yml file. |
|
Hi @eternalcuriouslearner, thanks for sharing info! Done - I've added the full base folder scaffolding following the Anthropic pattern (PR #4179): New files added:
CI/repo integration:
Maintainer:
Yes, I absolutely have bandwidth to maintain the Cohere instrumentation! Happy to own it going forward. Let me know if anything else is needed! |
|
@Nik-Reddy Can you please remove the chat completions from this PR? Can you add that as a part of subsequent pr after this is merged. This reduces verbosity of the pr and helps merging it faster. |
|
Hey @eternalcuriouslearner — totally makes sense, I'll strip the chat completions out and keep this PR to just the scaffolding/base folder. Will push the slimmed-down version shortly. Chat completions can go in as a follow-up once this lands. Thanks for the guidance! |
c91cfc4 to
dc0fd34
Compare
251ae2e to
1fdfdfe
Compare
There was a problem hiding this comment.
Pull request overview
Adds initial scaffolding for a new GenAI instrumentation package, opentelemetry-instrumentation-cohere, and wires it into repo tooling (tox, CI, release workflows) so it can be developed and released following the existing GenAI instrumentation patterns.
Changes:
- Introduces the
opentelemetry-instrumentation-coherepackage structure (pyproject, stubCohereInstrumentor, versioning, docs, examples, and basic tests). - Integrates Cohere into repo automation: tox envs, GitHub Actions test/lint jobs, release workflows, and instrumentation bootstrap exclusion.
- Updates repo metadata/docs: GenAI instrumentation index, releasing docs, and component ownership.
Reviewed changes
Copilot reviewed 33 out of 33 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| tox.ini | Adds Cohere test/lint envs and dependencies. |
| scripts/generate_instrumentation_bootstrap.py | Excludes Cohere from default bootstrap list while in early development. |
| pyproject.toml | Includes Cohere package in workspace and excludes its tests/examples from repo-level tooling scope. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/tests/test_instrumentor.py | Adds basic tests for stub instrumentor lifecycle and dependency declaration. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/tests/requirements.oldest.txt | Defines oldest-deps test set for Cohere instrumentation. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/tests/requirements.latest.txt | Defines latest-deps test set for Cohere instrumentation. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/tests/conftest.py | Adds OTel SDK fixture providers for tests. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/tests/init.py | Initializes test package. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/src/opentelemetry/instrumentation/cohere/version.py | Defines package version constant. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/src/opentelemetry/instrumentation/cohere/utils.py | Placeholder for future Cohere helper utilities. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/src/opentelemetry/instrumentation/cohere/patch.py | Placeholder for future Cohere patching logic. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/src/opentelemetry/instrumentation/cohere/package.py | Declares instrumented dependency (cohere >= 5.0.0). |
| instrumentation-genai/opentelemetry-instrumentation-cohere/src/opentelemetry/instrumentation/cohere/init.py | Adds stub CohereInstrumentor and module docs/usage example. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/pyproject.toml | Adds Cohere package metadata, deps, and entry point. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/examples/zero-code/requirements.txt | Adds dependencies for zero-code example. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/examples/zero-code/main.py | Adds placeholder zero-code example script. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/examples/zero-code/README.rst | Adds zero-code example instructions. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/examples/zero-code/.env | Adds example env vars. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/examples/manual/requirements.txt | Adds dependencies for manual example. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/examples/manual/main.py | Adds placeholder manual setup example script. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/examples/manual/README.rst | Adds manual example instructions. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/examples/manual/.env | Adds example env vars. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/README.rst | Adds package-level documentation and configuration notes. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/LICENSE | Adds Apache 2.0 license file for the package. |
| instrumentation-genai/opentelemetry-instrumentation-cohere/CHANGELOG.md | Adds initial changelog entry for scaffolding. |
| instrumentation-genai/README.md | Registers Cohere in the GenAI instrumentation list. |
| RELEASING.md | Adds Cohere to package release documentation lists. |
| .github/workflows/test.yml | Adds Cohere test jobs across Python versions (oldest/latest). |
| .github/workflows/package-release.yml | Adds Cohere to release workflow allowlist. |
| .github/workflows/package-prepare-release.yml | Adds Cohere to prepare-release workflow allowlist. |
| .github/workflows/package-prepare-patch-release.yml | Adds Cohere to prepare-patch-release workflow allowlist. |
| .github/workflows/lint.yml | Adds Cohere lint job. |
| .github/component_owners.yml | Adds Cohere instrumentation component owner entry. |
| requires-python = ">=3.9" | ||
| authors = [ |
There was a problem hiding this comment.
requires-python is set to ">=3.9", but this package depends on opentelemetry-util-genai, which currently requires Python >=3.10 (see util/opentelemetry-util-genai/pyproject.toml). As-is, installing this package on 3.9 will fail and the metadata/classifiers would be incorrect—either bump requires-python (and drop the 3.9 classifier) or remove/replace the dependency so 3.9 is actually supported.
| "opentelemetry-api ~= 1.39", | ||
| "opentelemetry-instrumentation ~= 0.60b0", | ||
| "opentelemetry-semantic-conventions ~= 0.60b0", | ||
| "opentelemetry-util-genai", |
There was a problem hiding this comment.
The opentelemetry-util-genai dependency is unversioned here. Other GenAI instrumentations pin a compatible range (e.g., >= 0.2b0, < 0.4b0 in Anthropic/Claude Agent SDK) to avoid pulling in incompatible future releases. Please add an explicit version range that matches the API surface this instrumentation will use.
| "opentelemetry-util-genai", | |
| "opentelemetry-util-genai >= 0.2b0, < 0.4b0", |
| Usage | ||
| ----- | ||
|
|
||
| This section describes how to set up Cohere instrumentation if you're setting OpenTelemetry up manually. | ||
| Check out the `manual example <examples/manual>`_ for more details. | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| from cohere import ClientV2 | ||
| from opentelemetry.instrumentation.cohere import CohereInstrumentor | ||
|
|
||
| CohereInstrumentor().instrument() | ||
|
|
||
| client = ClientV2() | ||
| response = client.chat( | ||
| model="command-r-plus", | ||
| messages=[ | ||
| {"role": "user", "content": "Hello, how are you?"}, | ||
| ], | ||
| ) |
There was a problem hiding this comment.
This usage snippet shows client.chat(...) immediately after calling CohereInstrumentor().instrument(), which implies the call will be instrumented. Since _instrument() is currently a no-op, this example is misleading—either replace it with a placeholder/no-op example or add an explicit note that no telemetry will be produced until chat completions patching is implemented.
| """ | ||
| Cohere client instrumentation supporting `cohere`, it can be enabled by | ||
| using ``CohereInstrumentor``. | ||
|
|
||
| .. _cohere: https://pypi.org/project/cohere/ | ||
|
|
||
| Usage | ||
| ----- | ||
|
|
||
| .. code:: python | ||
|
|
||
| from cohere import ClientV2 | ||
| from opentelemetry.instrumentation.cohere import CohereInstrumentor | ||
|
|
||
| CohereInstrumentor().instrument() | ||
|
|
||
| client = ClientV2() | ||
| response = client.chat( | ||
| model="command-r-plus", | ||
| messages=[ | ||
| {"role": "user", "content": "Write a short poem on open telemetry."}, | ||
| ], | ||
| ) | ||
|
|
There was a problem hiding this comment.
This module docstring includes a full client.chat(...) usage example, but the instrumentor’s _instrument() method is currently a no-op. Please update the docstring to either reflect the current scaffold state or explicitly state that chat instrumentation will be added in a follow-up PR, to avoid implying spans will be generated today.
| - `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` configures | ||
| Cohere instrumentation to capture prompt and completion contents on | ||
| events. |
There was a problem hiding this comment.
This README says the .env setting configures Cohere instrumentation to capture prompt/completion content on events, but this instrumentation is currently a no-op scaffold (and there is no chat patching yet). Please add a note here (similar to the package README) that the setting has no effect until chat completions support is implemented.
| - `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` configures | |
| Cohere instrumentation to capture prompt and completion contents on | |
| events. | |
| - `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` is included for | |
| consistency with other GenAI instrumentations, but it currently has no | |
| effect for Cohere. Cohere chat completions support has not yet been | |
| implemented, so prompt and completion contents are not captured on events. |
| - `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` configures | ||
| Cohere instrumentation to capture prompt and completion contents on | ||
| events. |
There was a problem hiding this comment.
This README says the .env setting configures Cohere instrumentation to capture prompt/completion content on events, but the instrumentor is currently a no-op scaffold. Please clarify here that the env var won’t have any effect until chat completions support is added.
| - `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` configures | |
| Cohere instrumentation to capture prompt and completion contents on | |
| events. | |
| - `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` is the setting | |
| that will control whether Cohere instrumentation captures prompt and | |
| completion contents on events, but it currently has no effect because | |
| the instrumentor is still a no-op scaffold and chat completions support | |
| has not yet been added. |
| ; instrumentation-cohere | ||
| py3{9,10,11,12,13,14}-test-instrumentation-cohere-{oldest,latest} | ||
| lint-instrumentation-cohere |
There was a problem hiding this comment.
This envlist includes py39 for Cohere, but the package depends on opentelemetry-util-genai (Python >=3.10). Unless you plan to make util-genai (and this instrumentation) work on 3.9, remove py39 from the Cohere tox factors to prevent guaranteed CI failures.
| py39-test-instrumentation-cohere-oldest_ubuntu-latest: | ||
| name: instrumentation-cohere-oldest 3.9 Ubuntu | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - name: Checkout repo @ SHA - ${{ github.sha }} | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python 3.9 | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.9" | ||
|
|
||
| - name: Install tox | ||
| run: pip install tox-uv | ||
|
|
||
| - name: Run tests | ||
| run: tox -e py39-test-instrumentation-cohere-oldest -- -ra | ||
|
|
||
| py39-test-instrumentation-cohere-latest_ubuntu-latest: | ||
| name: instrumentation-cohere-latest 3.9 Ubuntu | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - name: Checkout repo @ SHA - ${{ github.sha }} | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python 3.9 | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.9" | ||
|
|
||
| - name: Install tox | ||
| run: pip install tox-uv | ||
|
|
||
| - name: Run tests | ||
| run: tox -e py39-test-instrumentation-cohere-latest -- -ra | ||
|
|
There was a problem hiding this comment.
These CI jobs run the Cohere instrumentation tests on Python 3.9. Given the dependency on opentelemetry-util-genai (requires Python >=3.10), these jobs will fail at install time unless the Python support is adjusted. Please remove the 3.9 jobs or raise the minimum supported Python version consistently across the package/tox/CI.
| py39-test-instrumentation-cohere-oldest_ubuntu-latest: | |
| name: instrumentation-cohere-oldest 3.9 Ubuntu | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 30 | |
| steps: | |
| - name: Checkout repo @ SHA - ${{ github.sha }} | |
| uses: actions/checkout@v4 | |
| - name: Set up Python 3.9 | |
| uses: actions/setup-python@v5 | |
| with: | |
| python-version: "3.9" | |
| - name: Install tox | |
| run: pip install tox-uv | |
| - name: Run tests | |
| run: tox -e py39-test-instrumentation-cohere-oldest -- -ra | |
| py39-test-instrumentation-cohere-latest_ubuntu-latest: | |
| name: instrumentation-cohere-latest 3.9 Ubuntu | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 30 | |
| steps: | |
| - name: Checkout repo @ SHA - ${{ github.sha }} | |
| uses: actions/checkout@v4 | |
| - name: Set up Python 3.9 | |
| uses: actions/setup-python@v5 | |
| with: | |
| python-version: "3.9" | |
| - name: Install tox | |
| run: pip install tox-uv | |
| - name: Run tests | |
| run: tox -e py39-test-instrumentation-cohere-latest -- -ra |
| .. |pypi| image:: https://badge.fury.io/py/opentelemetry-instrumentation-cohere.svg | ||
| :target: https://pypi.org/project/opentelemetry-instrumentation-cohere/ | ||
|
|
||
| This library allows tracing applications that use the `Cohere Python SDK <https://pypi.org/project/cohere/>`_. |
There was a problem hiding this comment.
The README currently states the library "allows tracing" Cohere SDK usage, but the CohereInstrumentor implementation in this PR is a no-op scaffold. Please adjust this description to make it clear that chat/completions instrumentation is not implemented yet (so users shouldn’t expect spans/logs until the follow-up PR lands).
| This library allows tracing applications that use the `Cohere Python SDK <https://pypi.org/project/cohere/>`_. | |
| This library provides the `Cohere Python SDK <https://pypi.org/project/cohere/>`_ instrumentation package, but chat/completions tracing is not implemented yet in this PR. Users should not expect spans or logs from Cohere SDK usage until the follow-up PR lands. |
| You should see a poem generated by Cohere while traces and logs export to your | ||
| configured observability tool. |
There was a problem hiding this comment.
This README claims you should see a poem generated by Cohere and telemetry exported, but examples/manual/main.py currently just prints a message and does not invoke the Cohere SDK. Please update the README to match the current placeholder behavior (or implement the example call).
| You should see a poem generated by Cohere while traces and logs export to your | |
| configured observability tool. | |
| At the moment, this example only runs the placeholder scaffold in | |
| `main.py <main.py>`_. It does not yet invoke the Cohere SDK or export traces | |
| and logs to your observability tool. |
Implements initial Cohere instrumentation following the GenAI semantic conventions. Supports sync and async chat completions with token usage, content capture, and error handling. Streaming support will follow in a separate PR. Ref open-telemetry#3050
Add base folder structure following the pattern established by the Anthropic instrumentation (PR open-telemetry#4179), as requested by @eternalcuriouslearner: - CHANGELOG.md with initial entry - Examples: manual and zero-code instrumentation examples - Test requirements: oldest and latest dependency matrices - CI integration: lint job, test matrix (Python 3.9-3.14), release workflows - tox.ini: test and lint environment configurations - component_owners.yml: Nik-Reddy as Cohere instrumentation maintainer - README updates: instrumentation-genai table, pyproject.toml pyright config - RELEASING.md: add Cohere to independently released packages list - Bootstrap script: exclude from default auto-instrumentation (early dev)
Per reviewer feedback, this PR now contains only the base folder scaffolding and CI integration for the Cohere instrumentation package. Chat completions implementation will be added in a follow-up PR.
Trim changelog to reflect scaffolding only, clarify README sections that referenced chat completions not yet implemented, remove _supports_metrics since the stub emits nothing, and add basic test_instrumentor.py following the Anthropic test pattern.
f9e39f7 to
4dc0dde
Compare
Fixes #3050 (partial - scaffolding only)
Adds the base folder structure and CI integration for the Cohere instrumentation package, following the same pattern as the Anthropic instrumentation (PR #4179). Chat completions implementation will come in a follow-up PR per reviewer feedback.
What's included:
Follow-up PRs: