You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adds compat/ package and docker-compose.compat.yml overlay that accepts
telemetry from Datadog, Jaeger (legacy wire protocol), and Splunk HEC
agents and forwards OTLP to the base observability-stack collector.
Includes customer-facing documentation under docs/starlight-docs.
Architecture: dedicated otel-collector-compat edge collector runs the
upstream collector-contrib receivers for each vendor, batches, and
forwards OTLP to the base collector — which handles all existing
processor/exporter logic unchanged. No drift risk with the base config.
Modern OTel-SDK apps (including Jaeger v1.42+) bypass the compat hop
and send OTLP directly to the base collector.
Activation uses the repo's existing INCLUDE_COMPOSE_* convention:
echo 'INCLUDE_COMPOSE_COMPAT=docker-compose.compat.yml' >> .env
docker compose up -d
Pipelines wired into compat collector:
- traces: [datadog, jaeger]
- metrics: [datadog, statsd, splunk_hec]
- logs: [datadog, splunk_hec]
Public documentation added under /docs/send-data/from-vendor/:
- index.md (overview + architecture + quickstart)
- datadog.md, jaeger.md, splunk.md (per-vendor migration guides)
- 'From Vendor Agents' sidebar entry added via astro.config.mjs
- Link added from /docs/send-data/ overview
Scope and framing:
- This is protocol-compatible ingest, not a 1:1 vendor platform replacement.
- We defer to upstream collector-contrib READMEs and vendor documentation
for attribute schemas and translation behavior. The vendor pages here
document only 3-5 canonical attributes per vendor and link out to the
authoritative sources.
- Component receivers have varying stability tiers upstream (alpha/beta/
development). Vendor pages document the specifics.
SignalFx deliberately omitted — upstream signalfxreceiver was deprecated
2026-02-13 with explicit guidance to migrate to OTLP.
Bundles Jaeger's hotrod demo on port 8080 as a built-in OTLP trace
generator (Jaeger's canonical demo, pointed at the base collector).
Validated: starlight-docs astro build passes with all internal links
valid (115 pages built successfully).
Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+48Lines changed: 48 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -398,6 +398,7 @@ AI coding assistants are welcome to contribute! When contributing as an AI agent
398
398
- Note performance considerations
399
399
- Reference relevant specifications
400
400
- Keep comments up to date
401
+
- Avoid narrative prose and internal team voice (e.g., "we deliberately...") — comments are for future maintainers, not decision history
401
402
402
403
### Examples
403
404
@@ -407,6 +408,53 @@ AI coding assistants are welcome to contribute! When contributing as an AI agent
407
408
- Follow language-specific conventions
408
409
- Test examples before committing
409
410
411
+
### Writing Tenets (for agents)
412
+
413
+
Applies to all user-facing documentation: READMEs, public docs under `docs/`, migration guides, and PR descriptions.
414
+
415
+
**Framing**
416
+
417
+
- No us-vs-them. Write "Point your agents at observability-stack," not "at us." In open-source projects, the reader is part of the same community.
418
+
- Don't leak internal conversation into docs. Design-doc voice ("this is the canonical case where...", "we deliberately omitted...") belongs in PR discussion or design documents, not user-facing artifacts.
419
+
- Factual, not promotional. Avoid marketing phrases like "does ONE thing," "zero drift risk," or "honest limits."
420
+
- Acknowledge nuance via asides (`:::note` in Starlight docs) or italic notes, not prose digressions.
421
+
422
+
**Maintenance hygiene**
423
+
424
+
- Don't pin version strings. Link to `main` of upstream repos (e.g., `opentelemetry-collector-contrib/tree/main/receiver/...`), not to a specific tag. Version pins go stale.
425
+
- Don't duplicate source code in docs. Config YAML, pipeline definitions, and translation tables drift from the real source. Link to the source file instead.
426
+
- Don't maintain per-vendor translation tables beyond 3–5 canonical well-known fields. Defer to upstream receiver READMEs and vendor documentation. Positioning this repo as a schema authority creates permanent maintenance burden.
427
+
- Repo READMEs should link to public docs, not duplicate them. One source of truth per content type.
428
+
429
+
**Accuracy**
430
+
431
+
- Verify specific claims before writing them. Dates, version numbers, protocol behavior, UI terminology — check primary sources.
432
+
- If a claim cannot be verified from primary sources, phrase it more vaguely. "Modern versions support X" beats "as of v1.42, X is supported" when the version claim is unverified.
433
+
- Check existing conventions. Before using a UI name or terminology, grep the rest of the docs to see what other pages call it.
434
+
- Run the documentation build (`npm run build` in `docs/starlight-docs`) before committing doc changes. Verify internal links are valid.
435
+
436
+
**Public-doc page structure**
437
+
438
+
Pages for users migrating TO observability-stack should cover, in order:
439
+
440
+
1. Action-oriented lead (one sentence — what the reader can do)
441
+
2. Decision table when multiple paths exist ("Do I need this?" / "Which path applies?")
442
+
3. Configuration — concrete environment variables, example config, code snippet per path
443
+
4. Verify step — one-command check that it's working
444
+
5. What lands in OpenSearch — concrete example of end state (field names, index patterns)
445
+
6. Caveats — real observed gotchas surfaced from validation, not theoretical ones
446
+
7. Not covered — honest scope boundaries
447
+
8. References — upstream sources, vendor docs
448
+
449
+
**Repo READMEs** are for contributors, not migrators. Keep them short (20–40 lines for leaf READMEs; 100 max for overview). Link out to public docs for user-facing content. Include repo-local context only: config file paths, upstream receiver links, local dev workflow commands.
450
+
451
+
**Caveats from real validation are more trustworthy than theoretical ones.** When end-to-end testing reveals a gotcha (e.g., an attribute gets overwritten, a field doesn't translate), document it in the caveats section. Lead with what the user will see, not why it happens.
452
+
453
+
**Scope discipline**
454
+
455
+
- Prune aggressively when in doubt. Deletion is cheaper than maintenance.
456
+
- Don't commit working files — audit tables, compatibility matrices, session notes, TODO lists, WIP drafts. If it's not useful to a future reader with no context, it's not a docs artifact.
This overlay adds a dedicated OpenTelemetry Collector that accepts Datadog, Jaeger, and Splunk HEC wire protocols, translates them to OTLP, and forwards to the base collector. No application code changes are required — vendor agents are pointed at observability-stack by changing an endpoint URL.
6
+
7
+
## Do I need this overlay?
8
+
9
+
| If your apps emit... | Need this overlay? |
10
+
|----------------------|--------------------|
11
+
| OpenTelemetry OTLP (gRPC or HTTP) | No. Send directly to the base collector on 4317 or 4318. |
OTLP apps ─────────────────────────────────▶ (direct to base, no compat hop)
24
+
```
25
+
26
+
The compat collector uses upstream [`opentelemetry-collector-contrib`](https://github.com/open-telemetry/opentelemetry-collector-contrib) receivers. All enrichment and downstream routing happens in the base pipeline — the compat config is purely ingest + forward.
Each receiver translates vendor-specific data to the OpenTelemetry data model. Translation behavior is defined by the upstream receivers. For schema details, consult:
76
+
77
+
- The upstream receiver READMEs under [`opentelemetry-collector-contrib/receiver/`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver)
78
+
- The vendor's own instrumentation and tagging documentation
├── datadog/README.md ← developer notes + link to migration guide
90
+
├── jaeger/README.md
91
+
└── splunk/README.md
92
+
93
+
docker-compose.compat.yml ← overlay service definitions
94
+
```
95
+
96
+
## Adding a vendor
97
+
98
+
1. Create `vendors/<name>/README.md` with a link to the (forthcoming) public migration guide and developer notes (receiver used, config location, quick local test).
99
+
2. Add the receiver stanza to `collector/config.compat.yaml` and wire it into the appropriate pipeline(s).
100
+
3. Add port mappings to `docker-compose.compat.yml`.
101
+
4. Add a page at `docs/starlight-docs/src/content/docs/send-data/from-vendor/<name>.md` with the user migration guide.
102
+
5. Verify end-to-end: send vendor-format data → confirm it lands in OpenSearch.
Developer notes for `otel-collector-compat` and [`config.compat.yaml`](./config.compat.yaml).
4
+
5
+
For migration guides and usage, see the public docs: https://observability.opensearch.org/docs/send-data/from-vendor/
6
+
7
+
## Role
8
+
9
+
Accepts vendor wire protocols on their native ports, translates to the OpenTelemetry data model via upstream [`opentelemetry-collector-contrib`](https://github.com/open-telemetry/opentelemetry-collector-contrib) receivers, and forwards OTLP to the base collector. All enrichment, filtering, and downstream routing (Data Prepper, Prometheus) happens in the base collector config.
Migration guide and user-facing documentation: https://observability.opensearch.org/docs/send-data/from-vendor/jaeger/
4
+
5
+
Jaeger is an open-source CNCF project rather than a proprietary vendor. This directory exists because users migrating from Jaeger deployments typically look for it alongside other vendor integrations.
6
+
7
+
## Receiver used
8
+
9
+
-[`jaegerreceiver`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/jaegerreceiver) — Jaeger native wire protocol (Thrift HTTP on 14268/tcp, gRPC on 14250/tcp)
Modern Jaeger apps emit OTLP natively and bypass this overlay. The bundled `hotrod` demo (port 8080) is configured this way and exercises the base collector directly.
14
+
15
+
## Quick local test
16
+
17
+
```bash
18
+
# Trigger the bundled hotrod demo (OTLP path)
19
+
curl http://localhost:8080/dispatch?customer=123
20
+
21
+
# View traces at http://localhost:5601 → Trace Analytics
22
+
```
23
+
24
+
The legacy Thrift HTTP path is harder to exercise without a `jaeger-client-*` app. See the upstream receiver README for wire format details.
0 commit comments