Feat/splunk distribution poc#13
Draft
kylehounslow wants to merge 7 commits into
Draft
Conversation
…roject#224) Adds compat/ package and docker-compose.compat.yml overlay that accepts telemetry from Datadog, Jaeger (legacy wire protocol), and Splunk HEC agents and forwards OTLP to the base observability-stack collector. Includes customer-facing documentation under docs/starlight-docs and a Writing Tenets section in CONTRIBUTING.md. The overlay is opt-in and follows the existing INCLUDE_COMPOSE_* activation pattern. The base collector, its config, and all existing pipelines are unchanged. Activation: echo 'INCLUDE_COMPOSE_COMPAT=docker-compose.compat.yml' >> .env docker compose up -d Compat collector pipelines: - traces: [datadog, jaeger] - metrics: [datadog, statsd, splunk_hec] - logs: [datadog, splunk_hec] Modern OpenTelemetry SDK applications bypass the compat hop and send OTLP directly to the base collector on 4317/4318. Public documentation added under /docs/send-data/from-vendor/: - index.md (overview + architecture + decision table) - datadog.md, jaeger.md, splunk.md (per-vendor migration guides) - From Vendor Agents sidebar entry added via astro.config.mjs - Cross-link added from /docs/send-data/ overview Validated end-to-end with real vendor SDKs: - Datadog: dd-trace-py + FastAPI + patch_all() -> 42 spans in OpenSearch with service.name, service.version, deployment.environment.name preserved; instrumentationScope Datadog; parent/child chains intact - Splunk HEC: splunk_handler (Python logging) -> 5 log records in logs-otel-v1-* with com.splunk.source/sourcetype/index preserved - Jaeger OTLP: jaegertracing/example-hotrod demo -> 40 spans, 6-service topology, visible in APM and Discover Traces Starlight docs build validated (115 pages, all internal links resolve). SignalFx is not supported. The upstream signalfxreceiver is deprecated with explicit guidance to migrate to OTLP. Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
…h-project#225) Signed-off-by: ps48 <pshenoy36@gmail.com>
…oject#227) Signed-off-by: ps48 <pshenoy36@gmail.com>
Local POC overlay that inserts Splunk's OTel Collector distribution as a tee
in front of the base otel-collector. Demo apps send OTLP to splunk-otel-collector
(redirected via OTEL_COLLECTOR_HOST override), which fans out to:
- Splunk Observability Cloud (APM via otlphttp, IM via signalfx, Log Observer via splunk_hec)
- base otel-collector -> Data Prepper -> OpenSearch (unchanged existing path)
Purpose: ground-truth what Splunk Observability Cloud accepts from otel-demo so we
can make informed decisions about compat overlay receiver scope (signalfxreceiver,
zipkinreceiver) and the 'drop-in for Splunk Observability Cloud' framing.
Files:
- docker-compose.splunk-demo.yml: compose overlay adding splunk-otel-collector service
- docker-compose/splunk-otel-collector/config.yaml: container-friendly Splunk collector config
(trimmed from Splunk's agent_config.yaml — host-level receivers and smartagent extensions
removed; OTLP-only receivers; tee exporter to base otel-collector:4317)
- docker-compose/splunk-otel-collector/README.md: how to run and verify
- .env.splunk-poc.example: credential + demo-redirect template
- .gitignore: ignore the real .env.splunk-poc
Not a production configuration. Local exploration only.
Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
.env.splunk-poc now sets INCLUDE_COMPOSE_OTEL_DEMO so the otel-demo overlay is pulled in automatically via the base compose's include directive. Run command drops from three -f flags to two. Also switches README example from 'docker compose' to 'finch compose' to match the project's default container runtime. Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
Applies technical-writing.md and otel-contrib-pr-rubric.md:
- README: strip narrative prose, us-vs-them framing, and AI-tone defensives;
drop docker commands in favor of the finch path users actually run.
- README: document finch's --env-file substitution limitation explicitly
(cannot feed ${VAR} expansion in compose files) and show the working
append-and-restore workflow.
- README: add real validation caveat for splunk_hec 404 on /v1/log with the
access token reused as HEC token; note logs still land in OpenSearch.
- config.yaml: terse header, one-line comments per exporter, no rationale
prose.
- docker-compose.splunk-demo.yml: two-line header, one-line port comment.
- .env.splunk-poc.example: remove explanatory prose, leave only locations
and required vars.
No behavioral changes.
Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
|
✅ Docs preview build completed successfully! All tests passed. |
- Point HEC at Splunk Cloud Platform (port 8088) instead of defunct Observability Cloud /v1/log endpoint - Add insecure_skip_verify for trial cert mismatch - Add SPLUNK_HEC_INDEX env var - Update .env.splunk-poc.example with correct guidance for both Splunk products (Observability Cloud + Cloud Platform) Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
|
✅ Docs preview build completed successfully! All tests passed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Describe what this change achieves.
Issues Resolved
List any issues this PR will resolve, e.g. Closes [...].
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.