Skip to content

feat: enable APM RED metrics ingestion into Prometheus via Data Prepper#35

Merged
anirudha merged 1 commit into
opensearch-project:mainfrom
ps48:apm-data-prepper
Mar 3, 2026
Merged

feat: enable APM RED metrics ingestion into Prometheus via Data Prepper#35
anirudha merged 1 commit into
opensearch-project:mainfrom
ps48:apm-data-prepper

Conversation

@ps48

@ps48 ps48 commented Mar 2, 2026

Copy link
Copy Markdown
Member

Description

Configures Data Prepper's service-map pipeline to route trace-derived RED (Rate, Error, Duration) metrics to Prometheus via its new prometheus sink, using Data Prepper 2.15.0-SNAPSHOT.

Issues Resolved

  • pipelines.yaml: Add service_processed_metrics route to the service-map pipeline and a prometheus sink that writes RED metrics to Prometheus via remote write
  • .env: Bump Data Prepper to 2.15.0-SNAPSHOT, add DATA_PREPPER_DOCKER_REPO variable for configurable image source
  • docker-compose.yml: Use DATA_PREPPER_DOCKER_REPO env var for the Data Prepper image

This enables the APM service map page in OpenSearch Dashboards to display request rate, error rate, and latency metrics sourced from Prometheus, completing the traces → service map → RED metrics → Prometheus pipeline.

Related upstream PR: opensearch-project/data-prepper#6595

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: ps48 <pshenoy36@gmail.com>
@anirudha anirudha merged commit 4b411a9 into opensearch-project:main Mar 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants