Skip to content

Commit a136151

Browse files
committed
Add task and worker lifecycle metrics to Celery instrumentation
Introduce Flower-compatible metrics for Celery task and worker events: - flower.events.total: counter for task-sent, task-received, task-started, task-succeeded, task-failed, task-retried, task-revoked - flower.task.runtime.seconds: histogram of task execution time - flower.worker.number.of.currently.executing.tasks: gauge of in-flight tasks per worker - flower.worker.online: gauge tracking worker online/offline status CeleryInstrumentor (child process) handles task tracing and task-level metrics. CeleryWorkerInstrumentor (main process) handles worker lifecycle signals (worker_ready, worker_shutdown) and main-process events (task_received, task_revoked). Prefetch-time metrics (task_prefetch_time_seconds, worker_prefetched_tasks) are intentionally omitted — Celery splits task_received and task_prerun across processes in prefork mode, making it impossible to compute the delta without external event consumption.
1 parent 87a01e6 commit a136151

9 files changed

Lines changed: 1659 additions & 89 deletions

File tree

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1717
([#4244](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4244))
1818
- `opentelemetry-instrumentation-sqlite3`: Add uninstrument, error status, suppress, and no-op tests
1919
([#4335](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4335))
20+
- `opentelemetry-instrumentation-celery`: Add task and worker lifecycle metrics matching Celery Flower
21+
([#4439](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4439))
2022

2123
### Fixed
2224

@@ -26,6 +28,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2628
([#4427](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4427))
2729
- `opentelemetry-instrumentation-flask`: Clean up environ keys in `_teardown_request` to prevent duplicate execution
2830
([#4341](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4341))
31+
- `opentelemetry-instrumentation-celery`: Fix memory leak in `task_id_to_start_time` dict never being cleaned up
32+
([#4439](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4439))
2933

3034
### Breaking changes
3135

docs/nitpick-exceptions.ini

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ py-class=
4848
fastapi.applications.FastAPI
4949
starlette.applications.Starlette
5050
_contextvars.Token
51+
celery.worker.request.Request
5152

5253
any=
5354
; API

instrumentation/opentelemetry-instrumentation-celery/pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ instruments = [
3737

3838
[project.entry-points.opentelemetry_instrumentor]
3939
celery = "opentelemetry.instrumentation.celery:CeleryInstrumentor"
40+
celery_worker = "opentelemetry.instrumentation.celery:CeleryWorkerInstrumentor"
4041

4142
[project.urls]
4243
Homepage = "https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-celery"

0 commit comments

Comments
 (0)