Skip to content

Commit 2b0c2d5

Browse files
committed
Add task and worker lifecycle metrics to Celery instrumentation
Introduce Flower-compatible metrics for Celery task and worker events: - flower.events.total: counter for task-sent, task-received, task-started, task-succeeded, task-failed, task-retried, task-revoked - flower.task.runtime.seconds: histogram of task execution time - flower.worker.number.of.currently.executing.tasks: gauge of in-flight tasks per worker - flower.worker.online: gauge tracking worker online/offline status CeleryInstrumentor (child process) handles task tracing and task-level metrics. CeleryWorkerInstrumentor (main process) handles worker lifecycle signals (worker_ready, worker_shutdown) and main-process events (task_received, task_revoked). Prefetch-time metrics (task_prefetch_time_seconds, worker_prefetched_tasks) are intentionally omitted — Celery splits task_received and task_prerun across processes in prefork mode, making it impossible to compute the delta without external event consumption.
1 parent cf0662d commit 2b0c2d5

9 files changed

Lines changed: 1659 additions & 89 deletions

File tree

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1919
([#4335](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4335))
2020
- Expand `AGENTS.md` with instrumentation/GenAI guidance and add PR review instructions.
2121
([#4457](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4457))
22+
- `opentelemetry-instrumentation-celery`: Add task and worker lifecycle metrics matching Celery Flower
23+
([#4439](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4439))
2224

2325
### Fixed
2426

@@ -34,6 +36,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
3436
([#4360](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4360))
3537
- `opentelemetry-instrumentation-aiohttp-server`: Use `canonical` attribute of the `Resource` as a span name
3638
([#3896](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3896))
39+
- `opentelemetry-instrumentation-celery`: Fix memory leak in `task_id_to_start_time` dict never being cleaned up
40+
([#4439](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4439))
3741

3842
### Breaking changes
3943

docs/nitpick-exceptions.ini

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ py-class=
4848
fastapi.applications.FastAPI
4949
starlette.applications.Starlette
5050
_contextvars.Token
51+
celery.worker.request.Request
5152

5253
any=
5354
; API

instrumentation/opentelemetry-instrumentation-celery/pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ instruments = [
3737

3838
[project.entry-points.opentelemetry_instrumentor]
3939
celery = "opentelemetry.instrumentation.celery:CeleryInstrumentor"
40+
celery_worker = "opentelemetry.instrumentation.celery:CeleryWorkerInstrumentor"
4041

4142
[project.urls]
4243
Homepage = "https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-celery"

0 commit comments

Comments
 (0)