docker-worker-1 adn docker-worker_beat-1 healthy check is unhealthy

[tankeqwe1231]

Self Checks

• I have read the Contributing Guide and Language Policy.
• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.13.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

i use the portainer dashboard ,i find the worker service state is unhealthy

, I noticed that the CPU usage of my two worker containers occasionally exceeded 100%，

But when I entered the worker container to check the logs, they showed that everything was normal. I found that there is a health check code in the main version of the work service as follows：
healthcheck:
test: ["CMD-SHELL", "celery -A celery_entrypoint.celery inspect ping"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
Could it be this that caused it?

✔️ Expected Behavior

worker service state is healthy

❌ Actual Behavior

No response

[kurokobo]

@crazywoola @laipz8200 @fatelei @majiayu000
The health check was introduced in #34345 and is currently running the celery inspect ping command every 30 seconds.
As a result, it initializes the Flask app every 30 seconds, which leads to frequent CPU spike.

In my local environment, each check takes about 7 seconds, which is quite a heavy process.

dify@20b4144e993b:/app/api$ start=$(date +%s%3N); celery -A celery_entrypoint.celery inspect ping; echo "Elapsed: $(( $(date +%s%3N) - start ))ms"
gRPC patched with gevent.
psycopg2 patched with gevent.
Warning! You didn't set docs_url, redoc_url or openapi_url.
API Documentation will be skipped.
->  celery@8046cd2bd0ba: OK
        pong

1 node online.
Elapsed: 7567ms     👈👈👈

Honestly, I feel like current health check is pretty expensive—what do you all think?
I do think a health check is necessary, but I wonder if there’s a lighter alternative…

Maybe we could increase the interval, or at the very least, I feel like a 10-second timeout is barely enough, if not too short.
Or maybe it's a good idea to leave the worker's health check as an optional feature and keep it commented out or disabled by default.

It's likely that the environment of the Issue author, @tankeqwe1231, is also timing out because the check takes more than 10 seconds due to a lack of resources or high load.

[fatelei]

Problem Chain

1. Docker health check runs celery -A celery_entrypoint.celery inspect ping every 30s
2. This starts a new Python process (not reusing the running worker)
3. New process imports celery_entrypoint → from app import app, celery → create_app()
4. create_app() initializes all 27 extensions (DB engine, Redis, storage, blueprints, Celery config, etc.)
5. Only then does it send the actual inspect ping to the running workers via the broker
6. Process exits — all that work thrown away

[fatelei]

possiblely fix

# api/celery_healthcheck_app.py
from celery import Celery
from configs import dify_config

celery = Celery(broker=dify_config.CELERY_BROKER_URL)

Then update the Docker health check:

healthcheck:
    test: ["CMD-SHELL", "celery -A celery_healthcheck_app.celery inspect ping"]

[kurokobo]

@fatelei
Thanks, I'll work on this 😃

[kurokobo]

@fatelei
Opened PR for this; #34572