[BUG] The health status of chat api server cannot be queried when the chat api server is generating responses.

[tjtanaa]

Describe the bug

The health status of chat api server cannot be queried when the chat api server is generating responses.

[tjtanaa]

The issue has been identified:
Since the server is launched using a single worker, this means that when the model is streaming or generating results, the worker will be occupied. Any incoming requests to the server will need to be queued until the one and only worker is free.

Welcome more suggestions by replying to this comment.