Multi-Query-Batch-Inference-Optimization/server.log at main · dakshjain-1616/Multi-Query-Batch-Inference-Optimization · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
llama_new_context_with_model: n_ctx_per_seq (2048) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
2026-01-30 09:15:52,533 - inference-server - INFO - Model loaded successfully.
/root/claude_tests/NEODEMO4/api.py:129: DeprecationWarning:
        on_event is deprecated, use lifespan event handlers instead.

        Read more about it in the
        [FastAPI docs for Lifespan Events](https://fastapi.tiangolo.com/advanced/events/).

  @app.on_event("startup")
INFO:     Started server process [810018]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:39770 - "GET /health HTTP/1.1" 200 OK
INFO:     127.0.0.1:39774 - "POST /generate HTTP/1.1" 200 OK
INFO:     127.0.0.1:36768 - "POST /generate HTTP/1.1" 200 OK
INFO:     127.0.0.1:38080 - "POST /generate HTTP/1.1" 200 OK
2026-01-30 09:17:29,835 - inference-server - INFO - Batch request pre-empted by interactive request.
INFO:     127.0.0.1:46106 - "POST /generate HTTP/1.1" 200 OK
INFO:     127.0.0.1:46118 - "POST /generate HTTP/1.1" 200 OK
WARNING:  Invalid HTTP request received.
INFO:     94.231.206.1:54951 - "GET / HTTP/1.1" 404 Not Found
2026-01-30 09:20:52,587 - inference-server - INFO - Auto-shutdown triggered.
2026-01-30 09:20:52,587 - inference-server - INFO - Auto-shutdown triggered.