feat(ai-proxy): abort upstream read on client disconnect during streaming

[nic-6443]

Summary

When a downstream client disconnects mid-stream (browser tab closed, Ctrl+C, request cancelled), the proxy continues reading all remaining chunks from the LLM and performing SSE parsing, token counting, and protocol conversion — burning CPU and LLM API quota for no benefit.

Root Cause

lua_response_filter used ngx.flush() (async, no wait), which never surfaces client disconnection errors. There was no mechanism to detect a dead downstream in the streaming loop.

Fix

Add an optional wait parameter to lua_response_filter. When wait=true, it uses ngx.flush(true) (synchronous flush) which returns an error if the client connection is gone. The streaming path in parse_streaming_response now passes wait=true and on flush failure immediately closes the upstream connection and exits the read loop.

The wait parameter defaults to false so all existing callers are unaffected.

Changes

• apisix/plugin.lua: add optional wait param to lua_response_filter; return (ok, err) for disconnect detection
• apisix/plugins/ai-providers/base.lua: use wait=true in parse_streaming_response output loop; on flush failure close upstream and return
• t/plugin/ai-proxy-client-disconnect.t: integration test verifying upstream is aborted after client disconnect

Test

The new test sets up a slow SSE mock (30ms/chunk, up to 2000 chunks) and a client that reads 3 chunks then closes. It verifies via shared dict that the upstream served well under 50 chunks total (stopped shortly after the disconnect).

[Copilot]

Pull request overview

This PR improves APISIX’s AI proxy streaming behavior by detecting downstream client disconnects and promptly aborting the upstream streaming read, avoiding wasted CPU and LLM quota.

Changes:

• Extend apisix.plugin.lua_response_filter with an optional wait parameter to allow synchronous flushing and surface disconnect errors via (ok, err) returns.
• Update the AI streaming loop to flush synchronously, detect disconnects, and close the upstream HTTP client immediately.
• Add an integration test that simulates a client disconnect and asserts upstream streaming stops early.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
apisix/plugin.lua	Adds wait option and return values to enable disconnect detection during streaming flush.
apisix/plugins/ai-providers/base.lua	Uses wait=true during streaming output and closes upstream on flush failure.
t/plugin/ai-proxy-client-disconnect.t	New integration test validating upstream abort behavior after downstream disconnect.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[nic-6443]

Also fixed a pre-existing test failure in ai-proxy-anthropic.t TEST 6: the regex /"input_tokens":10.*"output_tokens":5/ was assuming a specific JSON field order, but Lua's cjson serializes hash tables without guaranteed ordering. Updated to a lookahead regex that matches both fields regardless of order.

[nic-6443]

Fixed the linux_apisix_current_luarocks stream DNS test failure. The root cause was a race condition: the curl http://127.0.0.1:9100 was firing immediately after the admin PUT, before the stream route had propagated from etcd to the stream workers. The APISIX log showed matched route: null followed by the etcd sync insert data by key: 1 happening afterwards. Added a 1-second sleep between the route creation and the test request to let the config sync complete.