Skip to content

test: add more chat completions cases#97

Merged
bzp2010 merged 2 commits intomainfrom
bzp/test-chat-completions-more-cases
May 9, 2026
Merged

test: add more chat completions cases#97
bzp2010 merged 2 commits intomainfrom
bzp/test-chat-completions-more-cases

Conversation

@bzp2010
Copy link
Copy Markdown
Collaborator

@bzp2010 bzp2010 commented May 9, 2026

Summary by CodeRabbit

  • Tests
    • Expanded chat-completions coverage for streaming parity, tool-calling (including parallel calls), usage-only chunks, trailing content after finish, multimodal content, and new response-format behaviors
  • Tests / Utilities
    • Added richer SSE-aware stream assertions and summarization helpers plus a generator for trailing-content stream events to validate advanced streaming behavior

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 9, 2026

Review Change Stack
No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3555084f-ad7d-4741-9080-659038d69388

📥 Commits

Reviewing files that changed from the base of the PR and between 3327e8a and 44e87b2.

📒 Files selected for processing (1)
  • tests/package.json
✅ Files skipped from review due to trivial changes (1)
  • tests/package.json

📝 Walkthrough

Walkthrough

This PR adds structured SSE frames to the mock upstream (per-frame event, delay, and disconnect), refactors stream assertions into a ChatCompletionStreamSummary model, and expands unit and end-to-end tests for streaming parity, trailing content, usage-only chunks, tool-call parsing, and OpenAI feature forwarding.

Changes

SSE Streaming Infrastructure and Test Coverage Expansion

Layer / File(s) Summary
SSE Frame Types & Protocol
tests/fixtures/mock-upstream.ts
Introduces OpenAiMockSseFrame interface with data, optional event, delayMs, and disconnectAfterWrite; updates OpenAiMockStreamEvent to accept structured frames or raw stream data.
SSE Rendering & Streaming Loop
tests/fixtures/mock-upstream.ts
Adds SSE renderer to output event: headers (when present) and data: payloads; refactors streaming loop to normalize events into frames, support per-frame socket destruction, and apply per-frame delays.
Trailing Content Builder
tests/fixtures/mock-upstream.ts
Adds buildOpenAiTrailingContentAfterFinishReasonStreamEvents helper generating a sequence with finish_reason stop, trailing delta, and [DONE] marker.
Stream Assertion Framework Refactor
tests/utils/stream-assert.ts
Introduces ChatCompletionStreamSummary model with aggregated text, usage chunks, finish reasons, and tool calls; adds chatCompletionEvents parser and summarizeChatCompletionStream function; introduces expectStreamHasUsageOnlyChunk and expectStreamRetainsTrailingContentAfterFinishReason; refactors existing assertions to use summary data.
Re-exports & Test Imports
tests/utils/mock-upstream.ts, tests/proxy/chat-completions.test.ts, tests/utils/stream-assert.test.ts
Expands re-exports to include buildOpenAiTrailingContentAfterFinishReasonStreamEvents and OpenAiMockSseFrame; updates test imports to include new stream assertion helpers.
Stream Assertion Unit Tests
tests/utils/stream-assert.test.ts, tests/utils/mock-upstream.test.ts
Adds test for SSE body conversion; validates trailing content detection and usage-only chunk filtering; tests tool-call JSON argument parsing and aggregation; verifies scripted SSE streaming behavior with event/data markers.
Chat Completions End-to-End Coverage
tests/proxy/chat-completions.test.ts
Adds 16 new tests covering streaming/non-stream parity, trailing content retention, usage-only streams, tool-call equivalence, response_format (json_schema/json_object), multimodal images, reasoning tokens, tool-choice variants (required/function form), and parallel tool calls.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • api7/aisix#31: Modifies mock upstream early-disconnect option; this PR refactors SSE events into structured frames with per-frame socket control, complementing the earlier disconnect model.
  • api7/aisix#30: Extends mock upstream and stream-assert utilities for tool-call and SSE handling; this PR builds on that foundation with frame-based protocol and comprehensive assertion refactors.
  • api7/aisix#29: Modifies mock upstream types and helpers; this PR introduces structured SSE frames and new trailing-content builder as a successor enhancement to the mock protocol.
🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'test: add more chat completions cases' accurately summarizes the main changes: expanded test coverage for chat completions with new test cases, helper functions, and refactored assertion utilities.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
E2e Test Quality Review ✅ Passed Excellent E2E tests: 33 cases with real service startup, comprehensive scenarios (auth, validation, streaming, tools), proper isolation, clear structured assertions.
Security Check ✅ Passed No sensitive data exposure, credential leakage, authorization bypass, TLS errors, or security vulnerabilities detected. All API keys are test-only and properly scoped.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch bzp/test-chat-completions-more-cases

Comment @coderabbitai help to get the list of available commands and usage tips.

@bzp2010 bzp2010 merged commit f149c5b into main May 9, 2026
3 checks passed
@bzp2010 bzp2010 deleted the bzp/test-chat-completions-more-cases branch May 9, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant