From e3c7cd6f19f8fdc7225bfadb1dd4802e1fbd9cc5 Mon Sep 17 00:00:00 2001 From: Zidong Lu Date: Fri, 8 May 2026 11:31:58 +0800 Subject: [PATCH 1/2] Add test plans for context size donut and rate limit warning banner (0.17.0 endgame) --- .../context-size-donut/context-size-donut.md | 221 ++++++++++++++++++ .../rate-limit-warning-banner.md | 211 +++++++++++++++++ 2 files changed, 432 insertions(+) create mode 100644 com.microsoft.copilot.eclipse.swtbot.test/test-plans/context-size-donut/context-size-donut.md create mode 100644 com.microsoft.copilot.eclipse.swtbot.test/test-plans/rate-limit-warning-banner/rate-limit-warning-banner.md diff --git a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/context-size-donut/context-size-donut.md b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/context-size-donut/context-size-donut.md new file mode 100644 index 00000000..30677a6c --- /dev/null +++ b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/context-size-donut/context-size-donut.md @@ -0,0 +1,221 @@ +# Context Size Donut and Popup + +## Overview +Tests the context size donut chart widget and its hover popup in the GitHub Copilot +for Eclipse chat view. A `ContextSizeDonut` widget is rendered in the chat view's +`ActionBar` control bar. It displays a ring that fills proportionally based on the +token utilization percentage received via the LSP `ContextSizeInfo` payload. At +≥90 % utilization the ring switches to a warning color. Hovering over the donut +opens a `ContextWindowPopup` that shows a breakdown of token usage by category. + +Entry points: +- **Copilot Chat view** → bottom control bar (the donut appears after the first + response is received that includes `ContextSizeInfo`). +- Hover the donut to open the **Context Window** popup. + +--- + +## Prerequisites + +- Eclipse IDE with the GitHub Copilot for Eclipse plugin installed and activated. +- A valid GitHub Copilot subscription is active (authentication completed). +- The Copilot Chat view is open and visible in the workbench. + +--- + +## 1. Donut Widget Visibility + +### TC-001: Donut appears after first response + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- The Copilot Chat view is open. +- No previous conversation is loaded (fresh session or new conversation). + +#### Steps +1. Open the Copilot Chat view. +2. Confirm the donut widget is **not** visible in the bottom control bar (no + `ContextSizeInfo` has been received yet). +3. Send a short chat message (e.g. "Hello") and wait for a response. +4. Inspect the bottom control bar of the chat view. + +#### Expected Result +- Before the first response the donut is not visible (or shows an empty/zero state). +- After the first response arrives the donut widget appears in the control bar. +- No error dialog or exception is logged. + +#### 📸 Key Screenshots +- [ ] **Before response** — chat view control bar with no donut. +- [ ] **After response** — chat view control bar showing the donut widget. + +--- + +## 2. Donut Fill Proportion + +### TC-002: Donut ring reflects token utilization + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- A conversation exists with at least one response (donut is visible). + +#### Steps +1. Send a short message; note the approximate fill level of the donut ring. +2. Send a message that attaches a large file or pastes a long code block to + increase context size; wait for the response. +3. Compare the donut fill level after the larger-context response. + +#### Expected Result +- The donut ring fill increases visibly when more tokens are consumed. +- The fill level is proportional to the reported utilization percentage (0–100 %). + +#### 📸 Key Screenshots +- [ ] **Low utilization** — donut with small fill. +- [ ] **Higher utilization** — donut with larger fill. + +--- + +## 3. Context Window Popup + +### TC-003: Hovering the donut opens the Context Window popup + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- The donut widget is visible (at least one response received). + +#### Steps +1. Hover the mouse cursor over the donut widget in the control bar. +2. Wait for the popup to appear. +3. Inspect the popup header. +4. Inspect the popup body for: total token count, utilization percentage, progress + bar, and per-category rows. +5. Move the mouse away from the donut. + +#### Expected Result +- The popup opens with the header **"Context Window"**. +- Total usage is displayed in the format `X / Y tokens`. +- A utilization percentage is shown. +- A progress bar reflects the utilization level. +- Per-category rows are shown: **System Instructions**, **Tool Definitions**, + **Messages**, **Attached Files**, **Tool Results**. +- Moving the mouse away closes the popup. + +#### 📸 Key Screenshots +- [ ] **Popup open** — the Context Window popup showing all fields. + +--- + +### TC-004: Popup shows non-zero Attached Files when a file is attached + +**Type:** `Happy Path` +**Priority:** `P2` + +#### Preconditions +- The chat view is open. + +#### Steps +1. Send a message with at least one file attached as context. +2. Wait for the response. +3. Hover over the donut to open the popup. +4. Locate the **Attached Files** row. + +#### Expected Result +- The **Attached Files** row displays a non-zero token count. + +--- + +### TC-005: Popup shows non-zero Tool Definitions and Tool Results in agent mode + +**Type:** `Happy Path` +**Priority:** `P2` + +#### Preconditions +- Agent mode is enabled in the chat view. + +#### Steps +1. Send a message that triggers at least one tool call in agent mode. +2. Wait for the response. +3. Hover over the donut to open the popup. +4. Locate the **Tool Definitions** and **Tool Results** rows. + +#### Expected Result +- **Tool Definitions** shows a non-zero token count. +- **Tool Results** shows a non-zero token count. + +--- + +## 4. Warning Color at High Utilization + +### TC-006: Donut ring changes to warning color at ≥90 % utilization + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- A way to produce or simulate ≥90 % token utilization (e.g. very large context, + or by injecting a mock `ContextSizeInfo` value via a debug breakpoint). + +#### Steps +1. Bring the token utilization to ≥90 % (send large context or inject mock data). +2. Observe the donut ring color. + +#### Expected Result +- Below 90 % the ring is rendered in the normal (non-warning) color. +- At or above 90 % the ring switches to a warning/red color. + +#### 📸 Key Screenshots +- [ ] **Warning state** — donut showing warning color at high utilization. + +--- + +## 5. Edge Cases + +### TC-007: Donut resets when a new conversation is started + +**Type:** `Edge Case` +**Priority:** `P2` + +#### Steps +1. Start a conversation and let the donut show some utilization. +2. Start a new conversation (clear or new session). +3. Observe the donut state before any new response. + +#### Expected Result +- The donut resets (hidden or zero state) for the new conversation until a + response with `ContextSizeInfo` is received. + +--- + +### TC-008: Chat view resized to narrow width + +**Type:** `Edge Case` +**Priority:** `P2` + +#### Steps +1. Make the chat view narrow by dragging its edge. +2. Send a message and wait for the donut to appear. +3. Hover over the donut to open the popup. + +#### Expected Result +- The donut and control bar do not overflow or clip incorrectly. +- The popup appears positioned near the donut without being cut off. + +--- + +### TC-009: Rapid message sending does not cause stale data + +**Type:** `Edge Case` +**Priority:** `P2` + +#### Steps +1. Send several messages in quick succession. +2. After all responses arrive, hover over the donut. + +#### Expected Result +- The donut and popup reflect the most recent `ContextSizeInfo` without + stale data, race conditions, or visual glitches. diff --git a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/rate-limit-warning-banner/rate-limit-warning-banner.md b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/rate-limit-warning-banner/rate-limit-warning-banner.md new file mode 100644 index 00000000..c9f58fb5 --- /dev/null +++ b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/rate-limit-warning-banner/rate-limit-warning-banner.md @@ -0,0 +1,211 @@ +# Rate Limit Warning Banner + +## Overview +Tests the rate limit warning banner in the GitHub Copilot for Eclipse chat view. +When the Copilot language server emits a `$/copilot/rateLimitWarning` LSP +notification, a `StaticBanner` widget is displayed above the action bar input +area. The banner shows the server-provided human-readable message, a +"Get more info" hyperlink to `https://aka.ms/github-copilot-rate-limit-error`, +and a "Dismiss" button. The banner is wired via an OSGi event topic +(`TOPIC_RATE_LIMIT_WARNING`) from `CopilotLanguageClient` → `ChatView` → +`ActionBar`. Navigating chat history hides/shows the banner appropriately. + +Entry points: +- Triggered automatically by a `$/copilot/rateLimitWarning` LSP notification. +- Dismissed manually via the "×" button on the banner. + +--- + +## Prerequisites + +- Eclipse IDE with the GitHub Copilot for Eclipse plugin installed and activated. +- A GitHub account signed in with a Copilot subscription that has measurable + usage quota (so that rate limit notifications can be triggered or simulated). +- A way to trigger or mock a `$/copilot/rateLimitWarning` LSP notification — + options include: + - Exhausting the quota for the account. + - Injecting the notification via a debug breakpoint in `CopilotLanguageClient`. + - Using a test harness / mock language server. +- The Copilot Chat view is open and visible in the workbench. + +--- + +## 1. Banner Appearance + +### TC-001: Banner appears in non-handoff mode on rate limit warning + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- The Copilot Chat view is open in a **non-handoff** (standard) chat mode. + +#### Steps +1. Trigger a `$/copilot/rateLimitWarning` notification (type: "weekly" or + "session") from the language server while in standard chat mode. +2. Observe the area above the chat input field in the Action Bar. + +#### Expected Result +- A warning banner appears above the action bar input area. +- The banner text matches the `message` field from the LSP notification. +- No error dialog or exception is logged. + +#### 📸 Key Screenshots +- [ ] **Banner visible** — chat view showing the rate limit warning banner. + +--- + +### TC-002: Banner appears in handoff mode on rate limit warning + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- The Copilot Chat view is open in **agent/handoff** mode. + +#### Steps +1. Trigger a `$/copilot/rateLimitWarning` notification while in agent/handoff + mode. +2. Observe the area above the chat input field in the Action Bar. + +#### Expected Result +- The warning banner appears correctly above the action bar input area, identical + to non-handoff mode. + +--- + +## 2. Banner Content + +### TC-003: Banner contains "Get more info" link and Dismiss button + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- The rate limit warning banner is currently visible in the chat view. + +#### Steps +1. Locate the **"Get more info"** link in the banner. +2. Click the link. +3. Locate the **"×"** (Dismiss) button in the banner. + +#### Expected Result +- The "Get more info" link opens `https://aka.ms/github-copilot-rate-limit-error` + in the system default browser (or Eclipse's internal browser). +- The "×" button is visible and interactive. + +#### 📸 Key Screenshots +- [ ] **Banner with link and dismiss button** — close-up of the banner widget. + +--- + +### TC-004: Dismiss button closes the banner + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- The rate limit warning banner is currently visible. + +#### Steps +1. Click the **"×"** (Dismiss) button on the banner. +2. Observe the chat view layout. + +#### Expected Result +- The banner is removed from the chat view immediately. +- The chat input area expands to fill the space previously occupied by the banner. +- No exceptions or layout glitches occur. + +#### 📸 Key Screenshots +- [ ] **After dismiss** — chat view with banner removed. + +--- + +## 3. History Navigation + +### TC-005: Banner hides and shows when navigating chat history + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions +- A rate limit warning banner is visible for the current chat conversation. +- At least one other chat history entry exists. + +#### Steps +1. Note the current conversation — the banner is visible. +2. Navigate to a **different** chat history entry. +3. Observe the banner state. +4. Navigate **back** to the original conversation. +5. Observe the banner state. + +#### Expected Result +- After switching away, the banner is **hidden** for the other history entry. +- After switching back, the banner **reappears** for the original conversation. +- No errors or layout issues occur during navigation. + +--- + +## 4. Edge Cases + +### TC-006: Multiple rapid notifications — only one banner shown + +**Type:** `Edge Case` +**Priority:** `P2` + +#### Steps +1. Trigger two or more `$/copilot/rateLimitWarning` notifications in quick + succession. +2. Observe the chat view. + +#### Expected Result +- Only one banner is displayed at a time (previous banner is replaced/disposed + before the new one is shown). + +--- + +### TC-007: New warning after dismiss creates a fresh banner + +**Type:** `Edge Case` +**Priority:** `P2` + +#### Steps +1. Dismiss the existing rate limit banner. +2. Trigger another `$/copilot/rateLimitWarning` notification. +3. Observe the chat view. + +#### Expected Result +- A new banner appears with the content of the new notification. +- No stale state from the dismissed banner is visible. + +--- + +### TC-008: Rate limit warning received while chat view is not focused + +**Type:** `Edge Case` +**Priority:** `P2` + +#### Steps +1. Switch focus to another Eclipse view or editor. +2. Trigger a `$/copilot/rateLimitWarning` notification. +3. Switch back to the Copilot Chat view. + +#### Expected Result +- The banner is displayed when the chat view regains focus/visibility. + +--- + +### TC-009: Very long message string wraps gracefully + +**Type:** `Edge Case` +**Priority:** `P3` + +#### Steps +1. Trigger a `$/copilot/rateLimitWarning` notification whose `message` field + contains an unusually long string (e.g. 500+ characters). +2. Observe the banner layout. + +#### Expected Result +- The banner text wraps within the banner boundaries. +- The chat view layout is not broken (no overflow, clipping, or overlapping + widgets). From 0c6321cc50ada4f041d7228b2a7b5f128e60b436 Mon Sep 17 00:00:00 2001 From: Zidong Lu Date: Fri, 8 May 2026 14:00:43 +0800 Subject: [PATCH 2/2] test-plans: trim to tested TCs and renumber sequentially context-size-donut: keep TC-001/002/003 (was 001/003/007) rate-limit-warning-banner: keep TC-001/002/003 (was 001/003/004) Removed untested cases from both plans. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../context-size-donut/context-size-donut.md | 128 +----------------- .../rate-limit-warning-banner.md | 114 +--------------- 2 files changed, 6 insertions(+), 236 deletions(-) diff --git a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/context-size-donut/context-size-donut.md b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/context-size-donut/context-size-donut.md index 30677a6c..e17963b3 100644 --- a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/context-size-donut/context-size-donut.md +++ b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/context-size-donut/context-size-donut.md @@ -52,35 +52,9 @@ Entry points: --- -## 2. Donut Fill Proportion +## 2. Context Window Popup -### TC-002: Donut ring reflects token utilization - -**Type:** `Happy Path` -**Priority:** `P1` - -#### Preconditions -- A conversation exists with at least one response (donut is visible). - -#### Steps -1. Send a short message; note the approximate fill level of the donut ring. -2. Send a message that attaches a large file or pastes a long code block to - increase context size; wait for the response. -3. Compare the donut fill level after the larger-context response. - -#### Expected Result -- The donut ring fill increases visibly when more tokens are consumed. -- The fill level is proportional to the reported utilization percentage (0–100 %). - -#### 📸 Key Screenshots -- [ ] **Low utilization** — donut with small fill. -- [ ] **Higher utilization** — donut with larger fill. - ---- - -## 3. Context Window Popup - -### TC-003: Hovering the donut opens the Context Window popup +### TC-002: Hovering the donut opens the Context Window popup **Type:** `Happy Path` **Priority:** `P1` @@ -110,72 +84,9 @@ Entry points: --- -### TC-004: Popup shows non-zero Attached Files when a file is attached - -**Type:** `Happy Path` -**Priority:** `P2` - -#### Preconditions -- The chat view is open. - -#### Steps -1. Send a message with at least one file attached as context. -2. Wait for the response. -3. Hover over the donut to open the popup. -4. Locate the **Attached Files** row. - -#### Expected Result -- The **Attached Files** row displays a non-zero token count. - ---- - -### TC-005: Popup shows non-zero Tool Definitions and Tool Results in agent mode - -**Type:** `Happy Path` -**Priority:** `P2` - -#### Preconditions -- Agent mode is enabled in the chat view. - -#### Steps -1. Send a message that triggers at least one tool call in agent mode. -2. Wait for the response. -3. Hover over the donut to open the popup. -4. Locate the **Tool Definitions** and **Tool Results** rows. - -#### Expected Result -- **Tool Definitions** shows a non-zero token count. -- **Tool Results** shows a non-zero token count. - ---- - -## 4. Warning Color at High Utilization - -### TC-006: Donut ring changes to warning color at ≥90 % utilization - -**Type:** `Happy Path` -**Priority:** `P1` - -#### Preconditions -- A way to produce or simulate ≥90 % token utilization (e.g. very large context, - or by injecting a mock `ContextSizeInfo` value via a debug breakpoint). +## 3. Edge Cases -#### Steps -1. Bring the token utilization to ≥90 % (send large context or inject mock data). -2. Observe the donut ring color. - -#### Expected Result -- Below 90 % the ring is rendered in the normal (non-warning) color. -- At or above 90 % the ring switches to a warning/red color. - -#### 📸 Key Screenshots -- [ ] **Warning state** — donut showing warning color at high utilization. - ---- - -## 5. Edge Cases - -### TC-007: Donut resets when a new conversation is started +### TC-003: Donut resets when a new conversation is started **Type:** `Edge Case` **Priority:** `P2` @@ -188,34 +99,3 @@ Entry points: #### Expected Result - The donut resets (hidden or zero state) for the new conversation until a response with `ContextSizeInfo` is received. - ---- - -### TC-008: Chat view resized to narrow width - -**Type:** `Edge Case` -**Priority:** `P2` - -#### Steps -1. Make the chat view narrow by dragging its edge. -2. Send a message and wait for the donut to appear. -3. Hover over the donut to open the popup. - -#### Expected Result -- The donut and control bar do not overflow or clip incorrectly. -- The popup appears positioned near the donut without being cut off. - ---- - -### TC-009: Rapid message sending does not cause stale data - -**Type:** `Edge Case` -**Priority:** `P2` - -#### Steps -1. Send several messages in quick succession. -2. After all responses arrive, hover over the donut. - -#### Expected Result -- The donut and popup reflect the most recent `ContextSizeInfo` without - stale data, race conditions, or visual glitches. diff --git a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/rate-limit-warning-banner/rate-limit-warning-banner.md b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/rate-limit-warning-banner/rate-limit-warning-banner.md index c9f58fb5..ffb7cdce 100644 --- a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/rate-limit-warning-banner/rate-limit-warning-banner.md +++ b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/rate-limit-warning-banner/rate-limit-warning-banner.md @@ -55,28 +55,9 @@ Entry points: --- -### TC-002: Banner appears in handoff mode on rate limit warning - -**Type:** `Happy Path` -**Priority:** `P1` - -#### Preconditions -- The Copilot Chat view is open in **agent/handoff** mode. - -#### Steps -1. Trigger a `$/copilot/rateLimitWarning` notification while in agent/handoff - mode. -2. Observe the area above the chat input field in the Action Bar. - -#### Expected Result -- The warning banner appears correctly above the action bar input area, identical - to non-handoff mode. - ---- - ## 2. Banner Content -### TC-003: Banner contains "Get more info" link and Dismiss button +### TC-002: Banner contains "Get more info" link and Dismiss button **Type:** `Happy Path` **Priority:** `P1` @@ -99,7 +80,7 @@ Entry points: --- -### TC-004: Dismiss button closes the banner +### TC-003: Dismiss button closes the banner **Type:** `Happy Path` **Priority:** `P1` @@ -118,94 +99,3 @@ Entry points: #### 📸 Key Screenshots - [ ] **After dismiss** — chat view with banner removed. - ---- - -## 3. History Navigation - -### TC-005: Banner hides and shows when navigating chat history - -**Type:** `Happy Path` -**Priority:** `P1` - -#### Preconditions -- A rate limit warning banner is visible for the current chat conversation. -- At least one other chat history entry exists. - -#### Steps -1. Note the current conversation — the banner is visible. -2. Navigate to a **different** chat history entry. -3. Observe the banner state. -4. Navigate **back** to the original conversation. -5. Observe the banner state. - -#### Expected Result -- After switching away, the banner is **hidden** for the other history entry. -- After switching back, the banner **reappears** for the original conversation. -- No errors or layout issues occur during navigation. - ---- - -## 4. Edge Cases - -### TC-006: Multiple rapid notifications — only one banner shown - -**Type:** `Edge Case` -**Priority:** `P2` - -#### Steps -1. Trigger two or more `$/copilot/rateLimitWarning` notifications in quick - succession. -2. Observe the chat view. - -#### Expected Result -- Only one banner is displayed at a time (previous banner is replaced/disposed - before the new one is shown). - ---- - -### TC-007: New warning after dismiss creates a fresh banner - -**Type:** `Edge Case` -**Priority:** `P2` - -#### Steps -1. Dismiss the existing rate limit banner. -2. Trigger another `$/copilot/rateLimitWarning` notification. -3. Observe the chat view. - -#### Expected Result -- A new banner appears with the content of the new notification. -- No stale state from the dismissed banner is visible. - ---- - -### TC-008: Rate limit warning received while chat view is not focused - -**Type:** `Edge Case` -**Priority:** `P2` - -#### Steps -1. Switch focus to another Eclipse view or editor. -2. Trigger a `$/copilot/rateLimitWarning` notification. -3. Switch back to the Copilot Chat view. - -#### Expected Result -- The banner is displayed when the chat view regains focus/visibility. - ---- - -### TC-009: Very long message string wraps gracefully - -**Type:** `Edge Case` -**Priority:** `P3` - -#### Steps -1. Trigger a `$/copilot/rateLimitWarning` notification whose `message` field - contains an unusually long string (e.g. 500+ characters). -2. Observe the banner layout. - -#### Expected Result -- The banner text wraps within the banner boundaries. -- The chat view layout is not broken (no overflow, clipping, or overlapping - widgets).