Skip to content

Commit b3f58b7

Browse files
xinyi-gongxinyi-gong
authored
docs: add test plans for model-picker, add context and nes (#151)
1 parent 1f2b9d1 commit b3f58b7

3 files changed

Lines changed: 608 additions & 0 deletions

File tree

‎com.microsoft.copilot.eclipse.swtbot.test/test-plans/add-context/add-context.md‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# Chat: Add Context (Attach Files)
2+
3+
## Overview
4+
5+
Tests the **Add Context** button in the Chat view, which lets users attach
6+
workspace files to a chat prompt. Attached files provide additional context to
7+
Copilot so it can give more relevant answers. The button is an icon-only flat
8+
button (tooltip: **Add Context...**) in the action bar's file reference area.
9+
10+
After attaching, files appear as **chips** below the input area showing the
11+
file name and a close (×) button. Users can click a chip to open the file, or
12+
click × to remove it.
13+
14+
Entry points exercised:
15+
- **Ctrl+Alt+I** (or status bar → **Open Chat**), then click the Add Context
16+
button in the action bar.
17+
18+
---
19+
20+
## Prerequisites
21+
22+
- Eclipse IDE with the GitHub Copilot for Eclipse plugin installed and
23+
activated.
24+
- A signed-in Copilot account.
25+
- A Java project is open in the workspace with at least two files.
26+
27+
---
28+
29+
## 1. Attach files, verify chips, remove, and send with context
30+
31+
### TC-001: Attach files via the Add Context button, verify file chips, remove a file, and send a prompt with attached context
32+
33+
**Type:** `Happy Path`
34+
**Priority:** `P0`
35+
36+
#### Preconditions
37+
38+
- The Chat view is open.
39+
- The workspace contains at least two Java files.
40+
41+
#### Steps
42+
43+
1. Locate the **Add Context** button in the action bar (icon-only button with
44+
tooltip **Add Context...**).
45+
2. Verify the button appears as a flat icon without a visible rectangular
46+
border.
47+
3. Click the **Add Context** button.
48+
4. Verify a file picker dialog opens (title: **Search attachments**).
49+
5. Select two files from the workspace and click **OK**.
50+
6. Verify the dialog closes and two **file chips** appear below the chat input
51+
area, each showing the file name and a close (×) button.
52+
7. Click the × button on one of the file chips.
53+
8. Verify that chip is removed and only one file chip remains.
54+
9. Click on the remaining file chip's file name.
55+
10. Verify the corresponding file opens (or is revealed) in the editor.
56+
11. Type a prompt (e.g. `explain this file`) in the chat input and click
57+
**Send**.
58+
12. Wait for the Copilot turn to complete.
59+
13. Verify the response references or uses content from the attached file.
60+
61+
#### Expected Result
62+
63+
- The Add Context button opens the file picker dialog.
64+
- Selected files appear as removable chips below the input area.
65+
- Clicking × removes the chip.
66+
- Clicking the file name opens the file in the editor.
67+
- Sending a prompt with an attached file includes that file as context in the
68+
conversation.
69+
70+
#### 📸 Key Screenshots
71+
72+
- [ ] **Add Context button** — flat icon button in the action bar, no border.
73+
- [ ] **File picker dialog** — dialog showing workspace files to select.
74+
- [ ] **File chips** — two file chips visible below the chat input area.
75+
- [ ] **After removing one chip** — only one chip remaining.
76+
- [ ] **Response with context** — Copilot turn that references the attached
77+
file content.
78+
79+
---
80+
81+
## 2. Currently active file is shown automatically
82+
83+
### TC-002: The currently open file appears as a reference in the action bar
84+
85+
**Type:** `Happy Path`
86+
**Priority:** `P0`
87+
88+
#### Preconditions
89+
90+
- The Chat view is open.
91+
- A Java file is open and active in the editor.
92+
93+
#### Steps
94+
95+
1. Open a Java file in the editor (click on it in the Package Explorer or
96+
switch to an already-open tab).
97+
2. Switch focus to the Chat view.
98+
3. Observe the file reference area in the action bar.
99+
100+
#### Expected Result
101+
102+
- The currently active editor file is automatically shown as a reference in
103+
the action bar (displaying the file name).
104+
- Switching to a different file in the editor updates the displayed reference.
105+
106+
#### 📸 Key Screenshots
107+
108+
- [ ] **Current file reference** — the action bar showing the active file name
109+
as a context reference.
110+
111+
---
112+
113+
## Notes on failure modes
114+
115+
- File picker dialog does not open → the Add Context button's selection
116+
listener may not be wired; check the Eclipse error log.
117+
- File chips do not appear after selection → the ReferencedFileService may not
118+
have received the files; verify the file selection result is non-empty.
119+
- Clicking × does not remove the chip → the close button's mouse listener
120+
may not be attached; check the ReferencedFile widget.

‎com.microsoft.copilot.eclipse.swtbot.test/test-plans/model-picker/model-picker.md‎

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Chat: Model Picker
2+
3+
## Overview
4+
5+
Tests the **model picker** dropdown in the Chat view action bar. The model
6+
picker lets users choose which AI model powers their chat conversations. It
7+
displays available models grouped by tier (Standard, Premium, Custom), shows
8+
per-model details on hover, and includes a **Manage Models...** shortcut for
9+
users with BYOK enabled.
10+
11+
Entry points exercised:
12+
- **Ctrl+Alt+I** (or status bar → **Open Chat**), then interact with the model
13+
picker button in the action bar.
14+
15+
---
16+
17+
## Prerequisites
18+
19+
- Eclipse IDE with the GitHub Copilot for Eclipse plugin installed and
20+
activated.
21+
- A signed-in Copilot account. Without a valid account the model list cannot
22+
be fetched and the picker will show no items.
23+
24+
---
25+
26+
## 1. Model picker loads, opens, selects, and routes a message
27+
28+
### TC-001: Open model picker, browse grouped models, switch model, and verify a response uses it
29+
30+
**Type:** `Happy Path`
31+
**Priority:** `P0`
32+
33+
#### Preconditions
34+
35+
- The Eclipse workbench is open.
36+
- The user is signed in to Copilot.
37+
- No previous Chat view is open.
38+
39+
#### Steps
40+
41+
1. Open the **Copilot Chat** view via the keyboard shortcut **Ctrl+Alt+I**
42+
(Windows/Linux) or **Ctrl+Cmd+I** (macOS), or click the Copilot status bar
43+
icon and select **Open Chat**.
44+
2. Wait for the Chat view to fully load (the input area is editable, the model
45+
picker shows a resolved model name instead of "Loading...").
46+
3. Verify the model picker button on the right side of the action bar displays
47+
a model name (e.g. `Claude Sonnet 4.6`) with a dropdown arrow.
48+
4. Hover the mouse over the model picker button and verify a tooltip appears
49+
(e.g. `Pick Model`).
50+
5. Click the model picker button to open the dropdown popup.
51+
6. Verify the dropdown lists models grouped under labelled headers (e.g.
52+
**Standard Models**, **Premium Models**), with each model showing a cost
53+
multiplier on the right (e.g. `0x`, `1x`, `3x`).
54+
7. Verify the currently selected model is marked with a **checkmark** (✓)
55+
in the list.
56+
8. Hover over a model item in the dropdown and verify a **detail card**
57+
appears showing the model's **family** (e.g. `Family: claude-sonnet-4.5`),
58+
**cost** (e.g. `Cost: 1x premium`), and **category tag** (e.g.
59+
`Versatile`).
60+
9. Click a model different from the currently selected one.
61+
10. Verify the dropdown closes and the model picker button label updates to
62+
show the newly selected model name.
63+
11. Type a prompt (e.g. `hello`) in the chat input and click **Send**.
64+
12. Wait for the Copilot turn to complete (the model-info-label footer
65+
appears).
66+
13. Verify the model info label at the bottom of the completed turn matches
67+
the model selected in step 9.
68+
69+
#### Expected Result
70+
71+
- The model picker loads and shows the default model.
72+
- The dropdown opens with grouped models and cost multipliers; the selected
73+
model has a checkmark.
74+
- Hovering a model shows a detail card with family, cost, and category.
75+
- Selecting a different model updates the picker label immediately.
76+
- The response is served by the newly selected model, confirmed by the turn
77+
footer.
78+
79+
#### 📸 Key Screenshots
80+
81+
- [ ] **Model picker loaded** — action bar showing the resolved model name.
82+
- [ ] **Dropdown open** — popup showing grouped model list with the active
83+
model marked with a checkmark.
84+
- [ ] **Model hover card** — detail card showing family, cost, and category
85+
tag for a model item.
86+
- [ ] **Model switched** — picker button showing the new model name.
87+
- [ ] **Response with new model** — completed Copilot turn whose footer shows
88+
the new model name.
89+
90+
---
91+
92+
## 2. Chat mode interaction
93+
94+
### TC-002: Switching chat mode updates the model picker
95+
96+
**Type:** `Happy Path`
97+
**Priority:** `P1`
98+
99+
#### Preconditions
100+
101+
- The Chat view is open.
102+
- Both **Ask** and **Agent** modes are available in the mode picker.
103+
104+
#### Steps
105+
106+
1. Note the model currently shown in the model picker.
107+
2. Switch the chat mode from **Agent** to **Ask** (or vice versa) using the
108+
mode picker on the left side of the action bar.
109+
3. Observe the model picker button — it should still show a valid model name
110+
(which may differ from the previous one if the old model is not available
111+
in the new mode).
112+
4. Open the model picker dropdown and verify the model list has loaded (it may
113+
contain different models than before).
114+
5. Close the dropdown (click outside or press Escape).
115+
116+
#### Expected Result
117+
118+
- The model picker updates automatically when the chat mode changes.
119+
- No crash, blank picker, or stale model name occurs.
120+
121+
#### 📸 Key Screenshots
122+
123+
- [ ] **After mode switch** — action bar showing the updated mode label and the
124+
model picker with a valid model name.
125+
126+
---
127+
128+
## 3. BYOK custom models and Manage Models shortcut
129+
130+
### TC-003: Custom models appear in the picker and "Manage Models..." opens the preference page
131+
132+
**Type:** `Happy Path`
133+
**Priority:** `P1`
134+
135+
#### Preconditions
136+
137+
- The user is signed in with an account that has BYOK enabled.
138+
- At least one custom model has been added and enabled via **Window →
139+
Preferences → GitHub Copilot → Model Management**.
140+
141+
#### Steps
142+
143+
1. Open the model picker dropdown.
144+
2. Verify a **Custom Models** group is present at the bottom of the list,
145+
containing the configured custom model(s).
146+
3. Select a custom model and verify the picker label updates.
147+
4. Open the model picker dropdown again.
148+
5. Click the **Manage Models...** item at the bottom of the dropdown.
149+
6. Verify the **Preferences** dialog opens directly on the **Model Management**
150+
page.
151+
7. Close the Preferences dialog.
152+
153+
#### Expected Result
154+
155+
- Custom models are listed under a dedicated **Custom Models** group.
156+
- Selecting a custom model works the same as selecting a built-in model.
157+
- **Manage Models...** navigates directly to the BYOK preference page.
158+
159+
#### 📸 Key Screenshots
160+
161+
- [ ] **Custom Models group** — dropdown showing the Custom Models section with
162+
the user's configured models.
163+
- [ ] **Manage Models... action** — Preferences dialog opened to the Model
164+
Management page after clicking the shortcut.
165+
166+
---
167+
168+
## Notes on failure modes
169+
170+
- Picker button stays blank / never shows a model name → the model list fetch
171+
failed; check that the account is signed in and network access is available.
172+
- Dropdown is empty → auth is valid but the server returned no models; try
173+
signing out and back in to refresh the token.
174+
- Selected model is not reflected in the completed turn's footer → the model
175+
switch event did not propagate; re-select the model and resend.
176+
- Custom Models group does not appear → BYOK is disabled by org policy, or no
177+
custom model has been added; verify in Model Management preferences.

0 commit comments

Comments
 (0)