Add Test Goal view in webstudio

Author: ferpasri

agent/resources/skills/testar-cli/SKILL.md

@@ -15,11 +15,6 @@ Treat the CLI distribution as an operational environment, not as a source-code w
 - The task requires step-by-step control of a live Windows, WebDriver, or Android session.
 - The task requires using `getState`, `getStateScreenshot`, and `getDerivedActions` as execution evidence.
 
-## Runtime layout
-
-- Distribution root contains the public launcher.
-- This skill is copied into `.agents/skills/testar-cli/` inside the CLI distribution.
-
 ## Public launcher paths
 
 - Windows: `testar-cli.bat`

docs/architecture_distribution_workspace.md

@@ -173,6 +173,26 @@ If reporting is enabled, the final CLI verdict must be written to the generated
 
 The `testar-cli` skill documentation is the operational source that tells agents how to use this contract.
 
+## Test Goals
+
+Reusable Test Goals for AI guided execution modes (e.g., CLI mode) are workspace assets stored as YAML files.
+
+Repository defaults live under:
+
+- `testar/resources/settings/{workspace}/test_goals`
+
+Distribution editable goals live under:
+
+- `testar/target/install/testar/bin/settings/{workspace}/test_goals`
+
+The distribution process should copy each workspace's Test Goals together with the rest of that workspace.
+
+WebStudio should edit only the selected workspace's distributed `test_goals` copy.
+
+Executable managed goal files use `.yaml` or `.yml`.
+
+The YAML contract is intentionally structured so WebStudio can validate required fields and agents can consume goal content reliably.
+
 ## Output Folder Naming
 
 Generated output folders must include the execution mode after the timestamp.

docs/webstudio/WEBSTUDIO_BUG_BACKLOG.md

@@ -284,3 +284,30 @@ Acceptance:
 - shared result status classification treats `OK` and `LLM_COMPLETE` as successful
 - `LLM_INVALID` and non-success verdicts remain issue outcomes
 - regression test `ResultVerdictStatusTest` passes
+
+### WS-011 Test Goals Folder Selection Kept YAML Editor Open
+
+- Area: Test Goals
+- Status: fixed
+
+Reproduction:
+
+1. Open `Test Goals`.
+2. Select a YAML goal file.
+3. Select a folder in the goal tree.
+
+Expected:
+
+- the right panel switches from YAML editing to the selected-folder creation panel
+- if the YAML editor has unsaved changes, the unsaved-change guard appears before switching
+
+Bug:
+
+- folder selection was local to the view while YAML file selection was stored in the parent view state
+- selecting a folder did not clear the selected YAML file, so the right panel kept showing the YAML editor
+
+Acceptance:
+
+- folder selection is stored in the parent Test Goals state
+- selecting a folder clears the selected YAML file after the unsaved-change guard is resolved
+- regression test `testGoalFolderSelectionState` passes

docs/webstudio/WEBSTUDIO_FUNCTIONAL_SPEC.md

@@ -339,6 +339,7 @@ CLI Mode supports:
 - CLI command execution
 - CLI session stop
 - focused editing of Agent CLI settings stored in the selected workspace `test.settings`
+- prompt-driven execution of managed Test Goals stored in the selected workspace
 
 Workspace selection:
 
@@ -347,6 +348,126 @@ Workspace selection:
 - CLI validates connector values server-side and reports startup errors to the user
 - Manual and Agent CLI execution use the selected workspace settings, including `CliStateProjectionMode` and `AgentCLI...` values
 
+## Test Goals
+
+Test Goals are reusable goal definitions consumed by AI guided execution modes (e.g., CLI mode).
+
+Test Goals are workspace assets. Each workspace may contain its own `test_goals` folder next to its settings, composition, policies, and Java sources.
+
+Repository defaults live under:
+
+- `testar/resources/settings/{workspace}/test_goals`
+
+The distributed editable copy lives under:
+
+- `testar/target/install/testar/bin/settings/{workspace}/test_goals`
+
+WebStudio works with the distributed `test_goals` folder for the selected workspace.
+
+Changing the selected workspace changes the Test Goals tree. WebStudio must not mix Test Goals from different workspaces in the same view.
+
+### Supported Goal Files
+
+Managed executable test goals use YAML files.
+
+Allowed executable extensions:
+
+- `.yaml`
+- `.yml`
+
+### YAML Contract
+
+Each executable goal file must use this shape:
+
+```yaml
+version: 1
+goals:
+  - id: unique-goal-id
+    title: Human readable title
+    category: functional
+    narrative: >
+      Optional user story or context.
+    objective: Clear objective the agent must complete.
+    inputs:
+      platform: webdriver
+      sut: https://example.org/
+    prerequisites:
+      - Optional setup condition.
+    expected_outcomes:
+      - Observable expected outcome.
+    validation_notes:
+      - Optional validation guidance.
+```
+
+Required fields:
+
+- `version`
+- `goals`
+- `goals[].id`
+- `goals[].title`
+- `goals[].objective`
+- `goals[].expected_outcomes`
+
+Recommended fields:
+
+- `goals[].category`
+- `goals[].inputs.platform`
+- `goals[].inputs.sut`
+- `goals[].validation_notes`
+
+Goal identifiers should use kebab-case.
+
+### Authoring Behavior
+
+WebStudio should provide a dedicated `Test Goals` view.
+
+The view should support:
+
+- browsing folders and YAML goal files under the selected workspace `test_goals` folder
+- creating folders
+- creating YAML goal files
+- editing YAML goal files
+- deleting folders and files with confirmation
+- saving edited YAML goal files
+- discarding unsaved edits
+- validating the required YAML shape before saving or before execution
+
+All file operations must stay inside the selected workspace `test_goals` folder.
+
+Path traversal, absolute paths, and unsafe symlink escapes must be rejected.
+
+If the selected workspace has no Test Goals yet, the view should show a stable empty state and allow creating folders or YAML files under that workspace.
+
+### CLI Mode Integration
+
+Test Goals are not selected for execution in the authoring view.
+
+The `Test Goals` view is only responsible for creating, editing, deleting, and organizing YAML goal files and directories.
+
+CLI Mode remains prompt-driven.
+
+The user prompt can instruct the agent to execute:
+
+- one named YAML test goal file
+- multiple named YAML test goal files
+- the YAML test goals in one named directory
+- the YAML test goals in multiple named directories
+- all YAML test goals under a named parent directory
+
+WebStudio must include the selected workspace `test_goals` root path in the generated Agent CLI prompt so the agent can resolve the user-requested goal files or directories.
+
+The Agent CLI prompt contract must tell the agent:
+
+- to resolve user-referenced goal paths relative to the selected workspace `test_goals` root
+- to read the requested YAML goal files before execution
+- to execute every goal defined in each requested YAML file
+- to execute all matching YAML goal files when the user instruction is broad or matches multiple goals
+- to use recursive directory execution when the user asks for all goals in a directory
+
+Directory execution means recursive YAML discovery when the user asks for all goals in a directory.
+
+Agent CLI execution must still finalize each goal session with `LLM_COMPLETE` or `LLM_INVALID` and execute `shutdownDaemon` after the final goal session.
+
 ### Agent CLI Settings
 
 Agent CLI settings are stored in the selected workspace `test.settings` file with `AgentCLI...` names.

docs/webstudio/WEBSTUDIO_UX_SPEC.md

@@ -231,6 +231,29 @@ Expected behavior:
 - command buttons do not overflow horizontally
 - Agent CLI settings edited in CLI mode use the same unsaved-change dialog pattern as configuration settings
 
+### Test Goals
+
+Expected behavior:
+
+- Test Goals use a dedicated authoring view
+- the left panel shows a folder/file tree rooted at the selected workspace `test_goals` folder
+- changing the selected workspace refreshes the Test Goals tree for that workspace
+- the editor panel shows the selected YAML goal file
+- folder and file lists have fixed panel size and scroll internally
+- the view contains only file and folder management controls
+- delete actions require modal confirmation
+- unclear confirmation always cancels deletion
+- unsaved YAML edits use the same save/discard/cancel guard pattern as other editors
+- YAML validation feedback appears in a stable panel and does not resize the layout dynamically
+- unsupported files may be shown as grey non-executable reference files
+
+Avoid:
+
+- adding execution controls to the Test Goals view
+- allowing file operations outside the selected workspace `test_goals` folder
+- mixing Test Goals from multiple workspaces in one tree
+- treating arbitrary free-text files as managed executable goals
+
 ## Test Results UX Contract
 
 The results page should provide fast visual understanding of output health.

testar/resources/settings/android_generic/test_goals/apidemos/functional/apidemos_play_streaming_button_exists.yaml

@@ -0,0 +1,24 @@
+version: 1
+goals:
+  - id: apidemos-media-player-streaming-button-exists
+    title: Play Streaming Video button exists in MediaPlayer
+    category: functional
+    narrative: >
+      As an Android API Demos user, I want to open the MediaPlayer demo screen and confirm that the Play Streaming Video button is available. 
+    objective: Open API Demos, navigate to Media > MediaPlayer, and verify that the Play Streaming Video button exists.
+    inputs:
+      sut: ApiDemos-debug.apk
+      platform: android
+      navigation_path:
+        - API Demos
+        - Media
+        - MediaPlayer
+      target_option: Play Streaming Video
+    expected_outcomes:
+      - The API Demos application launches successfully.
+      - The Media category is visible and can be opened.
+      - The MediaPlayer demo screen is opened.
+      - A button or selectable button labeled "Play Streaming Video" is visible on the MediaPlayer screen.
+    validation_notes:
+      - This is a dynamic Android UI goal.
+      - Agents should validate the presence of the button by visible text or accessibility label.

testar/resources/settings/webdriver_generic/test_goals/parabank/functional/parabank_find_transaction.yaml

@@ -0,0 +1,22 @@
+version: 1
+goals:
+  - id: parabank-find-transaction
+    title: Find at least one transaction with amount 100
+    category: functional
+    narrative: >
+      As an authenticated user, I want to access my banking data after logging in
+      with the demo credentials john/demo.
+      Then I can verify that there is at least one transaction with amount 100.
+    objective: Log in as john and verify that at least one transaction of 100 can be found.
+    inputs:
+      sut: https://para.testar.org/
+      platform: webdriver
+      username: john
+      password: demo
+    prerequisites:
+      - Log in successfully using username "john" and password "demo".
+      - An authenticated session is established before transaction verification begins.
+    expected_outcomes:
+      - A successful authenticated session is established.
+      - The application allows access to transaction data after login.
+      - At least one transaction with amount "100" is present and can be found.

testar/resources/settings/webdriver_generic/test_goals/parabank/functional/parabank_login_valid.yaml

@@ -0,0 +1,17 @@
+version: 1
+goals:
+  - id: parabank_login_valid
+    title: Login with demo credentials
+    category: functional
+    narrative: >
+      As a user, I want to log in with the credentials john/demo.
+      Then the Welcome John Smith message is shown.
+    objective: Log in successfully as the user john.
+    inputs:
+      sut: https://para.testar.org/
+      platform: webdriver
+      username: john
+      password: demo
+    expected_outcomes:
+      - A successful authenticated session is established.
+      - A welcome message containing "Welcome John Smith" is shown.

testar/resources/settings/windows_generic/test_goals/notepad/functional/notepad_edit_select_date_valid.yaml

@@ -0,0 +1,24 @@
+version: 1
+goals:
+  - id: notepad-edit-select-date
+    title: Selecting the date option in edit mode
+    category: functional
+    narrative: >
+      As a Notepad user, I want to open the Edit menu and choose the Date/Time option.
+      Then Notepad inserts the current system date and time into the text editor.
+    objective: Open Notepad, select Edit > Time/Date, and verify that the editor receives the current date and time.
+    inputs:
+      sut: notepad.exe
+      platform: windows
+      menu_path:
+        - Edit
+        - Time/Date
+    expected_outcomes:
+      - The Notepad editor is focused and available for text insertion.
+      - The Edit menu can be opened.
+      - The "Time/Date" command is selected from the Edit menu.
+      - Notepad inserts text into the editor.
+      - The inserted text corresponds to the current system date and time shown in the operating-system locale format.
+    validation_notes:
+      - This is a dynamic desktop goal.
+      - Agents should compare the text inserted into the editor with the current local system date/time instead of relying on one fixed literal string.