Now that we moved to automated: assisted tests, it really feels as if packages/rangelink-vscode-extension/scripts/generate-qa-test-plan.sh and packages/rangelink-vscode-extension/scripts/generate-qa-issue.sh need a simplification ?
I still see a lot of values in packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yaml, but seeing that we've kept editing packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0-003.yaml for so many rounds of work, I feel like we should perhaps stick to one version of file per release and leverage git for tracking changes. This will most likely result in replacing the content from packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yaml with the content from packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0-003.yaml and simplify the scripts accounting for potential version suffixes on packages/rangelink-vscode-extension/qa/qa-test-cases-v*.yaml files.
The goal is to challenge the complexity from #460: instead of building 31 sub-issues (with as many checkmarks in the issue's description), what about having a single issue where the description contains checkbox for pnpm test:release:grep <proper filter> commands to run by the tester ? We've logically grouped integration tests per feature/theme. As long as the scripts generate one checkbox per feature group (where it's assumed the Test Case IDs (TC IDs) have a common prefix per feature), the tester will simply need to go down the list, copy + paste the command to run tests and validate the report was successfully created. The CI pipeline will be in charge of running automated: true, the issue will contain checkboxes + commands to run for all automated: assisted tests and we'll have an additional one to test the Ubuntu support for CTRL keybindings #510 will have solved.
The new way of automating tests for a release (like #460) must look at this through the lens of features and stop referring to Bug Fix and Changed categorization: every integration test must belong to a feature -- Bug Fixes and Changes from the CHANGELOG are not logical grouping for tests.
While we're in this area, we need to cleanup preconditions in the YAML file to remove obvious items like "Extension installed from .vsix build": this is pure noise that should not have made its way to the YAML files.
We'll need to simplify packages/rangelink-vscode-extension/TESTING.md too. What else to simplify ?
Now that we moved to
automated: assistedtests, it really feels as if packages/rangelink-vscode-extension/scripts/generate-qa-test-plan.sh and packages/rangelink-vscode-extension/scripts/generate-qa-issue.sh need a simplification ?I still see a lot of values in packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yaml, but seeing that we've kept editing packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0-003.yaml for so many rounds of work, I feel like we should perhaps stick to one version of file per release and leverage git for tracking changes. This will most likely result in replacing the content from packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yaml with the content from packages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0-003.yaml and simplify the scripts accounting for potential version suffixes on
packages/rangelink-vscode-extension/qa/qa-test-cases-v*.yamlfiles.The goal is to challenge the complexity from #460: instead of building 31 sub-issues (with as many checkmarks in the issue's description), what about having a single issue where the description contains checkbox for
pnpm test:release:grep <proper filter>commands to run by the tester ? We've logically grouped integration tests per feature/theme. As long as the scripts generate one checkbox perfeaturegroup (where it's assumed the Test Case IDs (TC IDs) have a common prefix per feature), the tester will simply need to go down the list, copy + paste the command to run tests and validate the report was successfully created. The CI pipeline will be in charge of runningautomated: true, the issue will contain checkboxes + commands to run for allautomated: assistedtests and we'll have an additional one to test the Ubuntu support for CTRL keybindings #510 will have solved.The new way of automating tests for a release (like #460) must look at this through the lens of features and stop referring to
Bug FixandChangedcategorization: every integration test must belong to a feature -- Bug Fixes and Changes from theCHANGELOGare not logical grouping for tests.While we're in this area, we need to cleanup
preconditionsin the YAML file to remove obvious items like "Extension installed from .vsix build": this is pure noise that should not have made its way to the YAML files.We'll need to simplify packages/rangelink-vscode-extension/TESTING.md too. What else to simplify ?