add a test plan

Author: dsyme

workflows/daily-perf-improver.md

@@ -69,7 +69,7 @@ Your name is ${{ github.workflow }}. Your job is to act as an agentic coder for
 
 1. Performance research (if not done before).
 
-   1a. Check if an open issue with title "${{ github.workflow }}: Research and Plan" exists. If it does, read the issue and its comments, paying particular attention to comment from repository maintainers, then continue to step 2. If not, follow the steps below to create it:
+   1a. Check if an open issue with title "${{ github.workflow }}: Research and Plan" exists. If it does, read the issue and its comments, paying particular attention to comments from repository maintainers, then continue to step 2. If not, follow the steps below to create it:
 
    1b. Do some deep research into performance matters in this repo.
      - How is performance testing is done in the repo?

workflows/daily-test-improver.md

@@ -68,53 +68,71 @@ steps:
 
 Your name is ${{ github.workflow }}. Your job is to act as an agentic coder for the GitHub repository `${{ github.repository }}`. You're really good at all kinds of tasks. You're excellent at everything.
 
-1. Build steps configuration.
+1. Testing research (if not done before).
 
-   1a. Check if `.github/actions/daily-test-improver/coverage-steps/action.yml` exists in this repo. Note this path is relative to the current directory (the root of the repo). If it exists then continue to step 2. If it doesn't then we need to create it:
+   1a. Check if an open issue with title "${{ github.workflow }}: Research and Plan" exists. If it does, read the issue and its comments, paying particular attention to comments from repository maintainers, then continue to step 2. If not, follow the steps below to create it:
+
+    1b. Research the repository to understand its purpose, functionality, and technology stack. Look at the README.md, project documentation, code files, and any other relevant information.
+
+    1c. Research the current state of test coverage in the repository. Look for existing test files, coverage reports, and any related issues or pull requests.
+
+    1d. Create an issue with title "${{ github.workflow }}: Research and Plan" that includes:
+      - A summary of your findings about the repository, its testing strategies, its test coverage
+      - A plan for how you will approach improving test coverage, including specific areas to focus on and strategies to use
+      - Details of the commands needed to run to build the project, run tests, and generate coverage reports
+      - Details of how tests are organized in the repo, and how new tests should be organized
+      - Opportunities for new ways of greatly increasing test coverage
+      - Any questions or clarifications needed from maintainers
+
+    1e. Continue to step 2. 
+
+2. Build steps configuration.
+
+   2a. Check if `.github/actions/daily-test-improver/coverage-steps/action.yml` exists in this repo. Note this path is relative to the current directory (the root of the repo). If it exists then continue to step 3. If it doesn't then we need to create it:
 
-   1b. Have a careful think about the CI commands needed to build the project, run tests, produce a coverage report and upload it as an artifact. Do this by carefully reading any existing documentation and CI files in the repository that do similar things, and by looking at any build scripts, project files, dev guides and so on in the repository. 
+   2b. Have a careful think about the CI commands needed to build the project, run tests, produce a coverage report and upload it as an artifact. Do this by carefully reading any existing documentation and CI files in the repository that do similar things, and by looking at any build scripts, project files, dev guides and so on in the repository. 
 
-   1c. Create the file `.github/actions/daily-test-improver/coverage-steps/action.yml` containing these steps, ensuring that the action.yml file is valid.
+   2c. Create the file `.github/actions/daily-test-improver/coverage-steps/action.yml` containing these steps, ensuring that the action.yml file is valid.
 
-   1d. Before running any of the steps, make a pull request for the addition of this file, with title "Updates to complete configuration of ${{ github.workflow }}", explaining that adding these build steps to your repo will make this workflow more reliable and effective.
+   2d. Before running any of the steps, make a pull request for the addition of this file, with title "Updates to complete configuration of ${{ github.workflow }}", explaining that adding these build steps to your repo will make this workflow more reliable and effective.
 
     - Use Bash `git add ...`, `git commit ...`, `git push ...` etc. to push the changes to your branch.
 
     - Use Bash `gh pr create --repo ${{ github.repository }} ...` to create a pull request with the changes.
 
-   1e. Try to run through the steps you worked out manually one by one. If the a step needs updating, then update the pull request you created in step 1d, using `update_pull_request` to make the update. Continue through all the steps. If you can't get it to work, then create an issue describing the problem and exit the entire workflow.
+   2e. Try to run through the steps you worked out manually one by one. If the a step needs updating, then update the pull request you created in step 2d, using `update_pull_request` to make the update. Continue through all the steps. If you can't get it to work, then create an issue describing the problem and exit the entire workflow.
 
-   1f. Exit the entire workflow with a message saying that the configuration needs to be completed by merging the pull request you created in step c.
+   2f. Exit the entire workflow with a message saying that the configuration needs to be completed by merging the pull request you created in step 2d.
 
-2. Decide what to work on.
+3. Decide what to work on.
 
-   2a. You can assume that the repository is in a state where the steps in `.github/actions/daily-test-improver/coverage-steps/action.yml` have been run and a test coverage report has been generated, perhaps with other detailed coverage information. Look at the steps in `.github/actions/daily-test-improver/coverage-steps/action.yml` to work out where the coverage report should be, and find it. If you can't find the coverage report, work out why the build or coverage generation failed, then create an issue describing the problem and exit the entire workflow.
+   3a. You can assume that the repository is in a state where the steps in `.github/actions/daily-test-improver/coverage-steps/action.yml` have been run and a test coverage report has been generated, perhaps with other detailed coverage information. Look at the steps in `.github/actions/daily-test-improver/coverage-steps/action.yml` to work out where the coverage report should be, and find it. If you can't find the coverage report, work out why the build or coverage generation failed, then create an issue describing the problem and exit the entire workflow.
 
-   2b. Read the coverge report. Be detailed, looking to understand the files, functions, branches, and lines of code that are not covered by tests. Look for areas where you can add meaningful tests that will improve coverage.
+   3b. Read the coverge report. Be detailed, looking to understand the files, functions, branches, and lines of code that are not covered by tests. Look for areas where you can add meaningful tests that will improve coverage.
 
-   2c. Check the most recent pull request with title starting with "${{ github.workflow }}" (it may have been closed) and see what the status of things was there. These are your notes from last time you did your work, and may include useful recommendations for future areas to work on.
+   3c. Check the most recent pull request with title starting with "${{ github.workflow }}" (it may have been closed) and see what the status of things was there. These are your notes from last time you did your work, and may include useful recommendations for future areas to work on.
 
-   2d. Check for any other pull requests you created before with title starting with "${{ github.workflow }}". Don't work on adding any tests that overlap with what was done there.
+   3d. Check for any other pull requests you created before with title starting with "${{ github.workflow }}". Don't work on adding any tests that overlap with what was done there.
 
-   2e. Based on all of the above, select multiple areas of relatively low coverage to work on that appear tractable for further test additions.
+   3e. Based on all of the above, select multiple areas of relatively low coverage to work on that appear tractable for further test additions.
 
-3. For each area identified, do the following:
+4. For each area identified, do the following:
 
-   3a. Create a new branch
+   4a. Create a new branch
 
-   3b. Write new tests to improve coverage. Ensure that the tests are meaningful and cover edge cases where applicable.
+   4b. Write new tests to improve coverage. Ensure that the tests are meaningful and cover edge cases where applicable.
 
-   3c. Build the tests if necessary and remove any build errors.
+   4c. Build the tests if necessary and remove any build errors.
 
-   3d. Run the new tests to ensure they pass.
+   4d. Run the new tests to ensure they pass.
 
-   3e. Once you have added the tests, re-run the test suite again collecting coverage information. Check that overall coverage has improved. If coverage has not improved then exit.
+   4e. Once you have added the tests, re-run the test suite again collecting coverage information. Check that overall coverage has improved. If coverage has not improved then exit.
 
-   3f. Apply any automatic code formatting used in the repo
+   4f. Apply any automatic code formatting used in the repo
 
-   3g. Run any appropriate code linter used in the repo and ensure no new linting errors remain.
+   4g. Run any appropriate code linter used in the repo and ensure no new linting errors remain.
 
-   3h. If you were able to improve coverage, create a draft pull request with your changes, including a description of the improvements made and any relevant context.
+   4h. If you were able to improve coverage, create a draft pull request with your changes, including a description of the improvements made and any relevant context.
 
     - Use Bash `git add ...`, `git commit ...`, `git push ...` etc. to push the changes to your branch.
 
@@ -138,11 +156,11 @@ Your name is ${{ github.workflow }}. Your job is to act as an agentic coder for
 
     - After creation, check the pull request to ensure it is correct, includes all expected files, and doesn't include any unwanted files or changes. Make any necessary corrections by pushing further commits to the branch.
 
-4. If you think you found bugs in the code while adding tests, also create one single combined issue for all of them, starting the title of the issue with "${{ github.workflow }}". Do not include fixes in your pull requests unless you are 100% certain the bug is real and the fix is right.
+5. If you think you found bugs in the code while adding tests, also create one single combined issue for all of them, starting the title of the issue with "${{ github.workflow }}". Do not include fixes in your pull requests unless you are 100% certain the bug is real and the fix is right.
 
-5. If you encounter any problems or have questions, include this information in the pull request or issue to seek clarification or assistance.
+6. If you encounter any problems or have questions, include this information in the pull request or issue to seek clarification or assistance.
 
-6. Create a file in the root directory of the repo called "workflow-complete.txt" with the text "Workflow completed successfully".
+7. Create a file in the root directory of the repo called "workflow-complete.txt" with the text "Workflow completed successfully".
 
 @include agentics/shared/no-push-to-main.md