JS-1905 [EXPERIMENT] Setup SIT-FPS diff validation jobs#7307
Conversation
Agentic Analysis: Early ResultsAgentic Analysis and Context Augmentation are available on your project. Here are some issues that could have been prevented. Follow the links to learn how to put them into action. 76 issue(s) found across 19 file(s):
Showing 50 of 76 issues. Analyzed by SonarQube Agentic Analysis in 9.2 s |
hashicorp-vault-sonar-prod
Bot
changed the title
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
This comment has been minimized.
This comment has been minimized.
nathsou
changed the title
nathsou
changed the title
|
Note: As noted by Michal, be should explore using the JS integration tests (à la ruling) and then generate the expected diff format for FPS to speed up the analysis even more. |
- Drop S2925 port (rule sources, fixtures, rule-data, java check) - Drop regenerated-and-gitignored files erroneously committed (AllChecks.java, metas.ts, rules.ts, plugin-rules.ts, rspec.sha) - Drop SIT validation design/plan docs - Drop sharding scripts and vendored peachee-public snapshot - Drop helper tests and SitExporterTest (out of scope for setup PR) - Drop orphan tools/diffsit/build.gradle.kts (SonarJS is Maven-only) - Restore stale eslint ruling entry (unrelated to SIT) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- git mv tools/diffsit dev-tools/diffsit to align with sonar-skunk
- Copy sit_pr_fps and sit_pr_diffsit Python helpers verbatim from
sonar-skunk, then apply minimal parametrization so they work for
SonarJS without forking divergence:
- normalize_sit_bundles.py: add optional --language flag for flat
input layouts
- build_publish_payload.py: rename --dre-plugin-version to
--plugin-version, add --rule-key-format and
--sit-artifact-name-template (defaults match sonar-skunk)
- prepare_diffsit_inputs.py: add --rule-key-format and --flat-bundles
(defaults match sonar-skunk)
- detect_changed_rules.py is JS-aware (reads compatibleLanguages from
rule metadata to fan out to javascript/typescript/css repositories)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace tools/sit/*.js Node helpers with the Python ports under
dev-tools/scripts/{sit_pr_fps,sit_pr_diffsit}/.
- Rewrite .github/workflows/sit-pr.yml as sit-pr-fps.yml mirroring
sonar-skunk's pipeline: prepare_rules -> target_sit_export &
baseline_sit_export -> diffsit -> fps -> publish. No sharding, no
language matrix.
- Slim .github/workflows/sit-export.yml to a single job that builds
the SIT exporter and runs it against the curated 20-project
manifest at dev-tools/peach-analysis/javascript/projects.json. The
in-workflow peachee clone loop is gone; the Java SitExporter clones
projects from projects.json itself.
- Update build.yml: rename caller job sit_pr -> sit-pr-fps, invoke
dev-tools/scripts/sit_pr_fps/detect_changed_rules.py to emit the
rules_json step output, and stop base64-encoding the payload.
- Drop tools/sit/ entirely.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The SitExporter is a CLI tool with no unit tests, so its src/test directory does not exist. Override the Sonar scanner default to avoid 'directory does not exist' failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds token/secret/apikey/auth/credential to the default password keyword list. This will fire on many code-base sites that previously were ignored, exercising both the SIT (new hits) and FPS (likely false positives in test fixtures and error messages) sides of the pipeline. Revert before merging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three independent CI failures:
1) SitExporter expected a JSON array of {name,testDir,exclusions}
pointing to the its/sources/projects git submodule, not the
peachee-public dict-with-clone-URLs format. Rewrite projects.json
to match SonarJS' ruling schema and keep 20 curated projects.
2) baseline_sit_export was checking out base_sha to build everything
including the SIT exporter itself, but base_sha (pre-PR) does not
have its/sit-exporter/. Restructure sit-export.yml so the exporter
is always built from the workflow ref; for baseline runs, check
out the baseline ref into baseline-src/ and build just the plugin
there. The exporter then runs against that baseline plugin jar.
3) Publish job's 'python = "3.13"' resolved to 3.13.14, which mise
tried to install as a freethreaded build and failed. Pin to
'3.13.7' across all sit-pr-fps jobs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Python detect_changed_rules.py helper emits 'javascript'/'typescript' (matching SQ repository keys), but SitExporter only accepted the short 'js'/'ts' filename-key form. Accept both. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace 'eslint' (absent from jsts-test-sources submodule) with 'ace' in the curated projects.json. Also make SitExporter skip projects with missing source directories instead of failing the whole export, which matches the resilient behaviour the sonar-skunk pipeline expects. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dev-tools/scripts depends on the private SonarSource/mega-false-positive-slayer-3000 repo via uv git source. The diffsit job needs to log in with the rspec GitHub token and run 'gh auth setup-git' before running uv, so that the underlying 'git clone' can authenticate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Vault role for SonarSource/SonarJS is not granted access to the RAD squad's anthropic-rad-squad path. Use the JS squad's own Portkey secret (development/team/analysis-js-squad/kv/data/sonarjs-portkey) that was already used in the previous Node-based pipeline. The mega-fp-slayer LM provider prefers PORTKEY_ANTHROPIC_API_KEY when set. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mark the squad-owned Vault path as the single per-analyser line that must be adjusted when copying this workflow to another analyser repo. Long-term: align all analysers on a single shared Vault path (one-time policy grant per repo) so the workflow can be truly identical. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The vault-action wrapper parses each non-empty line of the 'secrets:' block as 'path key | env_var' and rejects lines that start with '#'. Move the per-analyser guidance comment to a YAML comment above the step instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
'gh auth login' refuses to run when GH_TOKEN is set in the environment (it would use that value instead, and the subsequent gh auth setup-git then no-ops). Rename the env var holding the rspec token so 'gh auth login --with-token' actually reads from stdin and stores credentials. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The -rspec token is scoped only to the rspec repo, so 'uv sync' could not clone the private SonarSource/mega-false-positive-slayer-3000 git dependency. Switch the 'gh auth' steps in both DiffSIT and FPS to the broader -submodules token, which already has access to private SonarSource repos (and is what sonar-skunk uses). FPS still receives the -rspec token via GITHUB_TOKEN/GH_TOKEN for rule-spec fetching. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
SIT/FPS PR Automation
FPS per rule
|
CI failed: The SIT-FPS validation job failed due to a Rust version mismatch where the `zip` dependency requires a newer rustc version (1.88+) than is currently configured in the CI environment (1.85.0).OverviewOne build failure was identified in the FailuresRustc Version Mismatch (confidence: high)
Summary
Code Review ✅ Approved 10 resolved / 10 findingsImplements automated SIT/FPS diff validation jobs with project manifest management and shard-based analysis. All previously reported security and functional issues, including secret handling and hang-prone loops, have been resolved. ✅ 10 resolved✅ Security: Secret token interpolated directly into shell echo
✅ Edge Case: Non-numeric DiffSIT summary fields can render as NaN
✅ Quality: Empty SIT timing placeholder still renders an all-n/a table
✅ Edge Case: Artifact pagination loop can hang instead of failing over
✅ Edge Case: Explicitly requested project is silently dropped if also excluded
...and 5 more resolved from earlier reviews Tip Comment OptionsAuto-apply is off → Gitar will not commit updates to this branch. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
1 participant
Part of JS-1905
Summary by Gitar
peachee-projects.jsto manage project manifest building and shard matrix generation for large-scale analysis.diffsitreporting logic inoutput.rsandreader.rsto handle multi-project diffs and summary serialization.sit-pr.ymlfor automated rule-impacted validation and shard management.peachee-workflow.test.ts,publish-workflow.test.ts, andfps-workflow.test.tsto verify SIT/FPS reporting and manifest integrity.peachee-public/projects.jsonas a registry for public projects and includeduv.lockfor environment reproducibility.docs/superpowers/plans/2026-06-17-sonarjs-sit-diff-validation.mdoutlining the implementation roadmap for theS5759canary-based diff validation.This will update automatically on new commits.