Improve VEX import performance#6141
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 25 |
| Duplication | 3 |
🟢 Coverage 81.76% diff coverage
Metric Results Coverage variation Report missing for 09d50b31 Diff coverage ✅ 81.76% diff coverage (70.00%) Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (09d50b3) Report Missing Report Missing Report Missing Head commit (e3f9bf9) 24885 20163 81.02% Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#6141) 170 139 81.76% Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%1 Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Addresses a few fundamental flaws of the VEX import functionality by: * Wrapping the entire procedure into a transaction, making it atomic and reducing the number of fsyncs the DB has to do. * Replacing N+1 query patterns with more efficient counterparts. * Memoizing lookups instead of re-computing / re-querying for every invocation. Note that there is still a lot more room for improvement, but these changes will already yield a noticeable improvement without us having to do a far-reaching refactor. Signed-off-by: nscuro <nscuro@protonmail.com>
nscuro
force-pushed
the
improve-vex-import-performance
branch
from
c7a2ff3 to
e3f9bf9
Compare
There was a problem hiding this comment.
Pull request overview
This PR improves CycloneDX VEX import performance by reducing repeated lookups, adding more efficient vulnerability/component queries, and introducing transactional processing around VEX application.
Changes:
- Adds optimized vulnerability presence and vulnerable-component queries.
- Refactors CycloneDX VEX import to index BOM refs and reuse component lookups.
- Updates analysis persistence paths and adds test coverage for multiple VEX responses.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/main/java/org/dependencytrack/parser/cyclonedx/CycloneDXVexImporter.java |
Refactors VEX application with BOM-ref indexing, memoized component lookups, and revised analysis/comment updates. |
src/main/java/org/dependencytrack/tasks/VexUploadProcessingTask.java |
Wraps VEX application in a transaction and modernizes event handling. |
src/main/java/org/dependencytrack/persistence/VulnerabilityQueryManager.java |
Adds optimized project vulnerability existence check and direct vulnerable-component lookup. |
src/main/java/org/dependencytrack/persistence/QueryManager.java |
Exposes new vulnerability and analysis helper methods. |
src/main/java/org/dependencytrack/persistence/FindingsQueryManager.java |
Adds transactional analysis create/update helpers. |
src/test/java/org/dependencytrack/parser/cyclonedx/CycloneDXVexImporterTest.java |
Adds coverage for persisting the last response while commenting each VEX response. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Ports DependencyTrack/dependency-track#6141 Note that VulnerabilityDao#hasVulnerabilities was replaced with an equivalent QueryManager method because we'd otherwise be calling JDBI from within a JDO transaction, which would lead to two DB connections being blocked by a VEX import. Signed-off-by: nscuro <nscuro@protonmail.com>
Description
Improves VEX import performance.
Addresses a few fundamental flaws of the VEX import functionality by:
Note that there is still a lot more room for improvement, but these changes will already yield a noticeable improvement without us having to do a far-reaching refactor.
Addressed Issue
N/A
Additional Details
N/A
Checklist
This PR fixes a defect, and I have provided tests to verify that the fix is effectiveThis PR introduces changes to the database model, and I have added corresponding update logicThis PR introduces new or alters existing behavior, and I have updated the documentation accordingly