Skip to content

Commit 572db25

Browse files
committed
Try to improve speed for marking unfurl
Signed-off-by: Tushar Goel <tushar.goel.dav@gmail.com>
1 parent 173b114 commit 572db25

2 files changed

Lines changed: 6 additions & 4 deletions

File tree

CHANGELOG.rst

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,11 @@ next release
55
---------------------
66

77
- WARNING: Vulnerablecode V1 API and UI has stopped supporting Ubuntu OVAL advisories, please shift to V3 API for new Ubuntu advisories.
8-
- WARNING: We will deprecate improver pipelines for calculating package version rank and grouping advisories for packages in the next release, we are doing it at advisory import time instead of as separate pipelines, this will improve the performance and consistency of the data.
9-
- Calculate package verion rank, group advisories for packages and package risk score and advisory risk score during import of advisories.
8+
- WARNING: We will deprecate improver pipelines for calculating package version rank and grouping advisories for packages.
109
- Add attribute ``pipeline_id`` to AdvisoryV2 to track the pipeline that created the advisory, also rename existing ``datasource_id`` and AVIDs.
10+
- We will group advisories for packages and calculate package risk score and advisory risk score only when an advisory is completely unfurled,
11+
this will improve consistency of the data. In future if we change our grouping approach we have to make a migration to mark all advisories
12+
as not unfurled, so they can be goruped again with the new algo.
1113

1214

1315
Version v38.6.0

vulnerabilities/pipelines/v2_improvers/mark_unfurl_version_range.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ def mark_all_impacts_unfurled(self):
4444
impacted_packages=impacted_packages
4545
)
4646

47-
batch_size = 100
47+
batch_size = 1000
4848
batch = []
4949

5050
successful_qs = latest_advisories_with_all_impacts_unfurled_successfully(
@@ -106,7 +106,7 @@ def latest_advisories_with_all_impacts_unfurled_attempted(
106106
)
107107
.annotate(has_unattempted_impacts=Exists(impacts_not_attempted))
108108
.filter(has_unattempted_impacts=False)
109-
.order_by("_all_impacts_unfurled", "id")
109+
.order_by("_all_impacts_unfurled", "datasource_id")
110110
.values_list("id", flat=True)
111111
)
112112

0 commit comments

Comments
 (0)