Merge branch 'main' into add-amazonlinux-advisories

Author: ambuj-1211

.github/workflows/pypi-release-aboutcode-hashid.yml

@@ -0,0 +1,38 @@
+name: Build aboutcode.hashid Python distributions and publish on PyPI
+
+on:
+  workflow_dispatch:
+  push:
+   tags:
+     - "aboutcode.hashid/*"
+
+jobs:
+  build-and-publish:
+    name: Build and publish library to PyPI
+    runs-on: ubuntu-22.04
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.11
+
+      - name: Install flot
+        run: python -m pip install flot --user
+
+      - name: Build binary wheel and source tarball
+        run:  python -m flot --pyproject pyproject-aboutcode.hashid.toml --sdist --wheel --output-dir dist/
+
+      - name: Publish to PyPI
+        if: startsWith(github.ref, 'refs/tags')
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          password: ${{ secrets.PYPI_API_TOKEN_ABOUTCODE_HASHID }}
+
+      - name: Upload built archives
+        uses: actions/upload-artifact@v4
+        with:
+          name: pypi_archives
+          path: dist/*

.github/workflows/pypi-release.yml

@@ -37,7 +37,7 @@ jobs:
         run: python -m build --sdist --wheel --outdir dist/
 
       - name: Upload built archives
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: pypi_archives
           path: dist/*
@@ -51,7 +51,7 @@ jobs:
 
     steps:
       - name: Download built archives
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         with:
           name: pypi_archives
           path: dist
@@ -71,7 +71,7 @@ jobs:
 
     steps:
       - name: Download built archives
-        uses: actions/download-artifact@v3
+        uses: actions/download-artifact@v4
         with:
           name: pypi_archives
           path: dist

CHANGELOG.rst

@@ -1,12 +1,156 @@
 Release notes
 =============
 
-Version (next)
+
+Version v36.0.0
+---------------------
+
+- Add indexes for models https://github.com/aboutcode-org/vulnerablecode/pull/1701
+- Add fixed by package in V2 API https://github.com/aboutcode-org/vulnerablecode/pull/1706
+- Add tests for num queries for views https://github.com/aboutcode-org/vulnerablecode/pull/1730
+- Add postgresql conf in docker-compose https://github.com/aboutcode-org/vulnerablecode/pull/1733
+- Add default postgresql.conf for local docker build https://github.com/aboutcode-org/vulnerablecode/pull/1735
+- Add models for CodeFix https://github.com/aboutcode-org/vulnerablecode/pull/1704
+- Migrate Alpine Linux importer to aboutcode pipeline https://github.com/aboutcode-org/vulnerablecode/pull/1737
+- VCIO-next: Allow CVSS3.1 Severities in NVD https://github.com/aboutcode-org/vulnerablecode/pull/1738
+- Add Pipeline to add missing CVSSV3.1 scores https://github.com/aboutcode-org/vulnerablecode/pull/1740
+- Add description and reference to the latest release on the homepage https://github.com/aboutcode-org/vulnerablecode/pull/1743
+- Use proper apk package type for Alpine https://github.com/aboutcode-org/vulnerablecode/pull/1739
+- Optimize vulnerabilities view https://github.com/aboutcode-org/vulnerablecode/pull/1728
+- Add CWE support in multiple importers https://github.com/aboutcode-org/vulnerablecode/pull/1526
+- Fast content ID migration https://github.com/aboutcode-org/vulnerablecode/pull/1795
+- Add captcha for user signup https://github.com/aboutcode-org/vulnerablecode/pull/1822
+- Move the package search box to the top by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1832
+
+
+Version v35.1.0
+---------------------
+
+- Use AboutCode mirror for collecting CISA KEV #1685
+- Do not report ghost package as a fix for vulnerability #1679
+- Add pipeline to sort packages #1686
+- Fix urls for API #1678
+
+
+Version v35.0.0
+---------------------
+
+- Add scores in bulk search V1 API #1675
+- Add improver pipeline to flag ghost packages #644 #917 #1395 by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1533
+- Add base pipeline for importers and migrate PyPa importer to aboutcode pipeline by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1559
+- Remove dupe Package.get_non_vulnerable_versions by @pombredanne in https://github.com/aboutcode-org/vulnerablecode/pull/1570
+- Import data from GSD #706 by @ziadhany in https://github.com/aboutcode-org/vulnerablecode/pull/787
+- Add curl advisories importer by @ambuj-1211 in https://github.com/aboutcode-org/vulnerablecode/pull/1439
+- Update dependencies by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1590
+- Bump django from 4.2.0 to 4.2.15 by @dependabot in https://github.com/aboutcode-org/vulnerablecode/pull/1591
+- Bump cryptography from 42.0.4 to 43.0.1 by @dependabot in https://github.com/aboutcode-org/vulnerablecode/pull/1582
+- Bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows by @dependabot in https://github.com/aboutcode-org/vulnerablecode/pull/1581
+- Improve export command by @pombredanne in https://github.com/aboutcode-org/vulnerablecode/pull/1571
+- Fix typo in Kev requests import by @ziadhany in https://github.com/aboutcode-org/vulnerablecode/pull/1594
+- Prepare for release v34.0.1 by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1595
+- Bump upload-artifact to v4 by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1596
+- Migrate Npm importer to aboutcode pipeline by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1574
+- Use correct regex for CVE by @pombredanne in https://github.com/aboutcode-org/vulnerablecode/pull/1599
+- Migrate Nginx importer to aboutcode pipeline by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1575
+- Migrate GitLab importer to aboutcode pipeline by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1580
+- Migrate GitHub importer to aboutcode pipeline by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1584
+- Migrate NVD importer to aboutcode pipeline by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1587
+- Match affected and fixed-by Packages by @johnmhoran in https://github.com/aboutcode-org/vulnerablecode/pull/1528
+- Add management command to commit exported data by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1600
+- Add support to Exploits model by @ziadhany in https://github.com/aboutcode-org/vulnerablecode/pull/1562
+- Fix 500 Server Error with DRF browsable API and resolve blank Swagger API documentation by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1603
+- Release v34.0.2 by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1604
+- Bump VCIO version by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1605
+- Bump django from 4.2.15 to 4.2.16 by @dependabot in https://github.com/aboutcode-org/vulnerablecode/pull/1608
+- Bump fetchcode from v0.3.0 to v0.6.0 by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1607
+- Use 4-tier system for storing package metadata by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1609
+- Fix vers range crash by @pombredanne in https://github.com/aboutcode-org/vulnerablecode/pull/1598
+- Add GitHub action to publish aboutcode.hashid PyPI by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1615
+- Segregate PackageRelatedVulnerability model to new models by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1612
+- Add documentation for new pipeline design by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1621
+- Fix 500 error in /api/cpes endpoint by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1629
+- Migrate pysec importer to aboutcode pipeline by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1628
+- Avoid memory exhaustion during data migration by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1630
+- Add support for Calculating Risk in VulnerableCode by @ziadhany in https://github.com/aboutcode-org/vulnerablecode/pull/1593
+- Bulk create in migrations by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1640
+- Update README.rst by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1641
+- Prepare for release v34.1.0 by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1642
+- Add V2 API endpoints by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1631
+- Prepare for release v34.2.0 by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1647
+- Refactor severity score model and fix incorrect suse scores by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1636
+- Add bulk search in v2 by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1649
+- Prepare release v34.3.0 by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1652
+- Add `on_failure` to handle cleanup during pipeline failure by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1651
+- Fix API bug by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1654
+- Add reference score to package endpoint  by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1655
+- Prepare for release v34.3.2 by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1656
+- Add support for storing  exploitability and weighted severity by @ziadhany in https://github.com/aboutcode-org/vulnerablecode/pull/1646
+- Avoid migrations on version bumps by @keshav-space in https://github.com/aboutcode-org/vulnerablecode/pull/1660
+- Prepare v35.0.0rc1 by @TG1999 in https://github.com/aboutcode-org/vulnerablecode/pull/1664
+
+
+
+Version v35.0.0rc1
+---------------------
+
+- Add support for storing exploitability and weighted severity #1646
+- Avoid migrations on version bumps #1660
+
+
+Version v34.3.2
+----------------
+
+- HOTFIX: Add reference score to package endpoint #1655
+
+
+Version v34.3.1
+----------------
+
+- HOTFIX: Fix API bug #1654
+
+
+Version v34.3.0
+-----------------
+
+- Add bulk search in v2 #1649 
+- Refactor severity score model and fix incorrect suse scores #1636
+
+
+Version v34.2.0
+-------------------
+
+- Add V2 API endpoints #1631
+
+
+Version v34.1.0
+-------------------
+
+- Add support for Calculating Package Vulnerability Risk #1593
+- Migrate pysec importer to aboutcode pipeline #1628
+- Fix 500 error in /api/cpes endpoint #1629
+- Add documentation for new pipeline design #1621
+- Segregate PackageRelatedVulnerability model to new models #1612
+- Add GitHub action to publish aboutcode.hashid PyPI #1615
+- Fix vers range crash #1598
+- Use 4-tier system for storing package metadata #1609
+
+
+Version v34.0.2
+-------------------
+
+- Add management command to commit exported vulnerability data (#1600)
+- Fix API 500 error (#1603)
+
+
+Version v34.0.1
 -------------------
 
 - Add Pipeline to flag ghost packages (#1533)
 - Add logging configuration (#1533)
 - Drop support for python 3.8 (#1533)
+- Drop using docker-compose and use the built-in "docker compose" instead
+- Upgrade core dependencies including Django and Rest Framework
+- Fix typo in KEV improver (#1594)
 
 
 Version v34.0.0

Makefile

@@ -42,6 +42,10 @@ else
 	SUDO_POSTGRES=
 endif
 
+ifeq ($(UNAME), Darwin)
+	GET_SECRET_KEY=`head /dev/urandom | base64 | head -c50`
+endif
+
 virtualenv:
 	@echo "-> Bootstrap the virtualenv with PYTHON_EXE=${PYTHON_EXE}"
 	@${PYTHON_EXE} ${VIRTUALENV_PYZ} --never-download --no-periodic-update ${VENV}
@@ -125,13 +129,13 @@ bump:
 
 docs:
 	rm -rf docs/_build/
-	@${ACTIVATE} sphinx-build docs/ docs/_build/
+	@${ACTIVATE} sphinx-build docs/source docs/_build/
 
 docker-images:
 	@echo "-> Build Docker services"
-	docker-compose build
+	docker compose build
 	@echo "-> Pull service images"
-	docker-compose pull
+	docker compose pull
 	@echo "-> Save the service images to a compressed tar archive in the dist/ directory"
 	@mkdir -p dist/
 	@docker save postgres vulnerablecode_vulnerablecode nginx | gzip > dist/vulnerablecode-images-`git describe --tags`.tar.gz