CXX-3309 Automate SBOM generation and Endor Labs scanning

[jasonhills-mongodb]

Objective

• Obviate manual SBOM component management via automated generation of a CycloneDX SBOM that includes all required and optional runtime components.
• Add Endor Labs SCA scanning of PRs (non-blocking)

Changes

• Add a GitHub Action workflow (.github/workflows/endor_scan_and_generate_sbom.yml) triggered by edits to cmake files that:
  • (push) performs an Endor Labs SCA scan and exports an SBOM for master or releases/v* branches, enriches SBOM with metadata, opens a PR for updated SBOM
  • (pull_request) performs an Endor Labs SCA scan for PRs (non-blocking) and, if potential vulnerabilities were found, add a comment to PR
  • (workflow_dispatch) workflow can also be triggered manually
• Remove the etc/purls.txt file
  • update all references to it in scripts and documentation
  • existing scripts and processes using Silkbomb for the augmented SBOM are unchanged, except that the sbom.json file is used as input instead of etc/purls.txt
• Change the location of the SBOM file from etc/cyclonedx.sbom.json to sbom.json for consistency with other MDB repos
• Add etc/sbom folder with:
  • Python scripts for SBOM generation using Endor Labs scan results and pre-defined enrichment data
    • Dependency group added to pyproject.toml
  • SBOM enrichment data in etc/sbom/metadata.cdx.json
• Updated SBOM-related documentation in etc/releasing.md

Testing

The workflow was thoroughly tested on a fork, including scenarios with missing or malformed files.

Permissions

The workflow requires the repository configuration to allow it to request write access and open PRs.

Miscellaneous

• Once this is merged to master, it may be cherry picked to the relevant release branches.
• Once approved, this will be ported to the C driver repo.
• The new SBOM format is in alignment with the NITA Minimum Elements for Software Bill of Materials and OWASP Software Component Verification Standard (SCVS). Both publications emphasize that all components used in creation of software are to be documented in an SBOM, including ecosystem-based dependencies, as a best practice.

[eramongodb]

Should the --languages flag use something like c,cpp instead? Catch2 (only used for tests) is a C++ library.

[jasonhills-mongodb]

In endorctl, you indicate "c" to enable both C and C++

[eramongodb]

Why not under etc as before? Is this required by Endor Labs?

[kevinAlbs]

Should zlib be marked "optional"? zlib is a default-enabled but optional component. The C driver may be built without zlib. And the C++ driver can be built using a custom C driver install:

# Install C driver without zlib:
cd "$HOME/code/mongo-c-driver"
cmake -DENABLE_ZLIB=OFF -S. -Bcmake-build -DCMAKE_INSTALL_PREFIX="$HOME/mongo-c-driver-nozlib"
cmake --build cmake-build --target install

# Build C++ driver with custom C driver:
cd "$HOME/code/mongo-cxx-driver"
cmake -DCMAKE_PREFIX_PATH="$HOME/mongo-c-driver-nozlib" -S. -Bcmake-build

However, I expect that is a rare scenario. I expect most users would have zlib enabled.

[kevinAlbs]

Suggested change

	Run a patch build which executes the `sbom` task and downloads the "Augmented SBOM (Updated)" file as `etc/augmented.sbom.json`. Evergreen CLI may be used to schedule only the `sbom` task:
	Run a patch build which executes the `sbom` task and download the "Augmented SBOM (Updated)" file as `etc/augmented.sbom.json`. Evergreen CLI may be used to schedule only the `sbom` task:

Tweak wording since the person doing the release is expected to download and commit the Augmented SBOM.

[kevinAlbs]

If Silk is no longer used, suggest similarly updating ssdlc_compliance_report.md.

Aside: ssdlc_compliance_report.md was a requirement of DRIVERS-2898 and refers to Proposal: SSDLC Compliance Verification Flow. I think that policy is no longer required in favor of Secure SDLC Best Practices. So maybe ssdlc_compliance_report.md is no longer strictly needed.