fix(bomsh): drop verifier check (C) and _bomsh.artefact

Check (C) compared two sha1s that are different by construction:
the ArtifactID we wrote to `_bomsh.artefact` (sha1 of library
bytes) vs. the bom_id `bomsh_sbom.py -f` inserts into the SPDX
(sha1 of the OmniBOR Input Manifest, looked up via
omnibor/metadata/bomsh/bomsh_omnibor_doc_mapping).  Two prior
attempts (be88063 snapshot, efa5977 `-g <ArtifactID>`) each
fixed (C) at the cost of breaking (A): bomsh's
omnibor/objects/ only stores Input Manifests keyed by bom_ids.

Customers walk the ADG from the SPDX gitoid; they need only
(A) "resolves in objects/" and (B) "objects/ self-consistent",
both retained.  (C) was CI-internal hygiene already covered by
the explicit WOLFSSL_LIB_DSO_BASENAMES loop in the recipe.

Drop the manifest, check (C), and its plumbing across
Makefile.am, bomsh_verify.py, test_gen_sbom.py, and sbom.yml.
Restore `bomsh_sbom.py -f <library>` (the bom_id it inserts
resolves in objects/ by construction).

Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>

Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>

Author: sameehj

.github/workflows/sbom.yml

@@ -1003,28 +1003,24 @@ jobs:
             pyspdxtools --infile "$f"
           done
 
-      - name: Bomsh provenance is end-to-end verifiable
-        # Three independent self-consistency checks on the bomsh
+      - name: Bomsh provenance bundle is internally consistent
+        # Two independent self-consistency checks on the bomsh
         # provenance bundle.  The PERSISTENT-ID assertion above only
-        # proves the gitoid externalRef *exists*; none of these
-        # follow-up properties are guaranteed by it:
+        # proves the gitoid externalRef *exists*; neither of these
+        # follow-up properties is guaranteed by it:
         #
         #   (A) every gitoid in the SPDX externalRefs resolves to a
         #       blob present in omnibor/objects/<aa>/<rest>
         #   (B) every blob in omnibor/objects/ round-trips through
         #       sha1(b"blob <len>\0" + content) so the object store
         #       is internally self-consistent (no bit-rot, no
         #       truncation, no stray non-blob file under objects/)
-        #   (C) the gitoid recorded against the wolfSSL package equals
-        #       the git-blob hash of the actual library artefact that
-        #       `make bomsh` traced (the SBOM ties to the binary that
-        #       would actually ship)
         #
         # Without this, a future bomsh_sbom.py change that emits a
         # plausibly-shaped but fictional gitoid (one that does not
-        # resolve in the ADG, or resolves but to the wrong artefact)
-        # would pass the existing PERSISTENT-ID assertion and ship a
-        # provenance bundle whose externalRef is a lie.
+        # resolve in the ADG) would pass the existing PERSISTENT-ID
+        # assertion and ship a provenance bundle whose externalRef is
+        # a lie.
         #
         # The verifier logic lives in scripts/bomsh_verify.py so it can
         # be unit-tested with synthetic fixtures (see the
@@ -1077,19 +1073,14 @@ jobs:
       - name: Upload bomsh trace diagnostics
         # Diagnostic-only, short retention.  Kept separate so the
         # provenance bundle above stays slim for downstream consumers
-        # who don't need to debug ptrace gaps.  `_bomsh.artefact` is
-        # included here (not in the provenance bundle) because it is
-        # CI-internal: a pointer file recording the path and gitoid of
-        # the bomtrace3-traced library that bomsh_sbom.py was told to
-        # cite in the SPDX externalRef.
+        # who don't need to debug ptrace gaps.
         if: always()
         uses: actions/upload-artifact@v4
         with:
           name: bomsh-trace-diag-${{ github.sha }}
           path: |
             bomsh_raw_logfile.sha1
             _bomsh.conf
-            _bomsh.artefact
           if-no-files-found: warn
           retention-days: 14
 
@@ -1104,5 +1095,3 @@ jobs:
               exit 1
           fi
           test ! -d omnibor || (echo "omnibor/ not cleaned"; exit 1)
-          test ! -f _bomsh.artefact \
-            || (echo "_bomsh.artefact not cleaned"; exit 1)

Makefile.am

@@ -474,18 +474,6 @@ BOMSH_RAWLOG           = $(BOMSH_RAWLOG_BASE).sha1
 BOMSH_CONF             = $(abs_builddir)/_bomsh.conf
 BOMSH_OMNIBORDIR       = $(abs_builddir)/omnibor
 BOMSH_SPDX_OUT         = omnibor.wolfssl-$(PACKAGE_VERSION).spdx.json
-# Single-source-of-truth manifest of the library artefact bomtrace3
-# actually traced.  Format: one line, '<path>\t<gitoid>'.  Both fields
-# are captured by the bomsh: recipe right after bomtrace3 finishes, so
-# downstream verification (CI: `Bomsh provenance is end-to-end
-# verifiable`) compares the SPDX gitoid against the gitoid bomsh
-# itself recorded -- decoupling check (C) from the file's *current*
-# bytes, which `make sbom`'s subsequent `make install` step relinks
-# in place via libtool (RPATH fixup), changing the gitoid that would
-# be re-computed off the on-disk file.  The verifier still warns when
-# the on-disk gitoid disagrees, so the install-time relink remains
-# visible.
-BOMSH_ARTEFACT_MANIFEST = $(abs_builddir)/_bomsh.artefact
 bomshdir          = $(datadir)/doc/$(PACKAGE)
 
 .PHONY: bomsh install-bomsh uninstall-bomsh
@@ -514,49 +502,29 @@ bomsh:
 	@printf 'raw_logfile=%s\n' '$(BOMSH_RAWLOG_BASE)' > '$(BOMSH_CONF)'
 	$(BOMTRACE3) -c '$(BOMSH_CONF)' $(MAKE)
 	$(BOMSH_CREATE_BOM) -r '$(BOMSH_RAWLOG)' -b '$(BOMSH_OMNIBORDIR)'
-	@# Capture the ArtifactID (file gitoid) of the bomtrace3-traced
-	@# library and record it in the manifest.  Below we feed this gitoid
-	@# to bomsh_sbom.py via -g (NOT -f): with -f, bomsh_sbom.py hashes
-	@# the file then maps that hash through omnibor/metadata/bomsh/
-	@# bomsh_omnibor_doc_mapping to a bom_id (the gitoid of the
-	@# artefact's OmniBOR document) -- a different sha1 than the
-	@# artefact's own content gitoid, which never matches what the
-	@# verifier records.  -g inserts our gitoid verbatim, so
-	@# SPDX externalRef == manifest gitoid == artefact ArtifactID.
-	@bomsh_artifact=""; \
-	for lib in \
-	    $(addprefix "$(abs_builddir)/src/.libs"/,$(WOLFSSL_LIB_DSO_BASENAMES)) \
-	    "$(abs_builddir)/src/.libs/libwolfssl.a" \
-	    "$(abs_builddir)/src/libwolfssl.a"; do \
-	    if test -f "$$lib"; then bomsh_artifact="$$lib"; break; fi; \
-	done; \
-	if test -n "$$bomsh_artifact"; then \
-	    bomsh_artifact_gid=`$(PYTHON3) -c 'import hashlib,sys;d=open(sys.argv[1],"rb").read();h=hashlib.sha1();h.update(("blob %d\0"%len(d)).encode());h.update(d);print(h.hexdigest())' "$$bomsh_artifact"`; \
-	    printf '%s\t%s\n' "$$bomsh_artifact" "$$bomsh_artifact_gid" \
-	        > '$(BOMSH_ARTEFACT_MANIFEST)'; \
-	fi
 	$(MAKE) sbom
 	@if test -z "$(BOMSH_SBOM)"; then \
 	    echo "NOTE: bomsh_sbom.py not in PATH; skipping SPDX enrichment."; \
 	    echo "      The OmniBOR graph in $(BOMSH_OMNIBORDIR) is still produced."; \
 	    exit 0; \
 	fi; \
-	if test ! -f '$(BOMSH_ARTEFACT_MANIFEST)'; then \
+	bomsh_artifact=""; \
+	for lib in \
+	    $(addprefix "$(abs_builddir)/src/.libs"/,$(WOLFSSL_LIB_DSO_BASENAMES)) \
+	    "$(abs_builddir)/src/.libs/libwolfssl.a" \
+	    "$(abs_builddir)/src/libwolfssl.a"; do \
+	    if test -f "$$lib"; then bomsh_artifact="$$lib"; break; fi; \
+	done; \
+	if test -z "$$bomsh_artifact"; then \
 	    echo "NOTE: no built libwolfssl artifact found in $(abs_builddir)/src/.libs/"; \
 	    echo "      OmniBOR graph produced; SPDX enrichment skipped."; \
 	    exit 0; \
 	fi; \
-	bomsh_artifact=`awk 'NR==1 {print $$1}' '$(BOMSH_ARTEFACT_MANIFEST)'`; \
-	bomsh_artifact_gid=`awk 'NR==1 {print $$2}' '$(BOMSH_ARTEFACT_MANIFEST)'`; \
-	if test -z "$$bomsh_artifact_gid"; then \
-	    echo "ERROR: $(BOMSH_ARTEFACT_MANIFEST) is missing the gitoid field"; \
-	    exit 1; \
-	fi; \
-	echo "Enriching SPDX with OmniBOR ExternalRefs (artifact: $$bomsh_artifact, gitoid: $$bomsh_artifact_gid)..."; \
+	echo "Enriching SPDX with OmniBOR ExternalRefs (artifact: $$bomsh_artifact)..."; \
 	$(BOMSH_SBOM) \
 	    -b '$(BOMSH_OMNIBORDIR)' \
 	    -i '$(abs_builddir)/$(SBOM_SPDX)' \
-	    -g "$$bomsh_artifact_gid" \
+	    -f "$$bomsh_artifact" \
 	    -s spdx-json \
 	    -O '$(abs_builddir)'
 
@@ -573,7 +541,7 @@ uninstall-bomsh:
 	-rm -rf '$(DESTDIR)$(bomshdir)/omnibor'
 	-rm -f '$(DESTDIR)$(bomshdir)/$(BOMSH_SPDX_OUT)'
 
-CLEANFILES += $(BOMSH_RAWLOG) $(BOMSH_RAWLOG_BASE).sha256 $(BOMSH_CONF) $(BOMSH_SPDX_OUT) $(BOMSH_ARTEFACT_MANIFEST)
+CLEANFILES += $(BOMSH_RAWLOG) $(BOMSH_RAWLOG_BASE).sha256 $(BOMSH_CONF) $(BOMSH_SPDX_OUT)
 
 # Hook SBOM/Bomsh cleanup into `make uninstall` so packagers don't leave
 # stale artefacts behind after install-sbom/install-bomsh.  uninstall-sbom

scripts/bomsh_verify.py

@@ -1,47 +1,27 @@
 #!/usr/bin/env python3
 """End-to-end verifier for the bomsh provenance bundle.
 
-Three independent self-consistency checks on the artefacts that
+Two independent self-consistency checks on the artefacts that
 `make bomsh` produces.  The PERSISTENT-ID assertion in the bomsh CI
 job only proves the gitoid externalRef *exists* in the enriched SPDX;
-none of these follow-up properties are guaranteed by it:
+neither of these follow-up properties is guaranteed by it:
 
   (A) Resolvability  -- every gitoid in the SPDX externalRefs resolves
-      to a blob present at omnibor/objects/<aa>/<rest>.
+      to a blob present at omnibor/objects/<aa>/<rest>.  Catches the
+      `bomsh_sbom.py` regression class that emits a syntactically
+      well-formed gitoid which does not actually point at anything in
+      the shipped ADG.
 
   (B) Object-store integrity -- every blob in omnibor/objects/
       round-trips through sha1(b"blob <len>\\0" + content), so a
       corrupt or truncated object store is caught at PR time, not by
       a downstream verifier weeks later.
 
-  (C) Artefact correspondence -- the gitoid recorded against the
-      wolfSSL package equals the gitoid bomsh itself recorded for the
-      library it traced (read from the `_bomsh.artefact` manifest the
-      bomsh: Makefile target writes as '<path>\\t<gitoid>' BEFORE
-      `make sbom` runs).  This is the strongest claim the bomsh
-      pipeline alone can make: the SPDX agrees with what bomsh saw.
-
-      Comparing against bomsh's own recorded gitoid (rather than
-      against the on-disk file's *current* bytes) is deliberate.
-      `make sbom`'s subsequent `make install` step relinks
-      src/.libs/lib*.so* in place via libtool to fix RPATH, mutating
-      the bytes after bomsh has already gitoid-ed them.  The verifier
-      still hashes the on-disk file and emits a NOTE if it has
-      diverged, so the install-time relink remains visible without
-      causing a false negative on the bomsh<->SPDX agreement.
-
-Without this, a future `bomsh_sbom.py` change that emits a
-plausibly-shaped but fictional gitoid (one that does not resolve in
-the ADG, or resolves but to a different artefact than bomsh recorded)
-would pass the existing PERSISTENT-ID assertion and ship a provenance
-bundle whose externalRef is a lie.
-
 CLI form (used by `.github/workflows/sbom.yml`):
 
     python3 scripts/bomsh_verify.py \\
         --spdx-glob 'omnibor.wolfssl-*.spdx.json' \\
-        --omnibor-dir omnibor \\
-        --artefact-manifest _bomsh.artefact
+        --omnibor-dir omnibor
 
 Library form (used by scripts/test_gen_sbom.py):
 
@@ -55,7 +35,7 @@
 import json
 import os
 import sys
-from typing import List, Tuple
+from typing import List
 
 
 GITOID_LOCATOR_PREFIX = 'gitoid:blob:sha1:'
@@ -159,59 +139,11 @@ def check_object_store_integrity(omnibor_objects_dir):
     return obj_count, bad
 
 
-def parse_artefact_manifest(manifest_path):
-    """Parse the `_bomsh.artefact` manifest written by the bomsh:
-    recipe.  Format: a single line, `<absolute-path>\\t<gitoid-hex>`
-    -- both fields captured by the recipe AFTER bomtrace3 finishes
-    but BEFORE `make sbom` relinks the library.
-
-    Returns (path, recorded_gid).  Raises FileNotFoundError if the
-    manifest does not exist (bomsh: skipped artefact discovery, e.g.
-    no built library); raises ValueError if the line is malformed."""
-    if not os.path.isfile(manifest_path):
-        raise FileNotFoundError(
-            f'{manifest_path} not produced by `make bomsh`; cannot '
-            f'verify gitoid <-> artefact correspondence.  This usually '
-            f'means the bomsh enrichment step skipped the artefact-'
-            f'discovery loop (no built library).')
-    with open(manifest_path) as f:
-        line = f.readline().rstrip('\n')
-    if not line:
-        raise ValueError(
-            f'{manifest_path} is empty; bomsh: recipe wrote nothing')
-    parts = line.split('\t')
-    if len(parts) != 2 or not all(parts):
-        raise ValueError(
-            f'{manifest_path}: expected "<path>\\t<gitoid>", got {line!r}.  '
-            f'Re-run `make bomsh` against an up-to-date Makefile.am.')
-    return parts[0], parts[1]
-
-
-def check_artefact_correspondence(spdx_gitoids, recorded_gid,
-                                  package_name_substr='wolfssl'):
-    """(C) The gitoid bomsh recorded for the traced library matches a
-    gitoid externalRef on the wolfSSL SPDX package.  This is the
-    bomsh<->SPDX agreement check; it does NOT compare against the
-    on-disk file's current bytes (see module docstring).
-
-    Returns (matched, wolfssl_gids).  Raises ValueError if no SPDX
-    gitoid is associated with a wolfSSL-named package."""
-    wolfssl_gids = [gid for name, gid in spdx_gitoids
-                    if package_name_substr in name.lower()]
-    if not wolfssl_gids:
-        raise ValueError(
-            f'no SPDX gitoid externalRef on a package whose name '
-            f'contains {package_name_substr!r}; cannot verify '
-            f'artefact correspondence')
-    return recorded_gid in wolfssl_gids, wolfssl_gids
-
-
-def verify(spdx_glob, omnibor_dir, artefact_manifest,
-           package_name_substr='wolfssl'):
-    """Orchestrate the three checks.  Returns (ok: bool, messages:
+def verify(spdx_glob, omnibor_dir):
+    """Orchestrate the two checks.  Returns (ok: bool, messages:
     List[str]).  `messages` is appended to in success and failure both,
-    so callers can log the success line ('OK: N gitoids verified ...')
-    even when ok is True."""
+    so callers can log the success lines ('OK: N gitoid(s) verified' +
+    '    objects round-trip: M blobs') even when ok is True."""
     messages: List[str] = []
 
     spdx_paths = sorted(_glob.glob(spdx_glob))
@@ -247,50 +179,8 @@ def verify(spdx_glob, omnibor_dir, artefact_manifest,
             f'round-trip (object store is corrupt)')
         return False, messages
 
-    try:
-        artefact, recorded_gid = parse_artefact_manifest(artefact_manifest)
-    except (FileNotFoundError, ValueError) as e:
-        messages.append(str(e))
-        return False, messages
-
-    try:
-        matched, wolfssl_gids = check_artefact_correspondence(
-            spdx_gitoids, recorded_gid, package_name_substr)
-    except ValueError as e:
-        messages.append(str(e))
-        return False, messages
-
-    if not matched:
-        messages.append(
-            f'wolfSSL package SPDX gitoids {wolfssl_gids} do not '
-            f'include the gitoid bomsh recorded for the traced '
-            f'artefact {artefact} ({recorded_gid}); the SBOM is '
-            f'inconsistent with what bomsh actually saw')
-        return False, messages
-
     messages.append(f'OK: {len(spdx_gitoids)} gitoid(s) verified')
     messages.append(f'    objects round-trip: {obj_count} blobs')
-    messages.append(
-        f'    artefact match: {artefact} -> {recorded_gid} (bomsh-traced)')
-
-    # Diagnostic-only: the on-disk file may have been rewritten since
-    # bomsh saw it (the canonical case is `make sbom`'s `make install`
-    # step relinking via libtool to fix RPATH).  We do NOT fail on
-    # this -- the SBOM<->bomsh agreement above is what matters for
-    # the provenance proof -- but surfacing it as a NOTE keeps the
-    # divergence visible so it does not silently grow into a
-    # bigger gap (e.g. someone adds a strip step that goes unflagged).
-    if os.path.isfile(artefact):
-        on_disk = gitoid_sha1(artefact)
-        if on_disk != recorded_gid:
-            messages.append(
-                f'NOTE: on-disk {artefact} now has gitoid {on_disk}, '
-                f'but bomsh recorded {recorded_gid}.  This is expected '
-                f'when `make sbom` runs `make install` (libtool relinks '
-                f'src/.libs/lib*.so* in place to fix RPATH).  The SBOM '
-                f'attests to the bomsh-traced bytes; if you need it to '
-                f'attest to the *installed* bytes, the bomsh: recipe '
-                f'must trace `make install` too.')
     return True, messages
 
 
@@ -304,17 +194,9 @@ def main():
     parser.add_argument('--omnibor-dir', default='omnibor',
                         help='Path to the OmniBOR directory containing '
                              'objects/ (default: %(default)s)')
-    parser.add_argument('--artefact-manifest', default='_bomsh.artefact',
-                        help='Path to the file containing the artefact '
-                             'path that bomsh: traced (default: %(default)s)')
-    parser.add_argument('--package-name-substr', default='wolfssl',
-                        help='Case-insensitive substring used to identify '
-                             'the wolfSSL SPDX package among any others in '
-                             'the document (default: %(default)s)')
     args = parser.parse_args()
 
-    ok, messages = verify(args.spdx_glob, args.omnibor_dir,
-                          args.artefact_manifest, args.package_name_substr)
+    ok, messages = verify(args.spdx_glob, args.omnibor_dir)
     for line in messages:
         print(line, file=sys.stderr if not ok else sys.stdout)
     sys.exit(0 if ok else 1)