Skip to content

Commit d674093

Browse files
ruromeroclaude
andcommitted
test(pip): replace manual ignore assertions with golden SBOM tests
The pip_pep621_ignore fixture was missing expected_stack_sbom.json and expected_component_sbom.json, relying on partial manual assertions instead of the standard SBOM_CASES golden-file pattern. Add the golden files and switch to deep.equal comparison. Document the golden SBOM convention in CONVENTIONS.md to prevent this gap in future implementations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent d4b7007 commit d674093

4 files changed

Lines changed: 48 additions & 15 deletions

File tree

CONVENTIONS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ test/
8282
- **Test patterns**: `expect(res).to.deep.equal(...)`, `expect(() => ...).to.throw('message')`
8383
- **Higher-order setup**: Functions like `interceptAndRun()` for test setup/teardown
8484
- **Prefer real tool invocations over env var overrides**: Tests should call the actual ecosystem tools (pip, uv, poetry, mvn, npm, etc.) rather than injecting pre-recorded output via `TRUSTIFY_DA_*` environment variables. The CI environment has these tools available. Env var overrides (`TRUSTIFY_DA_PIP_REPORT`, `TRUSTIFY_DA_UV_EXPORT`, etc.) exist for users who lack the tool locally, but tests should exercise the real tool path to catch integration issues.
85+
- **Golden SBOM files for every test fixture**: Every provider test fixture directory must include `expected_stack_sbom.json` and `expected_component_sbom.json` golden files. Tests must use the `SBOM_CASES` pattern to do a full `deep.equal` comparison of the provider output against these golden files. Manual partial assertions (e.g. checking a single component name) are not a substitute — they may be added as supplementary tests but never as the only verification for a fixture.
8586

8687
## Commit Messages
8788

test/providers/python_pyproject.test.js

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -219,21 +219,19 @@ suite('testing the python-pyproject data provider', () => {
219219
expect(names).to.not.include('pysocks')
220220
}).timeout(TIMEOUT)
221221

222-
/** Verifies exhortignore marker in PEP 621 dependencies excludes the dep. */
223-
test('exhortignore marker excludes dep from component analysis', async () => {
224-
let result = await pipProvider.provideComponent(path.join(pipIgnoreDir, 'pyproject.toml'))
225-
let sbom = JSON.parse(result.content)
226-
let names = sbom.components.map(c => c.name)
227-
expect(names).to.not.include('requests')
228-
}).timeout(TIMEOUT)
229-
230-
/** Verifies exhortignore excludes dep and its exclusive transitive deps from stack analysis. */
231-
test('exhortignore marker excludes dep from stack analysis', async () => {
232-
let result = await pipProvider.provideStack(path.join(pipIgnoreDir, 'pyproject.toml'))
233-
let sbom = JSON.parse(result.content)
234-
let names = sbom.components.map(c => c.name)
235-
expect(names).to.not.include('requests')
236-
}).timeout(TIMEOUT)
222+
/** Verifies exhortignore marker produces expected SBOM for stack and component analysis. */
223+
SBOM_CASES.forEach(({type, method, fixture}) => {
224+
test(`verify exhortignore produces expected sbom for ${type} analysis with pip`, async () => {
225+
let expectedSbom = fs.readFileSync(path.join(pipIgnoreDir, fixture)).toString().trim()
226+
expectedSbom = JSON.stringify(JSON.parse(expectedSbom))
227+
let result = await pipProvider[method](path.join(pipIgnoreDir, 'pyproject.toml'))
228+
expect(result).to.deep.equal({
229+
ecosystem: 'pip',
230+
contentType: 'application/vnd.cyclonedx+json',
231+
content: expectedSbom
232+
})
233+
}).timeout(TIMEOUT)
234+
})
237235

238236
/** Verifies name canonicalization (charset_normalizer -> charset-normalizer). */
239237
test('name canonicalization: charset_normalizer resolved as charset-normalizer', async () => {
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"bomFormat": "CycloneDX",
3+
"specVersion": "1.4",
4+
"version": 1,
5+
"metadata": {
6+
"timestamp": "2023-10-01T00:00:00.000Z",
7+
"component": {
8+
"name": "test-project",
9+
"version": "1.0.0",
10+
"purl": "pkg:pypi/test-project@1.0.0",
11+
"type": "application",
12+
"bom-ref": "pkg:pypi/test-project@1.0.0"
13+
}
14+
},
15+
"components": [],
16+
"dependencies": []
17+
}
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"bomFormat": "CycloneDX",
3+
"specVersion": "1.4",
4+
"version": 1,
5+
"metadata": {
6+
"timestamp": "2023-10-01T00:00:00.000Z",
7+
"component": {
8+
"name": "test-project",
9+
"version": "1.0.0",
10+
"purl": "pkg:pypi/test-project@1.0.0",
11+
"type": "application",
12+
"bom-ref": "pkg:pypi/test-project@1.0.0"
13+
}
14+
},
15+
"components": [],
16+
"dependencies": []
17+
}

0 commit comments

Comments
 (0)