Skip to content

feat(syft): add Dockerfile/Containerfile image analysis scenarios for hardened recommendations#38

Open
a-oren wants to merge 3 commits into
trustification:mainfrom
a-oren:TC-4813
Open

feat(syft): add Dockerfile/Containerfile image analysis scenarios for hardened recommendations#38
a-oren wants to merge 3 commits into
trustification:mainfrom
a-oren:TC-4813

Conversation

@a-oren

@a-oren a-oren commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Implement syft runtime support in common_test_functions.py (previously TODO placeholders)
  • Add three new integration test scenarios under scenarios/syft/:
    • dockerfile-hardened-recommendations: verifies recommendationSource="hardened" in backend response
    • dockerfile-recommendations-disabled: verifies recommend=false suppresses all recommendations
    • containerfile-ubi-recommendations: verifies recommendationSource="ubi" with Containerfile manifest
  • Add recommendation validation (validate_recommendations) to the test runner
  • Add manifest_file override and env variable support in spec.yaml
  • Add syft env vars to no-runtime test coverage

Test plan

  • CI runs syft-based scenarios against staging backend successfully
  • Hardened image recommendations are present for node:18
  • Recommendations are suppressed with TRUSTIFY_DA_RECOMMEND=false
  • UBI recommendations are present for python:3.11-slim
  • Existing non-syft scenarios are unaffected (no behavioral changes)

Implements TC-4813

🤖 Generated with Claude Code

… hardened recommendations

Implement the syft runtime support (previously TODO placeholders) in
common_test_functions.py and add three new integration test scenarios:
- dockerfile-hardened-recommendations: verifies recommendationSource="hardened"
- dockerfile-recommendations-disabled: verifies recommend=false suppresses recommendations
- containerfile-ubi-recommendations: verifies recommendationSource="ubi"

Also adds recommendation validation to the test runner, manifest_file override
support in spec.yaml, and per-scenario environment variable configuration.

Implements TC-4813

Assisted-by: Claude Code
@a-oren

a-oren commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator Author

Verification Report for TC-4813 (commit 34cd137)

Check Result Details
Review Feedback N/A No review comments on PR
Root-Cause Investigation N/A No sub-tasks created
Scope Containment FAIL 2 task-specified files missing (setup-runtime.sh, integration.yml); scenarios nested under syft/ subdirectory vs task spec; 2 extra files modified (run_tests.py, run_tests_no_runtime.py)
Diff Size PASS 9 files changed — proportionate to task scope
Commit Traceability PASS Commit 34cd137 references TC-4813
Sensitive Patterns PASS No secrets or credentials detected
CI Status WARN No CI checks reported — workflow uses workflow_call trigger (invoked externally)
Acceptance Criteria PASS All 5 criteria satisfied — syft runtime implemented, 3 scenarios created with correct spec.yaml
Test Quality N/A No test files in PR diff; Eval Quality: N/A
Test Change Classification N/A No test files in PR diff
Verification Commands N/A Task commands reference non-existent run_scenarios.py; actual runner requires CLI artifact

Overall: FAIL

Scope Containment details:

The Scope Containment FAIL reflects a mismatch between the Jira task specification and actual implementation:

  • Missing from PR (per task spec): shared-scripts/setup-runtime.sh and .github/workflows/integration.yml — these files were inspected during implementation and found to already have syft support configured. No changes were needed.
  • Path deviation: Scenarios placed under scenarios/syft/ instead of scenarios/ — this follows the existing codebase architecture where get_scenario_base_dir("syft") returns "syft", grouping scenarios by runtime.
  • Extra files: run_tests.py and run_tests_no_runtime.py were modified to add recommendation validation logic and syft environment variables — necessary for the new scenarios to function.

These deviations were reviewed and approved during implementation. The task specification was inaccurate regarding the files that needed modification.


This comment was AI-generated by sdlc-workflow/verify-pr v0.11.0.

@a-oren a-oren requested a review from ruromero June 24, 2026 08:34

@ruromero ruromero left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but try to validate it from the Java and Javascript clients first using a test PR pointing to a-oren:TC-4813

a-oren and others added 2 commits June 24, 2026 13:30
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove containerfile-ubi-recommendations scenario per updated TC-4813
scope. Update recommendation validation to check hardened image
recommendations at providers.*.recommendations.hardened.summary.total.
Update Dockerfiles to use node:22.

TC-4813
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants