Skip to content

Commit 1a84190

Browse files
DumbrisPaperclip-Paperclip
Dumbris
and
Paperclip-Paperclip
committed
fix(scanner): arm64 ramparts + trivy vuln-DB pre-cache (MCP-2150)
ramparts (GLIBC_2.39): - Split CI matrix into per-arch native builds: ubuntu-latest for amd64, ubuntu-24.04-arm for arm64. Eliminates QEMU overhead that caused arm64 cargo-install to time out (MCP-2395), which left a stale arm64 manifest in GHCR linked against glibc 2.39 (trixie-era rust:1-slim). - Add merge-ramparts job: assembles the multi-arch :latest manifest from :latest-amd64 + :latest-arm64 using docker buildx imagetools create. - Dockerfile already has the bookworm pin (PR #665); no Dockerfile change needed. trivy-mcp (vuln-DB download timeout): - Add docker/scanners/trivy/Dockerfile: wraps ghcr.io/aquasecurity/trivy:latest and pre-caches the ~96 MiB vuln DB at build time via trivy image --download-db-only. TRIVY_CACHE_DIR=/trivy-cache baked in so runtime reads the same path without --cache-dir flags. - Update registry_bundled.go: DockerImage -> ghcr.io/smart-mcp-proxy/scanner-trivy, Timeout 300s -> 120s (no download needed for fs scans), update NetworkReq comment. - Add trivy to scanner-images.yml matrix (linux/amd64,linux/arm64). - Add weekly cron schedule (Monday 03:00 UTC) so the pre-cached DB stays current. Co-Authored-By: Paperclip <noreply@paperclip.ing>
1 parent 6998910 commit 1a84190

3 files changed

Lines changed: 133 additions & 29 deletions

File tree

‎.github/workflows/scanner-images.yml‎

Lines changed: 97 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
name: scanner-images
22

33
# Build and publish the custom security scanner Docker images to
4-
# ghcr.io/smart-mcp-proxy/scanner-*. Vendor images (trivy, semgrep, the
5-
# AI scanner which lives in a separate repo) are NOT built here.
4+
# ghcr.io/smart-mcp-proxy/scanner-*. Vendor images (semgrep, the AI scanner
5+
# which lives in a separate repo) are NOT built here. trivy IS wrapped in our
6+
# own image (docker/scanners/trivy) to pre-cache the vuln DB (MCP-2150).
67
#
78
# Triggers:
89
# - push to main that touches docker/scanners/** or this workflow file
10+
# - weekly schedule: rebuilds trivy to pick up a fresh vulnerability DB
911
# - manual dispatch (for ad-hoc rebuilds)
1012
# - pull requests: builds without pushing (to catch Dockerfile drift)
1113

@@ -19,6 +21,9 @@ on:
1921
paths:
2022
- 'docker/scanners/**'
2123
- '.github/workflows/scanner-images.yml'
24+
schedule:
25+
# Weekly Monday 03:00 UTC — keeps the trivy vuln DB reasonably current.
26+
- cron: '0 3 * * 1'
2227
workflow_dispatch:
2328
inputs:
2429
scanner:
@@ -33,40 +38,62 @@ permissions:
3338

3439
jobs:
3540
build:
36-
runs-on: ubuntu-latest
41+
# Most scanners use ubuntu-latest (amd64) with QEMU for multi-arch builds.
42+
# ramparts uses per-arch native runners to avoid QEMU timeout on the Rust
43+
# `cargo install` step: `matrix.runner` selects the runner per entry.
44+
runs-on: ${{ matrix.runner }}
3745
strategy:
3846
fail-fast: false
3947
matrix:
40-
# `platforms` defaults to multi-arch. ramparts overrides to linux/amd64
41-
# only: it is the lone Rust-from-source wrapper, and `cargo install
42-
# ramparts` for an emulated linux/arm64 leg runs under QEMU (~5-10x
43-
# slower). A cold build of both arches blows past the runner budget and
44-
# the job gets cancelled mid-compile (MCP-2395). amd64-only finishes in
45-
# time; the binary is consumed on the host arch by Docker isolation.
48+
# `arch` is set only for ramparts per-arch entries. Entries without
49+
# `arch` push a combined multi-arch :latest directly; per-arch entries
50+
# push :<sha>-<arch> / :latest-<arch> tags and the merge-ramparts job
51+
# assembles the final multi-arch :latest manifest.
4652
include:
4753
- id: snyk
54+
runner: ubuntu-latest
4855
image: ghcr.io/smart-mcp-proxy/scanner-snyk
4956
context: docker/scanners/snyk
5057
platforms: linux/amd64,linux/arm64
5158
- id: cisco
59+
runner: ubuntu-latest
5260
image: ghcr.io/smart-mcp-proxy/scanner-cisco
5361
context: docker/scanners/cisco
5462
platforms: linux/amd64,linux/arm64
55-
- id: ramparts
56-
image: ghcr.io/smart-mcp-proxy/scanner-ramparts
57-
context: docker/scanners/ramparts
58-
platforms: linux/amd64
63+
- id: trivy
64+
runner: ubuntu-latest
65+
image: ghcr.io/smart-mcp-proxy/scanner-trivy
66+
context: docker/scanners/trivy
67+
platforms: linux/amd64,linux/arm64
5968
- id: proximity
69+
runner: ubuntu-latest
6070
image: ghcr.io/smart-mcp-proxy/scanner-proximity
6171
context: docker/scanners/proximity
6272
platforms: linux/amd64,linux/arm64
73+
# ramparts: native per-arch builds (MCP-2150).
74+
# Using QEMU to emulate arm64 on an amd64 runner blows past the job
75+
# budget (~5-10x slower for `cargo install`; MCP-2395). Instead each
76+
# arch builds on its native runner and the merge-ramparts job combines
77+
# them into a multi-arch :latest manifest.
78+
- id: ramparts
79+
arch: amd64
80+
runner: ubuntu-latest
81+
image: ghcr.io/smart-mcp-proxy/scanner-ramparts
82+
context: docker/scanners/ramparts
83+
platforms: linux/amd64
84+
- id: ramparts
85+
arch: arm64
86+
runner: ubuntu-24.04-arm
87+
image: ghcr.io/smart-mcp-proxy/scanner-ramparts
88+
context: docker/scanners/ramparts
89+
platforms: linux/arm64
6390
steps:
6491
- name: Checkout
6592
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
6693

6794
- name: Skip if not selected
6895
if: ${{ github.event_name == 'workflow_dispatch' && inputs.scanner != '' && inputs.scanner != matrix.id }}
69-
run: echo "Skipping ${{ matrix.id }} (workflow_dispatch selected ${{ inputs.scanner }})" && exit 0
96+
run: echo "Skipping ${{ matrix.id }}${{ matrix.arch && format('-{0}', matrix.arch) || '' }} (workflow_dispatch selected ${{ inputs.scanner }})" && exit 0
7097

7198
- name: Set up QEMU
7299
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
@@ -88,10 +115,22 @@ jobs:
88115
set -euo pipefail
89116
short_sha="${GITHUB_SHA:0:7}"
90117
echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT"
91-
if [ "${GITHUB_REF}" = "refs/heads/main" ]; then
92-
echo "tags=${{ matrix.image }}:latest,${{ matrix.image }}:$short_sha" >> "$GITHUB_OUTPUT"
118+
arch="${{ matrix.arch }}"
119+
if [ -n "$arch" ]; then
120+
# Per-arch build (ramparts): push arch-suffixed tags only.
121+
# The merge-ramparts job assembles :latest and :<sha> from these.
122+
if [ "${GITHUB_REF}" = "refs/heads/main" ]; then
123+
echo "tags=${{ matrix.image }}:latest-${arch},${{ matrix.image }}:${short_sha}-${arch}" >> "$GITHUB_OUTPUT"
124+
else
125+
echo "tags=${{ matrix.image }}:pr-${{ github.event.pull_request.number || 'dev' }}-${arch}" >> "$GITHUB_OUTPUT"
126+
fi
93127
else
94-
echo "tags=${{ matrix.image }}:pr-${{ github.event.pull_request.number || 'dev' }}" >> "$GITHUB_OUTPUT"
128+
# Multi-arch build: push :latest and :<sha> directly.
129+
if [ "${GITHUB_REF}" = "refs/heads/main" ]; then
130+
echo "tags=${{ matrix.image }}:latest,${{ matrix.image }}:$short_sha" >> "$GITHUB_OUTPUT"
131+
else
132+
echo "tags=${{ matrix.image }}:pr-${{ github.event.pull_request.number || 'dev' }}" >> "$GITHUB_OUTPUT"
133+
fi
95134
fi
96135
97136
- name: Build and push
@@ -101,8 +140,8 @@ jobs:
101140
platforms: ${{ matrix.platforms }}
102141
push: ${{ github.event_name != 'pull_request' }}
103142
tags: ${{ steps.tags.outputs.tags }}
104-
cache-from: type=gha,scope=scanner-${{ matrix.id }}
105-
cache-to: type=gha,scope=scanner-${{ matrix.id }},mode=max
143+
cache-from: type=gha,scope=scanner-${{ matrix.id }}${{ matrix.arch && format('-{0}', matrix.arch) || '' }}
144+
cache-to: type=gha,scope=scanner-${{ matrix.id }}${{ matrix.arch && format('-{0}', matrix.arch) || '' }},mode=max
106145
labels: |
107146
org.opencontainers.image.source=https://github.com/smart-mcp-proxy/mcpproxy-go
108147
org.opencontainers.image.revision=${{ github.sha }}
@@ -111,12 +150,12 @@ jobs:
111150
# multi-platform builds (linux/amd64,linux/arm64) cannot be loaded to the runner with
112151
# `load: true`, so there is no local image to scan on PRs. On push the image is
113152
# already in GHCR and Trivy pulls it directly.
114-
# exit-code: '0' → report-only, never fails the build. Scanner base images routinely
115-
# carry unfixable CVEs (upstream OS packages); blocking builds on those would create
116-
# constant noise with no actionable remediation. Visibility via the Security tab and
117-
# the workflow log is the goal.
153+
# Skipped for per-arch ramparts builds (matrix.arch set): those entries push
154+
# arch-suffixed tags that get merged by merge-ramparts; the merged :latest is
155+
# what should be scanned, not the intermediate arch-specific tags.
156+
# exit-code: '0' → report-only, never fails the build.
118157
- name: Scan image with Trivy
119-
if: github.event_name != 'pull_request'
158+
if: github.event_name != 'pull_request' && !matrix.arch
120159
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
121160
with:
122161
image-ref: ${{ matrix.image }}:${{ steps.tags.outputs.short_sha }}
@@ -127,7 +166,7 @@ jobs:
127166
format: table
128167

129168
- name: Upload Trivy SARIF to Security tab
130-
if: github.event_name != 'pull_request'
169+
if: github.event_name != 'pull_request' && !matrix.arch
131170
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
132171
with:
133172
image-ref: ${{ matrix.image }}:${{ steps.tags.outputs.short_sha }}
@@ -139,8 +178,40 @@ jobs:
139178
output: trivy-results-${{ matrix.id }}.sarif
140179

141180
- name: Upload SARIF
142-
if: github.event_name != 'pull_request'
181+
if: github.event_name != 'pull_request' && !matrix.arch
143182
uses: github/codeql-action/upload-sarif@8aad20d150bbac5944a9f9d289da16a4b0d87c1e # v4.36.2
144183
with:
145184
sarif_file: trivy-results-${{ matrix.id }}.sarif
146185
category: trivy-${{ matrix.id }}
186+
187+
# Assemble the multi-arch ramparts manifest from the two per-arch images
188+
# pushed by the build job. Runs only on push to main and workflow_dispatch
189+
# (not pull_request, since per-arch images aren't pushed on PRs).
190+
merge-ramparts:
191+
needs: build
192+
if: github.event_name != 'pull_request'
193+
runs-on: ubuntu-latest
194+
steps:
195+
- name: Log in to GHCR
196+
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
197+
with:
198+
registry: ghcr.io
199+
username: ${{ github.actor }}
200+
password: ${{ secrets.GITHUB_TOKEN }}
201+
202+
- name: Set up Buildx
203+
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
204+
205+
- name: Create multi-arch :latest manifest
206+
run: |
207+
docker buildx imagetools create \
208+
-t ghcr.io/smart-mcp-proxy/scanner-ramparts:latest \
209+
ghcr.io/smart-mcp-proxy/scanner-ramparts:latest-amd64 \
210+
ghcr.io/smart-mcp-proxy/scanner-ramparts:latest-arm64
211+
212+
- name: Create multi-arch SHA-tagged manifest
213+
run: |
214+
docker buildx imagetools create \
215+
-t ghcr.io/smart-mcp-proxy/scanner-ramparts:${GITHUB_SHA:0:7} \
216+
ghcr.io/smart-mcp-proxy/scanner-ramparts:${GITHUB_SHA:0:7}-amd64 \
217+
ghcr.io/smart-mcp-proxy/scanner-ramparts:${GITHUB_SHA:0:7}-arm64

‎docker/scanners/trivy/Dockerfile‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Trivy MCP Scanner — wraps the upstream Aqua image with a pre-cached
2+
# vulnerability database so scans don't race the download timeout on first run.
3+
#
4+
# Upstream: https://github.com/aquasecurity/trivy
5+
# Base: ghcr.io/aquasecurity/trivy:latest
6+
#
7+
# Published as: ghcr.io/smart-mcp-proxy/scanner-trivy:latest
8+
# Rebuilt weekly via scanner-images.yml schedule to keep the DB current.
9+
#
10+
# Why a custom wrapper (MCP-2150):
11+
# The upstream image downloads its ~96 MiB vuln DB on first scan. On slow
12+
# connections this races the per-scanner timeout and causes exit 1 with no
13+
# output. Baking the DB into the image eliminates the download entirely for
14+
# filesystem scans (trivy fs) and avoids the race for image scans too.
15+
#
16+
# Runtime notes:
17+
# TRIVY_CACHE_DIR is set here so the trivy binary picks up the baked-in DB
18+
# at the same path without needing --cache-dir flags in the scan command.
19+
# NetworkReq is still required for `trivy image` scanning (pulling remote
20+
# container images from registries) but NOT for `trivy fs` (vuln DB is local).
21+
FROM ghcr.io/aquasecurity/trivy:latest
22+
LABEL org.opencontainers.image.source="https://github.com/smart-mcp-proxy/mcpproxy-go"
23+
LABEL org.opencontainers.image.description="Trivy vulnerability scanner with pre-cached vuln DB for MCPProxy"
24+
LABEL org.opencontainers.image.licenses="Apache-2.0"
25+
26+
# Pre-cache the vulnerability database at image build time.
27+
# --download-db-only: fetch the DB and exit, no scan performed (trivy 0.34+).
28+
# --no-progress: suppress the progress bar so build logs are clean.
29+
ENV TRIVY_CACHE_DIR=/trivy-cache
30+
RUN trivy --cache-dir /trivy-cache image --download-db-only --no-progress

‎internal/security/scanner/registry_bundled.go‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,10 @@ var bundledScanners = []*ScannerPlugin{
152152
Description: "Comprehensive vulnerability scanner for filesystem, dependencies, and container images. Detects known CVEs and misconfigurations.",
153153
License: "Apache-2.0",
154154
Homepage: "https://trivy.dev",
155-
DockerImage: "ghcr.io/aquasecurity/trivy:latest",
155+
// Our wrapper pre-caches the vuln DB at image build time (MCP-2150),
156+
// eliminating the ~96 MiB first-run download that raced the timeout.
157+
// Rebuilt weekly via scanner-images.yml to keep the DB current.
158+
DockerImage: "ghcr.io/smart-mcp-proxy/scanner-trivy:latest",
156159
Inputs: []string{"source", "container_image"},
157160
Outputs: []string{"sarif"},
158161
RequiredEnv: nil,
@@ -163,7 +166,7 @@ var bundledScanners = []*ScannerPlugin{
163166
// daemon/containerd/podman and falls back to pulling from the remote
164167
// registry (network is enabled), so no docker socket mount is required.
165168
ImageCommand: []string{"image", "--format", "sarif", "{{IMAGE}}"},
166-
Timeout: "300s", // First run downloads vuln DB (~90MB)
167-
NetworkReq: true, // Needs to download vulnerability database
169+
Timeout: "120s", // DB is pre-cached; network only needed for image pulls
170+
NetworkReq: true, // Required for `trivy image` pulling remote container images
168171
},
169172
}

0 commit comments

Comments
 (0)