Skip to content

Commit 3efb2ae

Browse files
feat(staticanalysis): trivy mannifest scan command (#41)
* trivy mannifest scan command * multiple manifest format supported * mivor refactorings
1 parent fbe0d1d commit 3efb2ae

5 files changed

Lines changed: 711 additions & 3 deletions

File tree

.claude/settings.local.json

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,14 @@
55
"Bash(go fmt:*)",
66
"Bash(go vet:*)",
77
"Bash(go build:*)",
8-
"Bash(chmod:*)"
8+
"Bash(chmod:*)",
9+
"Bash(grep -oE \"[^/]*Manifest[^/]*\\\\.toml$\")",
10+
"Bash(rm -rf scantest)",
11+
"Bash(mkdir -p scantest/multi scantest/single)",
12+
"Bash(/home/ubuntu/jh/jh scan *)",
13+
"Bash(gofmt -l scan.go main.go)"
914
],
1015
"deny": [],
1116
"ask": []
1217
}
13-
}
18+
}

CLAUDE.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ The application follows a command-line interface pattern using the Cobra library
2121
- **credentials.go**: Registry credential management (list, add, update, delete) with REST API integration
2222
- **landing.go**: Landing page management (show, update, remove) with REST API integration
2323
- **vuln.go**: Vulnerability scanning for Julia packages via REST API
24+
- **scan.go**: Trivy manifest scanning — discover and upload a Julia manifest (+ optional project file) and fetch results via the StaticAnalysis service. Manifest discovery recognizes every name Julia/Pkg accepts (`Manifest.toml`, `JuliaManifest.toml`, and version-specific `Manifest-v1.11.toml` / `JuliaManifest-v1.11.toml`); when a directory holds more than one, the user is prompted to select (defaults to the highest-precedence candidate). Project file is `Project.toml` or `JuliaProject.toml`
2425
- **git.go**: Git integration (clone, push, fetch, pull) with JuliaHub authentication
2526
- **julia.go**: Julia installation and management
2627
- **run.go**: Julia execution with JuliaHub configuration
@@ -62,6 +63,7 @@ The application follows a command-line interface pattern using the Cobra library
6263
- `jh admin credential delete`: Delete a credential — subcommands: `token`, `ssh`, `github-app`; takes positional identifier
6364
- `jh admin landing-page`: Landing page management (show/update/remove custom markdown landing page with REST API)
6465
- `jh vuln`: Vulnerability scanning for Julia packages (REST API; defaults to latest stable version via `GET /docs/<registry>/<package>/versions.json`; supports `--version` for a specific version, `--registry` to specify the registry for version lookup (default: General), `--advisory` to filter to a specific advisory ID, `--all` to show all advisories regardless of affected status, and `--verbose` for additional details)
66+
- `jh scan`: Manifest vulnerability scanning via the StaticAnalysis service (REST API under `/api/v0/static_analysis/`); uploads a `Manifest.toml` (and sibling `Project.toml` unless `--no-project`) and POSTs to `/manifest/scan` to obtain a `run_uuid`; subcommands `status` and `results` hit `/manifest/scan/status/{run_uuid}` and `/results/manifest/{run_uuid}`; **polls by default** and prints results when the scan completes — Ctrl+C detaches and the `run_uuid` (printed up front) can be passed to `jh scan status` / `jh scan results` later. Pass `--no-wait` to submit-and-exit. `--csv` requests `text/csv` (content-negotiated via `Accept` header)
6567
- `jh clone`: Git clone with JuliaHub authentication and project name resolution
6668
- `jh push/fetch/pull`: Git operations with JuliaHub authentication
6769
- `jh git-credential`: Git credential helper for seamless authentication
@@ -229,6 +231,26 @@ go run . vuln MyPkg --registry MyRegistry
229231
go run . vuln SomePackage -s nightly.juliahub.dev
230232
```
231233

234+
### Test manifest scan operations
235+
```bash
236+
# Submit a scan for the current directory's Manifest.toml (+ sibling Project.toml);
237+
# polls until done by default. Ctrl+C detaches — the scan keeps running on the server.
238+
go run . scan
239+
240+
# Scan a project directory (still polls until done)
241+
go run . scan ./my-project
242+
243+
# Scan an explicit manifest, write CSV results to a file
244+
go run . scan Manifest.toml --project=Project.toml --csv --output results.csv
245+
246+
# Submit and return immediately without polling
247+
go run . scan --no-wait
248+
249+
# Check status / fetch results for an earlier run_uuid
250+
go run . scan status <run-uuid>
251+
go run . scan results <run-uuid> --csv
252+
```
253+
232254
### Test Git operations
233255
```bash
234256
go run . clone john/my-project # Clone from another user

README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ A command-line interface for interacting with JuliaHub, a platform for Julia com
1313
- **Julia Integration**: Install Julia and run with JuliaHub package server configuration
1414
- **User Management**: Display user information, list users and groups
1515
- **Vulnerability Scanning**: Scan Julia packages for known security vulnerabilities
16+
- **Manifest Scanning**: Upload a `Manifest.toml` for a server-side Trivy vulnerability scan of its pinned dependencies
1617
- **Administrative Commands**: Manage users, groups, tokens, credentials, and system resources (requires admin permissions)
1718

1819
## Installation
@@ -266,6 +267,24 @@ echo '{"name":"MyToken","url":"https://github.com","value":"ghp_xxxx"}' | jh adm
266267
- Advisory IDs are clickable links to the JuliaLang SecurityAdvisories repository
267268
- Each advisory shows: severity scores, affected status, full summary, affected versions, and version ranges
268269

270+
### Manifest Scanning (`jh scan`)
271+
272+
- `jh scan [path]` - Upload a `Manifest.toml` (and the sibling `Project.toml` when present), queue a Trivy vulnerability scan, and (by default) poll until it finishes
273+
- `path` may be a directory (looks for `Manifest.toml` and `Project.toml` inside) or a manifest file; defaults to the current directory
274+
- `--project <path>` - Use an explicit `Project.toml` instead of the auto-detected sibling
275+
- `--no-project` - Skip the `Project.toml` even if a sibling exists
276+
- `--tool <id>` - Static analysis tool id (default: `juliahub-trivy`)
277+
- `--no-wait` - Submit and return immediately, printing only the `run_uuid`
278+
- `--csv` - Request results as CSV (negotiated via the `Accept` header; ignored with `--no-wait`)
279+
- `--output <file>` / `-o` - Write the results to a file instead of stdout
280+
- `--poll-interval <sec>` - Poll cadence while waiting (default: 3)
281+
- `--timeout <sec>` - Max seconds to wait for the scan (default: 600)
282+
- The `run_uuid` is printed up front, so Ctrl+C during polling leaves the scan running on the server; resume with `jh scan status` / `jh scan results`
283+
- `jh scan status <run-uuid>` - Show lifecycle status (`staging`, `initiated`, `queued`, `completed`, `failed`) plus tool/timing info and any failure reason
284+
- `jh scan results <run-uuid>` - Fetch results for a finished scan
285+
- `--csv` - CSV output instead of JSON
286+
- `--output <file>` / `-o` - Write to a file instead of stdout
287+
269288
### Update (`jh update`)
270289

271290
- `jh update` - Check for updates and automatically install the latest version
@@ -483,6 +502,32 @@ jh vuln MyPkg --registry MyRegistry
483502
jh vuln SomePackage -s nightly.juliahub.dev
484503
```
485504

505+
### Manifest Scanning
506+
507+
```bash
508+
# Scan the Manifest.toml + Project.toml in the current directory; polls until done
509+
jh scan
510+
511+
# Scan the manifest from a specific project directory
512+
jh scan ./my-project
513+
514+
# Use an explicit manifest file and override the Project.toml path
515+
jh scan Manifest.toml --project=Project.toml
516+
517+
# Write CSV results to a file instead of JSON on stdout
518+
jh scan --csv --output results.csv
519+
520+
# Submit the scan and return immediately (prints just the run_uuid)
521+
jh scan --no-wait
522+
523+
# Check the status of an earlier scan (e.g. after Ctrl+C or --no-wait)
524+
jh scan status <run-uuid>
525+
526+
# Pull results for a finished scan
527+
jh scan results <run-uuid>
528+
jh scan results <run-uuid> --csv --output results.csv
529+
```
530+
486531
### Git Workflow
487532

488533
```bash

main.go

Lines changed: 170 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,11 @@ import (
44
"bufio"
55
"fmt"
66
"os"
7+
"os/signal"
78
"path/filepath"
89
"strings"
10+
"syscall"
11+
"time"
912

1013
"github.com/spf13/cobra"
1114
)
@@ -734,6 +737,158 @@ Use --advisory to look up a specific advisory by ID.`,
734737
},
735738
}
736739

740+
var scanCmd = &cobra.Command{
741+
Use: "scan [path]",
742+
Short: "Scan a Julia Manifest.toml for vulnerable dependencies",
743+
Long: `Upload a Julia Manifest.toml (and optionally Project.toml) to JuliaHub
744+
and run a Trivy vulnerability scan against the pinned dependencies.
745+
746+
If [path] is a directory, the manifest is discovered inside it. Every name
747+
Julia recognizes is detected — Manifest.toml, JuliaManifest.toml, and
748+
version-specific variants like Manifest-v1.11.toml — and if more than one is
749+
present you are prompted to pick one. A sibling project file (Project.toml or
750+
JuliaProject.toml) is included unless --no-project is set. If [path] is a file,
751+
it is treated as the manifest. Defaults to the current directory when [path]
752+
is omitted.
753+
754+
By default the command submits the scan and polls until it finishes, then
755+
prints the results. The server-assigned run_uuid is printed up front, so
756+
you can interrupt with Ctrl+C at any time and resume later with:
757+
758+
jh scan status <run-uuid>
759+
jh scan results <run-uuid>
760+
761+
Pass --no-wait to submit the scan and return immediately. Use --csv to
762+
fetch results as CSV instead of JSON.`,
763+
Example: ` jh scan
764+
jh scan ./my-project
765+
jh scan Manifest.toml --project=Project.toml
766+
jh scan --no-wait
767+
jh scan --csv --output results.csv`,
768+
Args: cobra.MaximumNArgs(1),
769+
Run: func(cmd *cobra.Command, args []string) {
770+
server, err := getServerFromFlagOrConfig(cmd)
771+
if err != nil {
772+
fmt.Printf("Failed to get server config: %v\n", err)
773+
os.Exit(1)
774+
}
775+
776+
path := ""
777+
if len(args) == 1 {
778+
path = args[0]
779+
}
780+
projectOverride, _ := cmd.Flags().GetString("project")
781+
noProject, _ := cmd.Flags().GetBool("no-project")
782+
tool, _ := cmd.Flags().GetString("tool")
783+
noWait, _ := cmd.Flags().GetBool("no-wait")
784+
csv, _ := cmd.Flags().GetBool("csv")
785+
output, _ := cmd.Flags().GetString("output")
786+
pollSec, _ := cmd.Flags().GetInt("poll-interval")
787+
timeoutSec, _ := cmd.Flags().GetInt("timeout")
788+
789+
inputs, err := resolveScanInputs(path, projectOverride, noProject)
790+
if err != nil {
791+
fmt.Fprintf(os.Stderr, "%v\n", err)
792+
os.Exit(1)
793+
}
794+
fmt.Fprintf(os.Stderr, "Uploading %s", inputs.ManifestPath)
795+
if inputs.ProjectPath != "" {
796+
fmt.Fprintf(os.Stderr, " (+ %s)", inputs.ProjectPath)
797+
}
798+
fmt.Fprintln(os.Stderr)
799+
800+
runUUID, err := submitManifestScan(server, tool, inputs)
801+
if err != nil {
802+
fmt.Fprintf(os.Stderr, "%v\n", err)
803+
os.Exit(1)
804+
}
805+
806+
fmt.Fprintf(os.Stderr, "Submitted scan: %s\n", runUUID)
807+
808+
if noWait {
809+
fmt.Println(runUUID)
810+
return
811+
}
812+
813+
// Trap Ctrl+C while polling so the user can leave the scan
814+
// running on the server and reattach later via `jh scan status`
815+
// / `jh scan results`.
816+
sigChan := make(chan os.Signal, 1)
817+
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
818+
go func() {
819+
<-sigChan
820+
fmt.Fprintf(os.Stderr, "\nLeaving scan %s running on the server.\n", runUUID)
821+
fmt.Fprintf(os.Stderr, "Check progress with:\n jh scan status %s\n", runUUID)
822+
fmt.Fprintf(os.Stderr, "Fetch results once complete with:\n jh scan results %s\n", runUUID)
823+
os.Exit(0)
824+
}()
825+
826+
fmt.Fprintf(os.Stderr, "Polling for completion (Ctrl+C to detach)...\n")
827+
_, err = pollManifestScanUntilDone(
828+
server, runUUID,
829+
time.Duration(pollSec)*time.Second,
830+
time.Duration(timeoutSec)*time.Second,
831+
)
832+
if err != nil {
833+
fmt.Fprintf(os.Stderr, "%v\n", err)
834+
os.Exit(1)
835+
}
836+
837+
body, err := fetchManifestScanResults(server, runUUID, csv)
838+
if err != nil {
839+
fmt.Fprintf(os.Stderr, "%v\n", err)
840+
os.Exit(1)
841+
}
842+
if err := writeResultsOutput(body, output, csv); err != nil {
843+
fmt.Fprintf(os.Stderr, "%v\n", err)
844+
os.Exit(1)
845+
}
846+
},
847+
}
848+
849+
var scanStatusCmd = &cobra.Command{
850+
Use: "status <run-uuid>",
851+
Short: "Show status of a manifest scan",
852+
Args: cobra.ExactArgs(1),
853+
Run: func(cmd *cobra.Command, args []string) {
854+
server, err := getServerFromFlagOrConfig(cmd)
855+
if err != nil {
856+
fmt.Printf("Failed to get server config: %v\n", err)
857+
os.Exit(1)
858+
}
859+
status, err := fetchManifestScanStatus(server, args[0])
860+
if err != nil {
861+
fmt.Fprintf(os.Stderr, "%v\n", err)
862+
os.Exit(1)
863+
}
864+
printManifestScanStatus(status)
865+
},
866+
}
867+
868+
var scanResultsCmd = &cobra.Command{
869+
Use: "results <run-uuid>",
870+
Short: "Fetch results for a completed manifest scan",
871+
Args: cobra.ExactArgs(1),
872+
Run: func(cmd *cobra.Command, args []string) {
873+
server, err := getServerFromFlagOrConfig(cmd)
874+
if err != nil {
875+
fmt.Printf("Failed to get server config: %v\n", err)
876+
os.Exit(1)
877+
}
878+
csv, _ := cmd.Flags().GetBool("csv")
879+
output, _ := cmd.Flags().GetString("output")
880+
body, err := fetchManifestScanResults(server, args[0], csv)
881+
if err != nil {
882+
fmt.Fprintf(os.Stderr, "%v\n", err)
883+
os.Exit(1)
884+
}
885+
if err := writeResultsOutput(body, output, csv); err != nil {
886+
fmt.Fprintf(os.Stderr, "%v\n", err)
887+
os.Exit(1)
888+
}
889+
},
890+
}
891+
737892
var packageCmd = &cobra.Command{
738893
Use: "package",
739894
Short: "Package search commands",
@@ -2328,6 +2483,19 @@ func init() {
23282483
vulnCmd.Flags().StringP("registry", "r", "General", "Registry name for version lookup")
23292484
vulnCmd.Flags().Bool("all", false, "Show all advisories regardless of affected status")
23302485
vulnCmd.Flags().BoolP("verbose", "v", false, "Show full advisory details (aliases, dates, details, references)")
2486+
scanCmd.Flags().StringP("server", "s", "juliahub.com", "JuliaHub server")
2487+
scanCmd.Flags().String("project", "", "Path to Project.toml (defaults to sibling of the manifest if present)")
2488+
scanCmd.Flags().Bool("no-project", false, "Skip Project.toml even if a sibling file exists")
2489+
scanCmd.Flags().StringP("tool", "t", "juliahub-trivy", "Static analysis tool id")
2490+
scanCmd.Flags().Bool("no-wait", false, "Submit the scan and return immediately, printing the run_uuid")
2491+
scanCmd.Flags().Bool("csv", false, "Fetch results as CSV instead of JSON (ignored with --no-wait)")
2492+
scanCmd.Flags().StringP("output", "o", "", "Write results to this file instead of stdout")
2493+
scanCmd.Flags().Int("poll-interval", 3, "Poll interval in seconds while waiting")
2494+
scanCmd.Flags().Int("timeout", 600, "Max seconds to wait for the scan to finish")
2495+
scanStatusCmd.Flags().StringP("server", "s", "juliahub.com", "JuliaHub server")
2496+
scanResultsCmd.Flags().StringP("server", "s", "juliahub.com", "JuliaHub server")
2497+
scanResultsCmd.Flags().Bool("csv", false, "Fetch results as CSV")
2498+
scanResultsCmd.Flags().StringP("output", "o", "", "Write results to this file instead of stdout")
23312499

23322500
authCmd.AddCommand(authLoginCmd, authRefreshCmd, authStatusCmd, authEnvCmd)
23332501
jobCmd.AddCommand(jobListCmd, jobStartCmd)
@@ -2371,8 +2539,9 @@ func init() {
23712539
juliaCmd.AddCommand(juliaInstallCmd)
23722540
runCmd.AddCommand(runSetupCmd)
23732541
gitCredentialCmd.AddCommand(gitCredentialHelperCmd, gitCredentialGetCmd, gitCredentialStoreCmd, gitCredentialEraseCmd, gitCredentialSetupCmd)
2542+
scanCmd.AddCommand(scanStatusCmd, scanResultsCmd)
23742543

2375-
rootCmd.AddCommand(authCmd, jobCmd, datasetCmd, projectCmd, packageCmd, registryCmd, userCmd, groupCmd, adminCmd, juliaCmd, cloneCmd, pushCmd, fetchCmd, pullCmd, runCmd, gitCredentialCmd, updateCmd, vulnCmd)
2544+
rootCmd.AddCommand(authCmd, jobCmd, datasetCmd, projectCmd, packageCmd, registryCmd, userCmd, groupCmd, adminCmd, juliaCmd, cloneCmd, pushCmd, fetchCmd, pullCmd, runCmd, gitCredentialCmd, updateCmd, vulnCmd, scanCmd)
23762545
}
23772546

23782547
func main() {

0 commit comments

Comments
 (0)