Skip to content

Commit 1baa6a8

Browse files
a-orenclaude
a-oren
and
claude
authored
feat: add JS workspace discovery and public batch API (guacsec#415)
## Summary - Add `JsWorkspaceDiscovery` utility for parsing `pnpm-workspace.yaml` and `package.json` workspaces field to discover member `package.json` manifests - Add parent-traversal lock file discovery to `JavaScriptProviderFactory` so workspace members find the lock file at the workspace root - Add `stackAnalysisBatch()` and `stackAnalysisBatchHtml()` to `Api` interface and `ExhortApi` implementation - Add `jackson-dataformat-yaml` dependency for YAML parsing - Add comprehensive test suites for workspace discovery and lock file walk-up ## Details `JsWorkspaceDiscovery` is a static utility class that discovers workspace member packages by parsing `pnpm-workspace.yaml` (packages array) or `package.json` (workspaces array/object format). It globs for `package.json` files, filters by ignore patterns, and validates each manifest has `name` and `version` fields. `JavaScriptProviderFactory.create()` now walks up parent directories when no lock file is found in the manifest directory. This enables workspace member packages to find the single lock file at the workspace root. The walk-up stops at workspace root boundaries or git root, and supports `TRUSTIFY_DA_WORKSPACE_DIR` override. The batch API methods delegate to the existing `performBatchAnalysis()` infrastructure, following the same pattern as `imageAnalysis()`. Config (`TRUSTIFY_DA_CONTINUE_ON_ERROR`) is read from environment internally. Implements [TC-3863](https://redhat.atlassian.net/browse/TC-3863) ## Test plan - [x] pnpm-workspace.yaml parsing discovers member packages - [x] package.json workspaces array and object formats parsed correctly - [x] Invalid manifests (missing name/version) are skipped - [x] Ignore patterns filter out matching paths - [x] Workspace root detection works for both pnpm and npm/yarn - [x] Lock file found at workspace root via parent walk-up (npm, yarn, pnpm) - [x] Walk-up stops at workspace root boundary - [x] TRUSTIFY_DA_WORKSPACE_DIR override works - [x] No lock file anywhere throws descriptive error - [x] Existing tests unaffected (19 new tests, 0 regressions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) [TC-3863]: https://redhat.atlassian.net/browse/TC-3863?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c7f761d commit 1baa6a8

23 files changed

Lines changed: 1031 additions & 1 deletion

File tree

‎pom.xml‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,11 @@
151151
<artifactId>jackson-datatype-jsr310</artifactId>
152152
<version>${jackson.version}</version>
153153
</dependency>
154+
<dependency>
155+
<groupId>com.fasterxml.jackson.dataformat</groupId>
156+
<artifactId>jackson-dataformat-yaml</artifactId>
157+
<version>${jackson.version}</version>
158+
</dependency>
154159
<dependency>
155160
<groupId>jakarta.annotation</groupId>
156161
<artifactId>jakarta.annotation-api</artifactId>
@@ -228,7 +233,10 @@
228233
<groupId>com.fasterxml.jackson.core</groupId>
229234
<artifactId>jackson-databind</artifactId>
230235
</dependency>
231-
<!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.datatype/jackson-datatype-jsr310 -->
236+
<dependency>
237+
<groupId>com.fasterxml.jackson.dataformat</groupId>
238+
<artifactId>jackson-dataformat-yaml</artifactId>
239+
</dependency>
232240

233241
<dependency>
234242
<groupId>jakarta.annotation</groupId>

‎src/main/java/io/github/guacsec/trustifyda/Api.java‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
import io.github.guacsec.trustifyda.api.v5.AnalysisReport;
2020
import io.github.guacsec.trustifyda.image.ImageRef;
2121
import java.io.IOException;
22+
import java.nio.file.Path;
2223
import java.util.Arrays;
2324
import java.util.Map;
2425
import java.util.Objects;
@@ -134,4 +135,29 @@ CompletableFuture<Map<ImageRef, AnalysisReport>> imageAnalysis(Set<ImageRef> ima
134135
* @throws IllegalStateException when the manifest file type is not supported
135136
*/
136137
String generateSbom(String manifestFile) throws IOException;
138+
139+
/**
140+
* Performs batch stack analysis for all workspace members discovered in the given workspace
141+
* directory. Discovers package.json manifests via pnpm-workspace.yaml or package.json workspaces
142+
* field, generates SBOMs for each, and sends them as a batch to the backend.
143+
*
144+
* @param workspaceDir the workspace root directory path
145+
* @param ignorePatterns glob patterns for paths to exclude from workspace discovery
146+
* @return a map of manifest name to analysis report, wrapped in a CompletableFuture
147+
* @throws IOException when workspace discovery or SBOM generation fails
148+
*/
149+
CompletableFuture<Map<String, AnalysisReport>> stackAnalysisBatch(
150+
Path workspaceDir, Set<String> ignorePatterns) throws IOException;
151+
152+
/**
153+
* Performs batch stack analysis returning an HTML report for all workspace members discovered in
154+
* the given workspace directory.
155+
*
156+
* @param workspaceDir the workspace root directory path
157+
* @param ignorePatterns glob patterns for paths to exclude from workspace discovery
158+
* @return the HTML report as a byte array wrapped in a CompletableFuture
159+
* @throws IOException when workspace discovery or SBOM generation fails
160+
*/
161+
CompletableFuture<byte[]> stackAnalysisBatchHtml(Path workspaceDir, Set<String> ignorePatterns)
162+
throws IOException;
137163
}

‎src/main/java/io/github/guacsec/trustifyda/impl/ExhortApi.java‎

Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,12 @@
3030
import io.github.guacsec.trustifyda.image.ImageUtils;
3131
import io.github.guacsec.trustifyda.license.LicenseCheck;
3232
import io.github.guacsec.trustifyda.logging.LoggersFactory;
33+
import io.github.guacsec.trustifyda.providers.javascript.workspace.JsWorkspaceDiscovery;
34+
import io.github.guacsec.trustifyda.providers.rust.model.CargoMetadata;
3335
import io.github.guacsec.trustifyda.tools.Ecosystem;
36+
import io.github.guacsec.trustifyda.tools.Operations;
3437
import io.github.guacsec.trustifyda.utils.Environment;
38+
import io.github.guacsec.trustifyda.utils.WorkspaceUtils;
3539
import jakarta.mail.MessagingException;
3640
import jakarta.mail.internet.MimeMultipart;
3741
import jakarta.mail.util.ByteArrayDataSource;
@@ -50,7 +54,9 @@
5054
import java.time.LocalDateTime;
5155
import java.time.temporal.ChronoUnit;
5256
import java.util.AbstractMap;
57+
import java.util.ArrayList;
5358
import java.util.Collections;
59+
import java.util.List;
5460
import java.util.Map;
5561
import java.util.Objects;
5662
import java.util.Optional;
@@ -703,6 +709,242 @@ public CompletableFuture<String> identifyLicense(Path licenseFilePath) {
703709
});
704710
}
705711

712+
@Override
713+
public CompletableFuture<Map<String, AnalysisReport>> stackAnalysisBatch(
714+
final Path workspaceDir, final Set<String> ignorePatterns) throws IOException {
715+
return this.performBatchAnalysis(
716+
() -> getBatchStackSboms(workspaceDir, ignorePatterns),
717+
MediaType.APPLICATION_JSON,
718+
HttpResponse.BodyHandlers.ofString(),
719+
this::getBatchStackAnalysisReports,
720+
Collections::emptyMap,
721+
"Batch Stack Analysis");
722+
}
723+
724+
@Override
725+
public CompletableFuture<byte[]> stackAnalysisBatchHtml(
726+
final Path workspaceDir, final Set<String> ignorePatterns) throws IOException {
727+
return this.performBatchAnalysis(
728+
() -> getBatchStackSboms(workspaceDir, ignorePatterns),
729+
MediaType.TEXT_HTML,
730+
HttpResponse.BodyHandlers.ofByteArray(),
731+
HttpResponse::body,
732+
() -> new byte[0],
733+
"Batch Stack Analysis");
734+
}
735+
736+
Map<String, JsonNode> getBatchStackSboms(
737+
final Path workspaceDir, final Set<String> ignorePatterns) {
738+
boolean continueOnError = Environment.getBoolean("TRUSTIFY_DA_CONTINUE_ON_ERROR", true);
739+
int concurrency = resolveBatchConcurrency();
740+
try {
741+
Set<String> resolved = resolveIgnorePatterns(ignorePatterns);
742+
List<Path> manifests = discoverWorkspaceManifests(workspaceDir, resolved);
743+
if (manifests.isEmpty()) {
744+
LOG.warning("No workspace members discovered in " + workspaceDir);
745+
return Collections.emptyMap();
746+
}
747+
748+
var executor = java.util.concurrent.Executors.newFixedThreadPool(concurrency);
749+
try {
750+
var futures =
751+
manifests.stream()
752+
.map(
753+
manifest ->
754+
java.util.concurrent.CompletableFuture.supplyAsync(
755+
() -> {
756+
try {
757+
var provider = Ecosystem.getProvider(manifest);
758+
var content = provider.provideStack();
759+
var sbomJson = mapper.readTree(content.buffer);
760+
var purl =
761+
sbomJson
762+
.at("/metadata/component/purl")
763+
.asText(
764+
sbomJson
765+
.at("/metadata/component/bom-ref")
766+
.asText(null));
767+
if (purl == null || purl.isBlank()) {
768+
throw new IllegalStateException(
769+
"Missing purl in SBOM metadata.component for " + manifest);
770+
}
771+
return new AbstractMap.SimpleEntry<>(purl, sbomJson);
772+
} catch (Exception ex) {
773+
if (continueOnError) {
774+
LOG.warning(
775+
String.format(
776+
"Skipping manifest %s due to error: %s",
777+
manifest, ex.getMessage()));
778+
return null;
779+
}
780+
throw new RuntimeException(
781+
"Failed to generate SBOM for " + manifest, ex);
782+
}
783+
},
784+
executor))
785+
.toList();
786+
787+
var results =
788+
futures.stream()
789+
.map(java.util.concurrent.CompletableFuture::join)
790+
.filter(Objects::nonNull)
791+
.collect(
792+
Collectors.toMap(
793+
AbstractMap.SimpleEntry::getKey, AbstractMap.SimpleEntry::getValue));
794+
795+
if (Environment.getBoolean("TRUSTIFY_DA_BATCH_METADATA", false)) {
796+
int failed = manifests.size() - results.size();
797+
LOG.info(
798+
String.format(
799+
"Batch metadata: workspaceRoot=%s, total=%d, successful=%d, failed=%d",
800+
workspaceDir, manifests.size(), results.size(), failed));
801+
}
802+
803+
return results;
804+
} finally {
805+
executor.shutdown();
806+
}
807+
} catch (IOException e) {
808+
throw new RuntimeException("Failed to discover workspace manifests", e);
809+
}
810+
}
811+
812+
Map<String, AnalysisReport> getBatchStackAnalysisReports(
813+
final HttpResponse<String> httpResponse) {
814+
if (httpResponse.statusCode() == 200) {
815+
try {
816+
Map<?, ?> reports = this.mapper.readValue(httpResponse.body(), Map.class);
817+
return reports.entrySet().stream()
818+
.collect(
819+
Collectors.toMap(
820+
e -> e.getKey().toString(),
821+
e -> mapper.convertValue(e.getValue(), AnalysisReport.class)));
822+
} catch (JsonProcessingException e) {
823+
throw new CompletionException(e);
824+
}
825+
} else {
826+
return Collections.emptyMap();
827+
}
828+
}
829+
830+
/** Resolves batch concurrency from TRUSTIFY_DA_BATCH_CONCURRENCY. default 10, max 256. */
831+
int resolveBatchConcurrency() {
832+
String raw = Environment.get("TRUSTIFY_DA_BATCH_CONCURRENCY", "10");
833+
try {
834+
int n = Integer.parseInt(raw.trim());
835+
if (n < 1) {
836+
return 10;
837+
}
838+
return Math.min(256, n);
839+
} catch (NumberFormatException e) {
840+
return 10;
841+
}
842+
}
843+
844+
private static final Set<String> DEFAULT_WORKSPACE_DISCOVERY_IGNORE =
845+
Set.of("**/node_modules/**", "**/.git/**");
846+
847+
/** Merges default ignore patterns, env var overrides, and caller-provided patterns. */
848+
Set<String> resolveIgnorePatterns(Set<String> callerPatterns) {
849+
var merged = new java.util.LinkedHashSet<>(DEFAULT_WORKSPACE_DISCOVERY_IGNORE);
850+
String fromEnv = Environment.get("TRUSTIFY_DA_WORKSPACE_DISCOVERY_IGNORE", null);
851+
if (fromEnv != null && !fromEnv.isBlank()) {
852+
for (String p : fromEnv.split(",")) {
853+
String trimmed = p.trim();
854+
if (!trimmed.isEmpty()) {
855+
merged.add(trimmed);
856+
}
857+
}
858+
}
859+
if (callerPatterns != null) {
860+
merged.addAll(callerPatterns);
861+
}
862+
return merged;
863+
}
864+
865+
/**
866+
* Detects the workspace ecosystem and discovers manifest paths. Checks for Cargo workspace first
867+
* (Cargo.toml + Cargo.lock), then falls back to JS workspace discovery.
868+
*/
869+
List<Path> discoverWorkspaceManifests(Path workspaceDir, Set<String> ignorePatterns)
870+
throws IOException {
871+
// Cargo workspace: Cargo.toml + Cargo.lock
872+
Path cargoToml = workspaceDir.resolve("Cargo.toml");
873+
Path cargoLock = workspaceDir.resolve("Cargo.lock");
874+
if (Files.isRegularFile(cargoToml) && Files.isRegularFile(cargoLock)) {
875+
return discoverCargoManifests(workspaceDir, ignorePatterns);
876+
}
877+
878+
// JS workspace: require package.json + a lock file
879+
Path packageJson = workspaceDir.resolve("package.json");
880+
boolean hasJsLock =
881+
Files.isRegularFile(workspaceDir.resolve("pnpm-lock.yaml"))
882+
|| Files.isRegularFile(workspaceDir.resolve("yarn.lock"))
883+
|| Files.isRegularFile(workspaceDir.resolve("package-lock.json"));
884+
885+
if (Files.isRegularFile(packageJson) && hasJsLock) {
886+
List<Path> manifests =
887+
JsWorkspaceDiscovery.discoverWorkspaceManifests(workspaceDir, ignorePatterns);
888+
if (manifests.isEmpty()) {
889+
return List.of(packageJson);
890+
}
891+
// Include root package.json if it is not private and not already discovered
892+
if (!manifests.contains(packageJson) && !isPrivatePackageJson(packageJson)) {
893+
var withRoot = new ArrayList<>(manifests);
894+
withRoot.addFirst(packageJson);
895+
manifests = withRoot;
896+
}
897+
return manifests;
898+
}
899+
900+
return Collections.emptyList();
901+
}
902+
903+
private List<Path> discoverCargoManifests(Path workspaceDir, Set<String> ignorePatterns) {
904+
try {
905+
String cargo = Operations.getCustomPathOrElse("cargo");
906+
Operations.ProcessExecOutput output =
907+
Operations.runProcessGetFullOutput(
908+
workspaceDir,
909+
new String[] {cargo, "metadata", "--format-version", "1", "--no-deps"},
910+
null);
911+
if (output.getExitCode() != 0) {
912+
LOG.warning("cargo metadata failed with exit code " + output.getExitCode());
913+
return Collections.emptyList();
914+
}
915+
CargoMetadata metadata = mapper.readValue(output.getOutput(), CargoMetadata.class);
916+
var memberIds = new java.util.HashSet<String>(metadata.workspaceMembers());
917+
List<Path> manifests = new ArrayList<>();
918+
for (var pkg : metadata.packages()) {
919+
if (memberIds.contains(pkg.id()) && pkg.manifestPath() != null) {
920+
Path manifestPath = Path.of(pkg.manifestPath());
921+
if (Files.isRegularFile(manifestPath)) {
922+
manifests.add(manifestPath);
923+
}
924+
}
925+
}
926+
return WorkspaceUtils.filterByIgnorePatterns(workspaceDir, manifests, ignorePatterns);
927+
} catch (Exception e) {
928+
LOG.warning("Failed to discover Cargo workspace manifests: " + e.getMessage());
929+
return Collections.emptyList();
930+
}
931+
}
932+
933+
/**
934+
* Checks whether a package.json has "private": true, meaning it should not be analyzed as a
935+
* publishable package.
936+
*/
937+
private boolean isPrivatePackageJson(Path packageJson) {
938+
try {
939+
JsonNode root = mapper.readTree(Files.newInputStream(packageJson));
940+
JsonNode privateField = root.get("private");
941+
return privateField != null && privateField.asBoolean(false);
942+
} catch (IOException e) {
943+
LOG.warning("Failed to read " + packageJson + ": " + e.getMessage());
944+
return true;
945+
}
946+
}
947+
706948
/**
707949
* Build an HTTP request for sending to the Backend API.
708950
*

0 commit comments

Comments
 (0)