Skip to content
Merged

Pnpm 9 #1561

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
f91cea6
Basic support for pnpm 9 lockfiles.
csasarak Jan 4, 2025
d78d81b
fix nonexhaustive patterns.
csasarak Jan 4, 2025
f3df987
fix: improve pnpm v9 lockfile parsing
ryanlink Jan 31, 2025
34e0d2b
Merge branch 'master' into pnpm-9
ryanlink Jan 31, 2025
242bd49
[ANE-2235] Add support for PNPM v9 catalog versions
ryanlink Jan 31, 2025
0b9b001
[ANE-2235] Fix dev dependency handling in PNPM v9 lockfiles
ryanlink Jan 31, 2025
d83cee3
[ANE-2235] Fix name collision with version field
ryanlink Jan 31, 2025
5676798
[ANE-2235] Fix CatalogMap parsing
ryanlink Feb 2, 2025
0b2083c
[ANE-2235] Fix CatalogMap parsing
ryanlink Feb 3, 2025
18c7b71
[ANE-2235] Fix CatalogMap parsing type error
ryanlink Feb 3, 2025
e2b2fcf
[ANE-2235] Revert commits to restore pnpm-9 branch to d78d81b
ryanlink Feb 26, 2025
b45494e
Merge remote-tracking branch 'origin/master' into pnpm-9
csasarak Jun 24, 2025
7e8151f
Process pnpm lockfile v9 workspaces
csasarak Jul 2, 2025
da97982
Read dependency graph from lockfile snapshots where necessary.
csasarak Jul 7, 2025
fd41636
Merge remote-tracking branch 'origin/master' into pnpm-9
csasarak Jul 7, 2025
ac3b14a
Fix snapshots parsing.
csasarak Jul 7, 2025
7ce5e07
Glob negation.
csasarak Jul 8, 2025
096163c
Fix snapshots parsing again
csasarak Jul 8, 2025
4f7d3e7
Dev deps for pnpm 9 projects.
csasarak Jul 8, 2025
c0eb7a0
Tests for pnpm9 lockfile and workspace project.
csasarak Jul 8, 2025
164df47
Fix compilation error.
csasarak Jul 8, 2025
081d3c1
Update docs and changelog
csasarak Jul 8, 2025
e5abbfd
Merge remote-tracking branch 'origin/master' into pnpm-9
csasarak Jul 8, 2025
cadb223
Apply suggestions from code review
csasarak Jul 8, 2025
f1339af
Address PR feedback.
csasarak Jul 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion Changelog.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
# FOSSA CLI Changelog

## 3.10.12

- PNPM: Initial support for lockfile version 9.0 ([#1561](https://github.com/fossas/fossa-cli/pull/1561))

## 3.10.11

- container scanning: fix unzipping JARs that symlink to other layers #1555 ([#1555](https://github.com/fossas/fossa-cli/pull/1555))

## 3.10.10

- go: support the `tool` directive introduced in go Feb 2025 ([#1553](https://github.com/fossas/fossa-cli/pull/1553))
Expand Down
29 changes: 11 additions & 18 deletions docs/references/strategies/languages/nodejs/pnpm.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Pnpm

[Pnpm](https://pnpm.io/) is a fast, disk space-efficient package manager.
[Pnpm](https://pnpm.io/) is a fast, disk space-efficient package manager.
Unlike npm and yarn, pnpm uses symbolic links to create a nested structure
of dependencies.

Expand All @@ -20,8 +20,8 @@ in `pnpm-lock.yaml` to analyze the dependency graph.

- `packages`
- `[packagesKey]`
- `resolution`: infer git URL, git commit, or package source URL.
- `dependencies`: list of transitive dependencies
- `resolution`: infer git URL, git commit, or package source URL.
- `dependencies`: list of transitive dependencies
- `peerDependencies`: list of peer dependencies (will be treated like any other dependency)
- `dev`: to infer if this is used dependency or not. If the value is `true` by default CLI will not include this in the final analysis.

Expand All @@ -35,15 +35,15 @@ importers:
specifiers:
some-pkg: https://some-url/pkg.tar.gz
react: '*'
my-local-pkg: file:../libs/my-local-pkg
my-local-pkg: file:../libs/my-local-pkg
dependencies:
some-pkg: '@some-url/pkg.tar.gz'
my-local-pkg: file:../libs/my-local-pkg
devDependencies:
react: 18.1.0

# workspace project in packages/some-ws-pkg directory from root.
packages/some-ws-pkg:
packages/some-ws-pkg:
specifiers:
commander: 9.2.0
dependencies:
Expand Down Expand Up @@ -104,7 +104,7 @@ FOSSA will use provided `repo` and `commit` attribute to analyze this dependency
dev: false
```

* If the dependency was resolved using tarball (`resolution` will have `tarball` attribute)
* If the dependency was resolved using tarball (`resolution` will have `tarball` attribute)
FOSSA will use provided URL address to download and analyze this dependency.

```yaml
Expand All @@ -118,11 +118,11 @@ FOSSA will use provided URL address to download and analyze this dependency.
```

* If the dependency was resolved using the local directory (`resolution` will have the `type: directory` attribute),
FOSSA will not analyze this dependency. Local dependency's transitive dependencies will be analyzed,
and they will be promoted in place of local dependency.
FOSSA will not analyze this dependency. Local dependency's transitive dependencies will be analyzed,
and they will be promoted in place of local dependency.

```yaml
# FOSSA will not analyze this dependency,
# FOSSA will not analyze this dependency,
# But FOSSA will analyze its transitive dependency (if they are not sourced from the local directory)
#
# FOSSA will promote loose-envify of 1.4.0 in place of unifier.
Expand All @@ -136,7 +136,7 @@ and they will be promoted in place of local dependency.
dev: false
```

* If the dependency was resolved using registry resolver, FOSSA will use the registry to analyze the dependency.
* If the dependency was resolved using registry resolver, FOSSA will use the registry to analyze the dependency.
CLI will infer the package name and version using `/${dependencyName}/${dependencyVersion}` scheme from the package's key.

```yaml
Expand All @@ -151,7 +151,7 @@ CLI will infer the package name and version using `/${dependencyName}/${dependen
* Pnpm workspaces are supported.
* Development dependencies (`dev: true`) are ignored by default from analysis. To include them in the analysis, execute CLI with `--include-unused` flag e.g. `fossa analyze --include-unused`.
* Optional dependencies are included in the analysis by default. They can be ignored in FOSSA UI.
* `fossa-cli` supports lockFileVersion: 4.x, 5.x, and 6.x.
* `fossa-cli` supports lockFileVersion: 4.x, 5.x, 6.x, 7.x, 8.x, and 9.x.


# F.A.Q
Expand All @@ -168,10 +168,3 @@ targets:
only:
- type: pnpm
```
### Are all versions of `pnpm` supported?

At this time, the latest version of pnpm (v9) and its associated v9 lockfiles are not correctly parsed by FOSSA. Please revert to v8 (v6 lockfile) if your dependencies are not resolved in the FOSSA UI: "FOSSA was unable to analyze this dependency. If it is behind a private registry or auth you may need to configure FOSSA's access, then rebuild this dependency." This is due to the version number being appended to the package name:

<img width="796" alt="image" src="https://github.com/user-attachments/assets/d1461506-d3e7-42da-b9be-2b53a87f79f1" />

Please [email](mailto:support@fossa.com) FOSSA support if you are affected by this limitation.
1 change: 1 addition & 0 deletions spectrometer.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -484,6 +484,7 @@ library
Strategy.Node.Npm.PackageLockV3
Strategy.Node.PackageJson
Strategy.Node.Pnpm.PnpmLock
Strategy.Node.Pnpm.Workspace
Strategy.Node.YarnV1.YarnLock
Strategy.Node.YarnV2.Lockfile
Strategy.Node.YarnV2.Resolvers
Expand Down
30 changes: 22 additions & 8 deletions src/Strategy/Node.hs
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ import Data.Map.Strict qualified as Map
import Data.Maybe (catMaybes, isJust, mapMaybe)
import Data.Set (Set)
import Data.Set qualified as Set
import Data.String.Conversion (decodeUtf8)
import Data.String.Conversion (decodeUtf8, toString)
import Data.Tagged (applyTag)
import Data.Text (Text)
import Data.Text qualified as Text
Expand All @@ -59,6 +59,7 @@ import Effect.ReadFS (
doesFileExist,
readContentsBSLimit,
readContentsJson,
readContentsYaml,
)
import GHC.Generics (Generic)
import Path (
Expand All @@ -84,13 +85,14 @@ import Strategy.Node.PackageJson (
PkgJsonGraph (..),
PkgJsonLicense (LicenseObj, LicenseText),
PkgJsonLicenseObj (licenseUrl),
PkgJsonWorkspaces (unWorkspaces),
PkgJsonWorkspaces (PkgJsonWorkspaces, unWorkspaces),
Production,
WorkspacePackageNames (WorkspacePackageNames),
pkgFileList,
)
import Strategy.Node.PackageJson qualified as PackageJson
import Strategy.Node.Pnpm.PnpmLock qualified as PnpmLock
import Strategy.Node.Pnpm.Workspace (PnpmWorkspace (workspaceSpecs))
import Strategy.Node.YarnV1.YarnLock qualified as V1
import Strategy.Node.YarnV2.YarnLock qualified as V2
import Types (
Expand Down Expand Up @@ -118,8 +120,8 @@ discover ::
m [DiscoveredProject NodeProject]
discover dir = withMultiToolFilter [YarnProjectType, NpmProjectType, PnpmProjectType] $
context "NodeJS" $ do
manifestList <- context "Finding nodejs projects" $ collectManifests dir
manifestMap <- context "Reading package.json files" $ (Map.fromList . catMaybes) <$> traverse loadPackage manifestList
manifestList <- context "Finding nodejs/pnpm projects" $ collectManifests dir
manifestMap <- context "Reading manifest files" $ (Map.fromList . catMaybes) <$> traverse loadPackage manifestList
if Map.null manifestMap
then -- If the map is empty, we found no JS projects, we return early.
pure []
Expand Down Expand Up @@ -281,9 +283,14 @@ extractDepLists PkgJsonGraph{..} = foldMap extractSingle $ Map.elems jsonLookup
loadPackage :: (Has Logger sig m, Has ReadFS sig m, Has Diagnostics sig m) => Manifest -> m (Maybe (Manifest, PackageJson))
loadPackage (Manifest file) = do
result <- recover $ readContentsJson @PackageJson file
case result of
Nothing -> pure Nothing
Just contents -> pure $ Just (Manifest file, contents)
-- PNPM projects using v9 of the lockfile have their own way to specify workspaces/child projects.
-- Since there is still a package.json, inject the pnpm-workspace.yaml projects into the ones for that manifest.
let possiblePnpmWorkspaceFile = Path.parent file </> $(mkRelFile "pnpm-workspace.yaml")
pnpmResult <- recover $ readContentsYaml @PnpmWorkspace possiblePnpmWorkspaceFile
pure $ do
contents@PackageJson{packageWorkspaces} <- result
let pnpmGlobs = maybe [] workspaceSpecs pnpmResult
Just (Manifest file, contents{packageWorkspaces = PkgJsonWorkspaces pnpmGlobs <> packageWorkspaces})

buildManifestGraph :: Map Manifest PackageJson -> PkgJsonGraph
buildManifestGraph manifestMap = PkgJsonGraph adjmap manifestMap
Expand Down Expand Up @@ -317,7 +324,14 @@ buildManifestGraph manifestMap = PkgJsonGraph adjmap manifestMap

-- True if qualified glob pattern matches the given file.
filterfunc :: Manifest -> Glob Rel -> Manifest -> Bool
filterfunc root glob (Manifest candidate) = candidate `Glob.matches` qualifyGlobPattern root glob
filterfunc root glob (Manifest candidate) = candidate `globMatches` qualifyGlobPattern root glob

-- PNPM workspaces can negate their globs with '!'.
globMatches :: Path Abs File -> Glob Abs -> Bool
globMatches p g = case toString g of
-- labeled unsafe, but we already had a glob before putting it back together again
('!' : rest) -> not $ p `Glob.matches` (Glob.unsafeGlobAbs rest)
_ -> p `Glob.matches` g

-- Yarn appends the filename to the glob, so we match that behavior
-- https://github.com/yarnpkg/yarn/blob/master/src/config.js#L821
Expand Down
2 changes: 1 addition & 1 deletion src/Strategy/Node/PackageJson.hs
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ buildGraph PackageJson{..} = run . withLabeling toDependency $ do
}

newtype PkgJsonWorkspaces = PkgJsonWorkspaces {unWorkspaces :: [Glob Rel]}
deriving (Eq, Ord, Show, ToJSON)
deriving (Eq, Ord, Show, ToJSON, Semigroup, Monoid)

-- Name and version are required for workspace sub-projects.
data PackageJson = PackageJson
Expand Down
Loading
Loading