Skip to content

Add Yarn PnP support#1966

Open
GGomez99 wants to merge 48 commits into
microsoft:mainfrom
GGomez99:guyllian.gomez/yarn-pnp
Open

Add Yarn PnP support#1966
GGomez99 wants to merge 48 commits into
microsoft:mainfrom
GGomez99:guyllian.gomez/yarn-pnp

Conversation

@GGomez99

@GGomez99 GGomez99 commented Oct 28, 2025

Copy link
Copy Markdown

Closes #460

Co-authored with @gun-yu as a result of their changes (#1876) being merged in this PR

Motivation

This PR adds Plug'n'Play support natively to Typescript Go, following this issue: #460
It has been reviewed and supported by @arcanis, the lead maintainer of Yarn, and the original author of Yarn PnP.

Datadog has a frontend monorepo using yarn with over 6k packages, and seeing how TS Strada struggles with our current scaling, we decided to invest time in adding a native Yarn PnP support for Typescript Go.
This PnP implementation has been actively used in the IDE of more than 230 engineers at Datadog, and we're committed to fixing all issues reported to us.

Challenges

We did not integrate it in our CI yet as we still have several packages failing on build mode (most errors seem to be reported in the issues section of TS Corsa). Because the TS Corsa API is not available yet, we also couldn't integrate it properly with a fast lage setup unlike with the TS Strada API.

Changes

It's based on the main changes from the original yarn patch (microsoft/TypeScript@99f3e13) that the community has been maintaining for years throughout Typescript Strada updates, except that we implemented the official PnP specification so it doesn't depend on third-party code.

Implemented features:

  • PnP VFS that handles virtual folders and zip files seamlessly, with caching and fallback to the original vfs if pnp is not available
  • Add PnP API and manifest handling, following the yarn PnP specification
  • Initialize the PnP API every time a Host is initialized for both build and LSP modes
  • Add PnP support when resolving modules in internal/module/resolver.go
  • Add PnP support for auto-imports and completion at internal/modulespecifiers/specifiers.go
  • Add PnP support for root types at internal/core/compileroptions.go
  • Handle zip paths when going to implementation with the LSP
  • Update the baseline testing framework to handle PnP when needed

Missing features:

  • PnP manifest auto-refresh by watching .pnp.cjs changes

Tests

  • Basic PnP setup
  • Types from transitive dependencies
  • Root types loading from PnP dependencies
  • Completion and autoimports

@GGomez99

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Datadog"

@GGomez99

Copy link
Copy Markdown
Author

I see my tests are not passing in CI, looking into it 🙇

Comment thread internal/pnp/pnp.go Outdated
Comment on lines +12 to +30
var (
isPnpApiInitialized atomic.Uint32
cachedPnpApi *PnpApi
pnpMu sync.Mutex
// testPnpCache stores per-goroutine PnP APIs for test isolation
// Key is goroutine ID (as int)
testPnpCache sync.Map // map[int]*PnpApi
)

// getGoroutineID returns the current goroutine ID
// It is usually not recommended to work with goroutine IDs, but it is the most non-intrusive way to setup a parallel testing environment for PnP API
func getGoroutineID() int {
var buf [64]byte
n := runtime.Stack(buf[:], false)
idField := strings.Fields(strings.TrimPrefix(string(buf[:n]), "goroutine "))[0]
id, _ := strconv.Atoi(idField)
return id
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of this global / goroutine local storage is something that won't work. If we are storing something, it needs to be attached to a Program, Project, a Host, etc, not global. Parsing out the goroutine ID is definitely a bad idea, especially given our LSP can handle multiple requests at the same time from multiple goroutines and so on.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jakebailey Thank you for having a first look at this PR!
Since your comments, we changed how we initialize the PnpApi by attaching it to the Host directly on initialization. The host can then provide it through host.PnpApi() which will return nil or the pnpapi instance if in a yarn project

Comment thread internal/pnp/pnp.go

pnpApi := &PnpApi{fs: fs, url: filePath}

manifestData, err := pnpApi.findClosestPnpManifest()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand this init. Surely one needs to be able to load multiple projects with differing pnp info at the same time? All of this info really does need to be handled differently.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the last changes mentioned in this comment, we initialize the PnP api with the Host. This means we assume that we have one yarn PnP project per host

I don't think we could initialize it on a smaller scope (like Project or Program) from looking at the changes we had to do to support PnP, but let me know what you think!

@gun-yu gun-yu mentioned this pull request Oct 29, 2025
@GGomez99 GGomez99 force-pushed the guyllian.gomez/yarn-pnp branch from d70cc8a to 854be04 Compare October 29, 2025 14:07
@GGomez99 GGomez99 force-pushed the guyllian.gomez/yarn-pnp branch 2 times, most recently from e92efeb to e4a3c4b Compare October 31, 2025 12:23
@GGomez99 GGomez99 force-pushed the guyllian.gomez/yarn-pnp branch from 36c114c to 9aa7f3e Compare October 31, 2025 12:57
* add pnpapi test

* remove cycle dependency

* add error handling

* add error message

* apply error message

* add error bubbling

* code clean up

* remove useless test

* add empty findBrokenPeerDependencies

* early return

* change function name

* apply code review

* merge duplicate logic

* apply collect usage

* apply codereview

* apply fromConfig
@GGomez99 GGomez99 force-pushed the guyllian.gomez/yarn-pnp branch from 440f2f8 to 478d4d9 Compare November 3, 2025 10:50
@GGomez99 GGomez99 force-pushed the guyllian.gomez/yarn-pnp branch from 478d4d9 to f3c6477 Compare November 3, 2025 10:52
@GGomez99 GGomez99 force-pushed the guyllian.gomez/yarn-pnp branch from a98f2f6 to c343dca Compare November 20, 2025 13:49
Yarn-pnp: Attach to session and hoist filesystem overrides
Copilot AI review requested due to automatic review settings May 12, 2026 15:36
@GGomez99 GGomez99 force-pushed the guyllian.gomez/yarn-pnp branch from 8e79c4f to daac702 Compare May 12, 2026 15:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds native Yarn Plug’n’Play (PnP) support across the TypeScript-Go compiler and language server, including zip/VFS handling, module/type resolution, and test harness updates to validate behavior in PnP environments.

Changes:

  • Introduces a PnP API + manifest parser and wires it through compiler/LSP/module-resolution hosts.
  • Adds a PnP-aware VFS that supports virtual paths and reading from zip archives, plus LSP URI support for zip paths.
  • Extends resolution/type-roots/auto-import logic and adds new compiler + fourslash tests and baselines.

Reviewed changes

Copilot reviewed 72 out of 76 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
testdata/tests/cases/compiler/pnpTypeRootsResolution.ts New compiler test for resolving @types/* via PnP.
testdata/tests/cases/compiler/pnpTransitiveDependencies.ts New compiler test for enforcing direct-dependency rules under PnP.
testdata/tests/cases/compiler/pnpSimpleTest.ts New compiler test for basic PnP package resolution.
testdata/tests/cases/compiler/pnpDeclarationEmitWorkspace.ts New compiler test for declaration emit with workspace exports under PnP.
testdata/fixtures/pnp/test-expectations.json Adds/updates PnP resolution expectation fixtures.
testdata/baselines/reference/compiler/pnpTypeRootsResolution.types Baseline for PnP type analysis output.
testdata/baselines/reference/compiler/pnpTypeRootsResolution.symbols Baseline for PnP symbol output.
testdata/baselines/reference/compiler/pnpTypeRootsResolution.js Baseline for PnP JS emit output.
testdata/baselines/reference/compiler/pnpTransitiveDependencies.types Baseline for transitive-deps type output under PnP.
testdata/baselines/reference/compiler/pnpTransitiveDependencies.symbols Baseline for transitive-deps symbol output under PnP.
testdata/baselines/reference/compiler/pnpTransitiveDependencies.js Baseline for transitive-deps JS emit output under PnP.
testdata/baselines/reference/compiler/pnpTransitiveDependencies.errors.txt Baseline for expected missing-dependency error under PnP.
testdata/baselines/reference/compiler/pnpSimpleTest.types Baseline for basic PnP type output.
testdata/baselines/reference/compiler/pnpSimpleTest.symbols Baseline for basic PnP symbol output.
testdata/baselines/reference/compiler/pnpSimpleTest.js Baseline for basic PnP JS emit output.
testdata/baselines/reference/compiler/pnpDeclarationEmitWorkspace.types Baseline for PnP declaration-emit type output.
testdata/baselines/reference/compiler/pnpDeclarationEmitWorkspace.symbols Baseline for PnP declaration-emit symbol output.
testdata/baselines/reference/compiler/pnpDeclarationEmitWorkspace.js Baseline for PnP declaration-emit JS output.
internal/vfs/pnpvfs/pnpvfs.go Adds PnP VFS: virtual path support + zip file access with caching.
internal/vfs/pnpvfs/pnpvfs_test.go Tests for PnP VFS behavior (virtual paths, zip access, errors).
internal/tspath/path.go Adds IsZipPath helper.
internal/tsoptions/tsoptionstest/vfsparseconfighost.go Updates test ParseConfigHost to include PnP API/VFS behavior.
internal/tsoptions/tsconfigparsing.go Extends ParseConfigHost to expose PnP API.
internal/transformers/tstransforms/importelision_test.go Updates test host to satisfy new PnP API surface.
internal/testutil/tsbaseline/js_emit_baseline.go Treats .pnp.cjs as a preserved harness file for baselines.
internal/testutil/harnessutil/harnessutil.go Initializes PnP API + wraps FS in harness compilation pipeline.
internal/project/snapshot.go Threads PnP API through LSP snapshot lifecycle.
internal/project/session.go Stores PnP API in session; exposes via resolution host interface.
internal/project/projectcollectionbuilder.go Passes PnP API into project/config registry builders.
internal/project/extendedconfigcache_test.go Updates ParseConfigHost test implementation for PnP.
internal/project/configfileregistrybuilder.go Adds PnP API exposure to config file registry builder.
internal/project/compilerhost.go Exposes PnP API via project compiler host.
internal/project/autoimport.go Threads PnP API into auto-import registry clone host.
internal/pnp/pnpapi.go Implements PnP API (manifest loading, locator lookup, resolution, type roots).
internal/pnp/pnpapi_test.go Adds unit tests for manifest parsing + resolution behavior.
internal/pnp/pnp.go Adds PnP initialization + .pnp.cjs detection helper.
internal/pnp/manifestparser.go Parses .pnp.data.json / extracts payload from .pnp.cjs.
internal/modulespecifiers/types.go Extends module specifier host interface with PnP API.
internal/modulespecifiers/specifiers.go Adds PnP-aware module specifier generation logic.
internal/modulespecifiers/specifiers_test.go Updates mock host for new PnP API method.
internal/module/types.go Extends module ResolutionHost interface with PnP API.
internal/module/resolver.go Adds PnP resolution paths, PnP type roots, and PnP external-library detection.
internal/module/resolver_test.go Updates resolver test host stub for PnP API.
internal/lsp/server.go Initializes PnP API for LSP sessions and wraps FS with PnP VFS when present.
internal/lsp/lsproto/lsp.go Supports zip: document URIs in URI->filename conversion.
internal/ls/string_completions.go Notes potential PnP impacts on string completions (TODO).
internal/ls/lsconv/converters.go Emits zip: URIs for zip-backed paths.
internal/ls/lsconv/converters_test.go Adds tests for zip: URI round-tripping.
internal/ls/host.go Extends LS host interface with PnP API.
internal/ls/autoimport/util.go Plumbs PnP API into autoimport module resolver host wrapper.
internal/ls/autoimport/registry.go Adjusts autoimport indexing to account for PnP-managed packages.
internal/ls/autoimport/aliasresolver.go Extends checker.Program stub with PnP API method.
internal/fourslash/tests/pnpAutoImportCompletion_test.go Adds fourslash test validating PnP auto-import completions.
internal/execute/watcher.go Passes PnP API into compiler host construction for watch mode.
internal/execute/tsctests/sys.go Adds PnP API + PnP VFS wrapping to test system.
internal/execute/tsc/compile.go Extends tsc.System interface with PnpApi().
internal/execute/tsc.go Passes PnP API into cached compiler hosts for builds.
internal/execute/build/orchestrator.go Passes PnP API into build orchestrator compiler host.
internal/execute/build/host.go Exposes PnP API from build host wrapper.
internal/execute/build/compilerHost.go Exposes PnP API from build compiler host wrapper.
internal/diagnostics/loc_generated.go Import ordering adjustment in generated diagnostics localization file.
internal/diagnostics/extraDiagnosticMessages.json Adds PnP-specific diagnostic messages.
internal/diagnostics/diagnostics_generated.go Adds generated constants for new PnP diagnostics.
internal/core/compileroptions.go Refactors effective type-root computation to support PnP type roots.
internal/compiler/projectreferencedtsfakinghost.go Propagates PnP API through project-reference host wrapper.
internal/compiler/program.go Exposes PnP API via checker.Program from compiler Program.
internal/compiler/program_test.go Updates host construction for new PnP API parameter.
internal/compiler/host.go Extends compiler host interface/constructors to carry PnP API.
internal/compiler/emitHost.go Exposes PnP API via emit host interface.
internal/compiler/emit_test.go Updates emit benchmarks for new host constructor signature.
internal/checker/checker_test.go Updates checker tests for new host constructor signature.
internal/api/server.go Initializes PnP API + wraps FS for the API server.
cmd/tsgo/sys.go Initializes PnP API + wraps FS for CLI runs; exposes PnpApi().
_extension/src/client.ts Adds zip scheme support to VS Code extension document selector.
Files not reviewed (2)
  • internal/diagnostics/diagnostics_generated.go: Language not supported
  • internal/diagnostics/loc_generated.go: Language not supported

Comment thread internal/tsoptions/tsoptionstest/vfsparseconfighost.go
Comment thread internal/vfs/pnpvfs/pnpvfs.go
Comment thread internal/vfs/pnpvfs/pnpvfs.go
Comment thread internal/pnp/pnpapi.go
Comment on lines +43 to +47
// TODO: implement this from yarn sourcecode
// https://github.com/yarnpkg/berry/blob/master/packages/yarnpkg-pnp/sources/loader/makeApi.ts#L458
func findBrokenPeerDependencies(specifier string, parent *Locator) []Locator {
return []Locator{}
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is only used for error reports on peer dependencies, and might require other changes in the manifest parser

I can either:

  • Implement properly this function and the required changes
  • Remove all mentions of findBrokenPeerDependencies and this stub
  • Do nothing in this PR and implement the changes in a followup PR

Comment thread internal/pnp/pnpapi_test.go
Comment on lines +1121 to +1127
// If pnp is available, node_modules buckets won't be built as all packages are located in `.yarn/cache`
// This is why we need to handle all files, except the ones that are not importable in the project
pnpApi := b.host.PnpApi()
if pnpApi != nil {
if !pnpApi.IsImportable(string(projectPath), file.FileName()) {
continue
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pnpApi.IsImportable also returns true for files within the same package because the pnp manifest always adds the package itself as the first entry of PackageDependencies. I can add a defensive check in case the manifest is malformed but that doesn't seem necessary to me

Also, when PnP is enabled, we only have PnP-managed package locations, so passing all source files through IsImportable is valid here

arcanis pushed a commit to yarnpkg/berry that referenced this pull request Jun 2, 2026
## What's the problem this PR addresses?

Add editor SDK support for oxc. This should enable optional editor
integration with both oxfmt and oxlint.

## How did you fix it?

Added oxlint, oxfmt and oxlint-tsgolint as supported integration and
documented it.

Oxlint and oxlint-tsgolint addition is a bit hacky and I don't know
whether this is the right direction or not. Rationale can be read from
the comment there.

Loader option added to sdk setupEnv to prevent oxfmt tinypool from being
broken due to failing to resolve tinypool/dist/process.js.

I haven't tried this with vim yet since I don't use it so if anyone
would kindly help test whether this will work on vim or not, thanks!

### oxlint type-aware feature (tsgolint)

The biggest challenge here is the type-aware feature (tsgolint) support.
The rust side of oxlint spawn tsgolint a bit different from how the
oxlint itself is spawned:


https://github.com/oxc-project/oxc/blob/d3dcf5bc9718ebb4839be27062b5d82da2118e2e/crates/oxc_linter/src/tsgolint.rs#L265-L270

Instead of using js entrypoint, it directly spawn executable so the
wrapper approach is not directly applicable here. Luckily, there are
some strategies that the oxc_linter uses to resolve tsgolint:
- via PATH
- via node_modules search
- direct path via OXLINT_TSGOLINT_PATH (which is what oxc.path.tsgolint
does basically)

The only realistic strategy is to use PATH. For the sdks, it will
generate a windows executable shim (cmd) that will basically just spawn
tsgolint.js, else should fallback to tsgolint.

Be noted however that this will not fix fundamental issue with
typescript-go module resolution, which requires this to be merged first:
- microsoft/typescript-go#1966

This PR just enable the usage of tsgo regardless of whether it can
resolve with pnp or not in the type-checking.

### Reference

Coc config:

https://github.com/oxc-project/coc-oxc/blob/7a7f0d8f503d5cdb65da37232da6b865f159b42a/src/common.ts#L27-L36

Vscode oxc binary resolution reference: 
https://github.com/oxc-project/oxc-vscode/blob/main/client/findBinary.ts

Oxlint-tsgolint executable resolution:

https://github.com/oxc-project/oxc/blob/d3dcf5bc9718ebb4839be27062b5d82da2118e2e/crates/oxc_linter/src/tsgolint.rs#L1164-L1225

## Checklist

- [x] I have read the [Contributing
Guide](https://yarnpkg.com/advanced/contributing).
- [x] I have set the packages that need to be released for my changes to
be effective.
- [x] I will check that all automated PR checks pass before the PR gets
reviewed.
@RyanCavanaugh RyanCavanaugh added the No linked issue This PR doesn't say what bug it fixes label Jun 8, 2026
@GGomez99

Copy link
Copy Markdown
Author

@RyanCavanaugh The issue is linked in the motivation field of the PR description. Here's the full link: #460

But perhaps you would like me to link it to this PR differently?

@SerenModz21

Copy link
Copy Markdown

I'd assume he wants it linked properly, not just a mention.

To do this, you can add the following in the PR description:

Closes #460

Docs: https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword

wincent added a commit to wincent/wincent that referenced this pull request Jun 19, 2026
See:

- https://devblogs.microsoft.com/typescript/announcing-typescript-7-0-rc/

Ok, so this is a bit fiddly, so let me explain what's going on:

- We used to have `tsc` from `typescript` (v6), written in JS, and
  `tsgo` from `@typescript/native-preview`, so they could both be
  installed at the same time.
- With v7, the RC package takes over the `tsc` executable; it's now the
  Go implementation and there is no separate `tsgo`; so...
- We change our global install to `7.0-rc`, and ditto for the in-project
  install. We no longer have a v6 install on disk.
- Rename (the project-local) `bin/tsgo` to `bin/tsc`, and make it run
  `tsc` instead of `tsgo`.
- We update our Neovim config to assume `tsc` is now the Go version
  instead of the JS version; but note that we still have to use the
  "tsgo" config from nvim-lspconfig, because that hasn't been renamed
  yet, it's just that we configure it to use the `tsc` executable now
  instead of `tsgo`.
- The Go implementation still doesn't support Yarn PnP and the
  corresponding PR (microsoft/typescript-go#1966)
  is still open, so we maintain our hack wherein we use the `ts_ls`
  config from nvim-lspconfig to use typescript-language-server with a
  wired-in path to a Yarn-managed shim. This looks fragile, and it
  probably is, but it has been working for me ever since I added it a
  couple of months ago (in 011cee5, "feat(nvim): point ts_ls at Yarn
  PnP tsserver shim when available", 2026-05-12).
- The Neovim config will use a repo-local `tsc` if one exists, but note
  that it assumes `tsc` is the Go implementation. So if you use it in a
  repo with v6 or older, it won't work, because v6 doesn't support
  `--lsp --stdio`. I'm not worried about this; I have only a few such
  repos, and I will update them next time I have to work in them.
- Update references to `tsgo` in docs to reference `tsc` instead.
- Note: the `_tsc` completions in the third-party zsh-completions repo
  still match the JS implementation instead of the Go one; presumably
  they will eventually update. I am not sure exactly where the command
  line options diverge, other than having seen that while v7 supports
  LSP via `tsc --lsp --stdio`, v6 does not.

Finally, and most importantly, merely updating my dotfiles on other
machines won't make all this magically start working: I will need to
manually uninstall the old `typescript` package and install the new one:

```
npm uninstall -g typescript @typescript/native-preview
npm install -g typescript@rc --min-release-age=0
```

Change-Id: rmoqwxkqwoywkxrlwspqqmwwmktwqrvz
arcanis added a commit to yarnpkg/berry that referenced this pull request Jul 1, 2026
## Summary

- Bound the generated TypeScript compatibility patch to TypeScript
versions before 7.0.0, and committed the matching generated patch
artifact so the compat patch generator stays reproducible.
- Fixed optional patch handling so `optional!` patches fall back when a
package no longer ships a target file (for example `lib/_tsc.js` in
TypeScript 7 / TypeScript compatibility shims), instead of failing the
install with `ENOENT`.
- Added regression coverage for both direct `typescript@7.0.1-rc`
installs and the TypeScript 7 side-by-side recommendation where the
`typescript` ident aliases to `npm:@typescript/typescript6@^6.0.0`.
- Stabilized the scoped `plugin-typescript` acceptance test by moving it
from `@babel/traverse` (now reported by npm search metadata as having
included types) to a fixture package that still exercises
DefinitelyTyped scoped package insertion.
- Added deferred version metadata for the changed workspaces.

## Test plan

- `yarn workspace @yarnpkg/plugin-compat test:plugin-compat`
- `yarn test:unit packages/plugin-patch`
- `node ./scripts/run-yarn.js test:integration
packages/acceptance-tests/pkg-tests-specs/sources/plugins/plugin-typescript.test.ts`
- `yarn version check`

## Notes

TypeScript 7 ships a native compiler package layout and no longer has
the legacy JS compiler files patched by the existing PnP compatibility
diff, such as `lib/_tsc.js`. Bounding the TypeScript compat patch makes
this intent explicit, while the optional patch fallback fixes the
broader bug that `optional!` patch failures caused by missing target
files were still fatal.

TypeScript 7 support should ship with microsoft/typescript-go#1966.

Made with Cursor

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: John Doe <you@example.com>
Co-authored-by: Maël Nison <mael.nison@mistral.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

No linked issue This PR doesn't say what bug it fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Yarn PnP