Commit 036e883
Coalesce concurrent native finder refreshes with the same key (microsoft#1598)
## Summary
Fixes microsoft#1587.
At startup, several environment managers (venv, system, pyenv, pipenv,
poetry, conda) each call `nativeFinder.refresh(true, undefined)`
concurrently — typically triggered when the Python extension calls
`api.refreshEnvironments(undefined)` via its `triggerRefresh` on
activation. Every one of those calls resolves to the same cache key
(`'all'`) but is queued as a distinct task on the native finder's worker
pool, which runs with `concurrency = 1`. Neither the worker pool nor
`doRefresh` deduplicates identical submissions, so N parallel callers
become N sequential full PET discoveries — same input, same output, N-1
wasted scans.
The reporter (pytorch workspace) saw the result of this stacking:
```
[Pipenv] Environment discovery took 199.2s (found 0 environments).
[Poetry] Environment discovery took 253.8s (found 0 environments).
```
plus the `Setup appears hung during stage: managerRegistration` watchdog
firing, and the same `Discovered env: <path>` lines repeating 4-5× in
the log — the smoking gun that PET ran the same scan multiple times
back-to-back.
## Fix
Add an in-flight tracking map (`Map<string, Promise<NativeInfo[]>>`) in
`NativePythonFinderImpl` and a guard at the top of `handleHardRefresh`:
if a refresh for the same key is already running, return the existing
promise instead of queueing another task. The slot is cleared in
`.finally` so a rejected refresh does not poison future callers —
sequential refreshes still each run a fresh PET scan.
`handleSoftRefresh` already falls through to `handleHardRefresh` on
cache miss, so it picks up the dedup automatically — a concurrent soft +
hard for the same key now share one PET scan.
## Why this layer
- All duplicate-refresh paths converge at `handleHardRefresh` (manager
refreshes, conda's `getConda` fallback via `native.refresh(false)`, the
Python extension's `triggerRefresh`, the `Refresh All Environment
Managers` command, external API consumers). Fixing here covers every
caller in one place.
- No public API change, no observable semantic change for sequential
refreshes, no test breakage.
- Mirrors the existing `inFlight: Map<string, Promise<…>>` idiom in
`src/features/inlineScriptLazyDetector.ts`.
## Expected impact for microsoft#1587
| Metric | Before | After |
|---|---:|---:|
| PET scans triggered at startup | ~6-7 | **1** |
| Pipenv discovery warning | 106-199 s | ~25-35 s |
| Poetry discovery warning | 121-253 s | ~25-35 s |
| `Setup appears hung during stage: managerRegistration` | fires |
should not fire |
| Total startup-to-ready | ~150 s | **~40-50 s** (~3× wall-clock) |
The fix eliminates duplicate-scan stacking; it does not speed up a
single PET scan (intrinsic ~25-35 s on pytorch on the reporter''s
machine — separate from this change).
## Verification
- ✅ Webpack production build succeeds.
- ✅ ESLint clean on the changed file.
- ✅ Unit tests: 1215 pass (same 3 unrelated pre-existing failures on
`main`).
- ✅ Built a VSIX with this branch and a VSIX from `main`, diffed the
minified `handleHardRefresh` body — the fix VSIX has the new structure
(`inFlightRefreshes.get → guard → addToQueue.then.finally`), `main` has
the old (`await pool.addToQueue`).
Tested manually by inspecting the minified output of both builds; would
appreciate eyes from someone with the pytorch repro to confirm
wall-clock improvement on a real run.
## What this does NOT address (intentionally)
- Single PET scan latency on large workspaces (separate, PET-side).
- Cross-session cache misses on remote (issue microsoft#1581, separate fix in
flight).
- The Python extension''s `triggerRefresh` on every activation (worth a
follow-up — could skip when our cache is non-empty).
- The double-init in `InternalEnvironmentManager.refresh`''s tail-call
to `getEnvironments(''all'')` (separate small follow-up; produces the
"phantom" lowercase-only refresh log lines).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 2645a48 commit 036e883
1 file changed
Lines changed: 32 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
326 | 326 | | |
327 | 327 | | |
328 | 328 | | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
329 | 334 | | |
330 | 335 | | |
331 | 336 | | |
| |||
555 | 560 | | |
556 | 561 | | |
557 | 562 | | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
558 | 570 | | |
559 | 571 | | |
560 | 572 | | |
561 | 573 | | |
562 | 574 | | |
563 | 575 | | |
564 | | - | |
565 | | - | |
566 | | - | |
567 | | - | |
568 | | - | |
569 | | - | |
570 | | - | |
571 | | - | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
572 | 596 | | |
573 | 597 | | |
574 | 598 | | |
| |||
0 commit comments