Skip to content

refactor(apex-lsp-parser-ast): make ISymbolManager async, delegate to ops - W-22201104#352

Merged
kylewalke merged 20 commits intomainfrom
feature/W-22201104-isymbolmanager-async
May 7, 2026
Merged

refactor(apex-lsp-parser-ast): make ISymbolManager async, delegate to ops - W-22201104#352
kylewalke merged 20 commits intomainfrom
feature/W-22201104-isymbolmanager-async

Conversation

@kylewalke
Copy link
Copy Markdown
Contributor

@kylewalke kylewalke commented Apr 29, 2026

What does this PR do?

Phase 2 of the Apex Language Server worker-topology refactor — flips
ISymbolManager from synchronous to Promise-returning and replaces
the ~6.3k-line ApexSymbolManager with an ~850-line coordinator that
delegates to the Effect ops/services added in Phase 1 (#351).
Behavior-preserving; the entire test suite still passes.

Stacked PR. Targets feature/W-22201103-effect-ops-services so
reviewers see only the Phase 2 diff. Will be retargeted to main
once #351 merges.

Why a single commit (the WI's 6a/6b note)

The interface flip is atomic: the moment ISymbolManager reads
become Promise<...>, every caller in the repo must await in the
same compile step. A clean 6a/6b split is not achievable without
adding backward-compat shims absent from phale/workers (much extra
work that deviates from the source-of-truth branch). Lands as one
atomic commit; one bisect step.

packages/apex-parser-ast/src/types/ISymbolManager.ts

  • All read methods (getSymbol, findSymbolByName, findSymbolByFQN,
    findFQNForStandardClass, findSymbolsInFile, findFilesForSymbol,
    getAllReferencesInFile, getAllSymbolsForCompletion,
    findReferencesTo, findReferencesFrom, findRelatedSymbols,
    resolveSymbol, …) return Promise<...>.
  • addSymbol returns Promise<void> so worker callers can await
    data-owner mutation completion.

packages/apex-parser-ast/src/symbols/ApexSymbolManager.ts

packages/apex-parser-ast/src/symbols/services/symbolManagerFacade.ts

packages/apex-parser-ast/src/symbols/{ApexSymbolIndexingService,ApexSymbolProcessingManager}.ts

  • Add await on the new async ISymbolManager calls.

packages/apex-parser-ast/src/semantics/validation/

  • All 25 validators (AnnotationProperty, AssignmentAccess,
    ClassHierarchy, Constructor, Deprecation, Exception, Expression,
    Instanceof, InterfaceHierarchy, MethodCall, MethodResolution,
    NewExpression, StaticContext, SwitchStatement, TypeAssignment,
    TypeResolution, TypeVisibility, UnknownAnnotation,
    VariableResolution, …) plus ArtifactLoadingHelper,
    ValidatorRegistry, and utils/visibilityUtils propagate await
    on ISymbolManager reads.

packages/apex-parser-ast/src/namespace/

  • NamespaceResolutionService, NamespaceUtils, and
    ResolutionRules await ISymbolManager reads.

packages/lsp-compliant-services/src/services/ (and utils/)

  • All processing services that read symbols (Hover, Definition,
    Implementation, References, Completion, Diagnostic, CodeAction,
    CodeLens, GraphData, Document, WorkspaceSymbol, SignatureHelp,
    LayerEnrichment, PrerequisiteOrchestration,
    MissingArtifactResolution, …) propagate await and convert local
    helpers that consume those reads into async /
    Promise-returning shapes.
  • displayFQNUtils and missingArtifactUtils updated similarly.

Tests

  • ~35 apex-parser-ast and 8 lsp-compliant-services test files
    updated to await the new async API. No coverage shape changes.
  • Snapshot file semantic-errors.test.ts.snap was deliberately not
    pulled from phale/workers (its updates accompany Phase 3
    behavior). Existing snapshots still pass against Phase 2.

Out of scope (deferred to later stories)

  • packages/apex-ls/ worker runtime (worker.platform,
    WorkerCoordinator, LCSAdapter, CoordinatorAssistanceMediator,
    ResourceLoaderProxy, WorkerExecArgvBuilder, WorkspaceBatchHandler)
  • lsp-compliant-services/src/workers/WorkerBackendBootstrap.ts
  • Phase 3 LSPQueueManager rewrite + ServiceFactory /
    ServiceConfiguration worker-topology wiring
  • VS Code worker observability scripts
  • docs/architecture/, docs/debugging/, docs/testing/ write-back
    documentation
  • apex-lsp-shared/src/workerWireSchemas.ts Phase 3 additions

Follow-up fixes on this branch since original push

Additional commits beyond the original refactor (all small, targeted):

fix: stdlib availability (3f7b723b)

Fixes a cold-start regression surfaced by hover on bare standard-
library types (String, Integer, Assert, …).

Root cause. ApexSymbolManager's constructor defaults its
stdlibProvider to ResourceLoaderNoOpInstance
(ApexSymbolManager.ts:296-299).
ApexSymbolProcessingManager.getInstance(stdlibProvider?) only
applies the argument on the first call that creates the singleton —
and every production call site (hover, completion, diagnostics,
definition, rename, code lens, workspace symbol, etc.) uses the
no-arg form. Meanwhile LCSAdapter.initializeResourceLoader
dutifully loads the ZIP into the ResourceLoader singleton but
never injects it. Result: findNamespaceForClass('String') returns
an empty set forever, isStandardLibraryType returns false, and
hover on stdlib types falls into the "🔍 Searching for symbol…"
placeholder / returns the enclosing variable symbol.

Fix. After ResourceLoader.initialize() resolves in
LCSAdapter.initializeResourceLoader, call
ApexSymbolManager.setStdlibProvider({...}) with an inline adapter
that delegates to the initialized ResourceLoader singleton. Uses
the already-defined but previously-never-called setStdlibProvider
setter
(ApexSymbolManager.ts:392),
whose entire purpose was this scenario.

  • Single-file, 32-line change in
    packages/apex-ls/src/server/LCSAdapter.ts.
  • New 3-case regression test
    e2e-tests/tests/apex-hover-stdlib.spec.ts
    covering String as a method-call qualifier
    (String.isBlank(...)), a declared field type
    (private String instanceId;), and a method return type
    (public static String getCurrentUserName() {...}). All three
    fail on the original refactor commit and pass after the fix.
  • Investigation notes captured in
    docs/hover-string-diagnosis.md.
  • Considered but deferred — Alternative B: await initializeResourceLoader() at server startup instead of
    fire-and-forget
    (LCSAdapter.ts:1598).
    That would close the remaining 1-2s cold-start race window, but
    trades LSP readiness latency and warrants its own measurement;
    tracked as a follow-up ticket.

fix: pjson update (8ed91c4c)

package-lock.json housekeeping (-1 net line). No runtime change.

fix: remove extra cmt (ac1b00a2)

Drops a stray comment block in apex-hover-stdlib.spec.ts left over
from the investigation.


Notes

  • Stats: 90 files changed, +3,047 / −7,248 (net −4,201 thanks to
    ApexSymbolManager.ts shedding ~5.4k LoC into the Phase 1 ops).
    Follow-up fix commits add ~100 LoC across LCSAdapter.ts and the
    new e2e spec; totals updated accordingly in the diff viewer.
  • Verification (husky pre-commit + local runs):
    • compile / lint / format: clean
    • apex-parser-ast jest: 2762 passed / 0 failed
    • lsp-compliant-services jest: 551 passed / 0 failed
    • apex-ls jest: 53 passed / 0 failed
    • e2e-tests/tests/apex-hover-stdlib.spec.ts (new, 3 cases):
      passed (27.8s)
    • e2e-tests/tests/apex-hover.spec.ts (existing, 21 cases):
      passed (2.6m)

What issues does this PR fix or reference?

@W-22201104@

Made with Cursor

kylewalke and others added 11 commits April 29, 2026 12:46
…W-22201103

Two additive type changes that exist to support the upcoming
worker-topology refactor. No existing call sites changed.

`types/queue.ts`

- Add optional `workerTopology` field to `SchedulerMetrics` carrying
  data-owner / enrichment-pool / resource-loader status, dispatched
  count, and coordinator-only request types. Surfaced by future queue
  state dashboards; absent today.

`types/symbol.ts`

- Defensive: `SymbolTable.isPositionInIdentifierRange` returns false
  when `location.identifierRange` is undefined, matching the optional
  shape callers can produce after a wire round-trip.
- Implement `SymbolTable.fromJSON` (was `// TODO: Implement` returning
  an empty table). Adds `fromSerializedData` as the primary entry and
  reroutes `fromJSON` to it, accepting both legacy `symbolArray` and
  new wire-format `symbols` shapes. Required for IPC reconstruction
  on worker thread boundaries.

Made-with: Cursor
…ment - W-22201103

Add seven Effect-based data services for the upcoming worker-topology
refactor. All seven are pure additions and dead code today — no
existing call site reads or writes through them. Each ships with a
shim adapter that lets non-Effect callers use the same API surface
once Phase 2 wires up the facade.

Services (under `src/symbols/services/`)

- `symbolIndexStore` — primary symbol index (HashMap of fileUri →
  SymbolTable + reverse symbol lookups)
- `referenceStore` — outgoing/incoming reference graph keyed by file
- `cacheStore` — derived caches (FQN → symbols, type → resolved chain)
- `fileStateStore` — file metadata, version, parse state
- `symbolIndexStoreLive` — Effect Layer providing the live
  SymbolIndexStore implementation backed by the existing in-memory map
- `concurrencyGuards` — per-file mutex/serial locks used by the data
  owner to keep mutations strictly ordered
- `ResourceLoaderService` — Effect Layer wrapping the existing
  `ResourceLoader` singleton so stdlib lookups can be substituted in
  worker contexts

These services exist independently — none import any other ops/services
file in this commit.

Made-with: Cursor
Add a standalone symbol resolver service that encapsulates the
recursive symbol-chain walking logic used by hover, definition, and
type-check paths.

`services/symbolResolver`

- Pure Effect Context.Tag service; no platform deps.
- Consumes ResourceLoaderServiceShape (type-only) for stdlib lookups.
- Exposes resolveSymbol, resolveTypeChain, and friends — used by the
  chain-resolution and ref-resolution ops in later commits.

Dead code today; no existing call site invokes the resolver.

Made-with: Cursor
…22201103

Add three small standalone op modules used by later ops and the future
facade. Pure additions; no existing call site invokes them.

Ops (under `src/symbols/ops/`)

- `positionUtils` — position/range helpers (containment checks, sort
  by location, pick innermost) shared by lookup, references, and
  hover ops.
- `typeHierarchy` — `getContainingType`, `getAncestorChain`,
  `constructFQN` extracted from ApexSymbolManager so they can run
  inside Effect contexts.
- `resolutionContext` — value type representing a single resolution
  request (file uri, position, request kind) plus a builder for it.

Made-with: Cursor
…tion ops - W-22201103

Add seven Effect ops that read or mutate the symbol index. All depend
on the data services from the prior commit and are dead code today.

Ops (under `src/symbols/ops/`)

- `symbolLookup` — getSymbol / findByName / findByFQN / findInFile /
  findFilesForSymbol; the read API surface of the index
- `symbolMutation` — removeFile, clear, optimizeMemory,
  setCommentAssociations; mutating ops gated on the future data-owner
- `symbolProvider` — namespace-aware finders (find, findScalarKeywordType,
  findSObjectType, findExternalType, findInDefaultNamespaceOrder, etc.)
- `symbolResolution` — resolveSymbol entry plus context builders
  (createResolutionContext, getDetailLevelForFile)
- `referenceOps` — findReferencesTo/From, findRelatedSymbols,
  getReferencesAtPosition, getAllReferencesInFile
- `graphAnalysis` — analyzeDependencies, detectCircularDependencies,
  getGraphData and friends
- `stdlibLoading` — isStandardApexClass / resolveStandardApexClass
  routed through the ResourceLoaderService

No existing call site invokes any of these.

Made-with: Cursor
Add the two large resolution ops that walk receiver/method chains and
resolve symbol references. Both consume the symbolResolver service
added earlier.

Ops (under `src/symbols/ops/`)

- `chainResolution` — port of the chained-expression resolver (method
  chains, property chains, generic substitution, this/super, casts).
  3.5k LoC; preserves the existing fixture coverage by holding all
  current branches in one file rather than fragmenting now.
- `symbolRefResolution` — `SymbolReference` → `ApexSymbol` resolution
  including overload picking and fallback chains.

Both stay dead until Phase 2 (W-22201104) routes ApexSymbolManager
through the facade.

Made-with: Cursor
…-22201103

Compose the seven lookup/mutation/provider/resolution ops behind a
single Effect Context.Tag (`IEffectSymbolManager`) so call sites can
later swap legacy ApexSymbolManager for the Effect-native facade.

`services/symbolManagerFacade`

- IEffectSymbolManager — Tag holding the IEffectSymbolManagerShape
  (find / findByName / findByFQN / removeFile / clear / resolveSymbol
  / getReferences / getGraphData / etc.).
- iEffectSymbolManagerFromLegacy — adapter that wraps an existing
  ApexSymbolManager instance behind the same Tag, used during the
  staged migration.
- Layer wiring composes SymbolIndexStore, ReferenceStore, CacheStore,
  FileStateStore, ConcurrencyGuards, and ResourceLoaderService.

The legacy bridge for `addSymbol` wraps the synchronous return in
`Promise.resolve` to match the Effect.promise contract; once Phase 2
makes ApexSymbolManager.addSymbol async, the wrapper drops out.

No existing call site requests this Tag yet.

Made-with: Cursor
…rviceShape helper - W-22201103

Add a mock implementation of the IEffectSymbolManagerShape and a small
addition to the existing test helpers so unit tests for the new ops
and facade can run without spinning up the real ApexSymbolManager or a
live ResourceLoader.

`test/helpers/mockSymbolManager`

- New helper that produces a configurable IEffectSymbolManagerShape
  backed by in-memory maps. Used by the upcoming op-level tests.

`test/helpers/testHelpers`

- `getResourceLoaderServiceShapeFromSingleton` — builds a
  ResourceLoaderServiceShape from the initialized ResourceLoader
  singleton, so tests that need stdlib resolution can call into Effect
  ops with the shim shape without provisioning a fresh loader.

Made-with: Cursor
…1103

Add barrel re-exports for the new Effect data services and ops added
in this story. Each op-level export uses the `*Op` suffix
(`getSymbolOp`, `findByNameOp`, …) so it cannot collide with the
methods of the same name on legacy ApexSymbolManager that consumers
already import.

`src/index.ts`

- Re-export Effect data services: SymbolIndexStore, ReferenceStore,
  CacheStore, FileStateStore, ConcurrencyGuards, IEffectSymbolManager,
  SymbolIndexStoreLive, plus per-store shim factories and Shape types.
- Re-export op functions with the `*Op` suffix grouped by module
  (lookup, typeHierarchy, referenceOps, graphAnalysis, symbolProvider,
  symbolMutation, symbolResolution, stdlibLoading).
- Re-export ResourceLoaderService Effect service module.

Surfaces all of the dead-code additions in this story for downstream
packages, but no internal consumer imports them yet.

Made-with: Cursor
… ops - W-22201104

Phase 2 of the Apex Language Server worker-topology refactor — the
foundation flip from the synchronous `ISymbolManager` API to a
Promise-returning interface, plus the ApexSymbolManager rewrite that
delegates to the Effect ops/services added in PR #351 (W-22201103).
Behavior-preserving; full test suite still passes (4150/4150,
+10 new from in-scope test additions).

The interface change is atomic by nature: every read on
`ISymbolManager` becomes async at the same instant. Per the WI
suggestion to "consider 6a/6b", a clean split is not achievable
without adding shim layers absent from `phale/workers`, so this
lands as a single commit.

`packages/apex-parser-ast/src/types/ISymbolManager.ts`

- All read methods (`getSymbol`, `findSymbolByName`, `findSymbolByFQN`,
  `findFQNForStandardClass`, `findSymbolsInFile`, `findFilesForSymbol`,
  `getAllReferencesInFile`, `getAllSymbolsForCompletion`,
  `findReferencesTo`, `findReferencesFrom`, `findRelatedSymbols`,
  `resolveSymbol`, …) now return `Promise<...>`.
- `addSymbol` returns `Promise<void>` so worker callers can await
  data-owner mutation completion.

`packages/apex-parser-ast/src/symbols/ApexSymbolManager.ts`

- ~6.3k → ~850 LoC. Becomes a thin coordinator that translates legacy
  call signatures into the Effect ops added in W-22201103
  (symbolLookup, symbolMutation, symbolProvider, symbolResolution,
  referenceOps, graphAnalysis, typeHierarchy, stdlibLoading) via the
  live SymbolIndexStore, ReferenceStore, CacheStore, FileStateStore,
  and ResourceLoaderService.

`packages/apex-parser-ast/src/symbols/services/symbolManagerFacade.ts`

- Drops the `Promise.resolve(legacyManager.addSymbol(...))` bridge
  from PR #351 now that `addSymbol` is genuinely async.

`packages/apex-parser-ast/src/symbols/{ApexSymbolIndexingService,ApexSymbolProcessingManager}.ts`

- Add `await` on the new async ISymbolManager calls.

`packages/apex-parser-ast/src/semantics/validation/`

- All 25 validators (AnnotationProperty, AssignmentAccess,
  ClassHierarchy, Constructor, Deprecation, Exception, Expression,
  Instanceof, InterfaceHierarchy, MethodCall, MethodResolution,
  NewExpression, StaticContext, SwitchStatement, TypeAssignment,
  TypeResolution, TypeVisibility, UnknownAnnotation,
  VariableResolution, …) plus `ArtifactLoadingHelper`,
  `ValidatorRegistry`, and `utils/visibilityUtils` propagate `await`
  on ISymbolManager reads.

`packages/apex-parser-ast/src/namespace/`

- `NamespaceResolutionService`, `NamespaceUtils`, `ResolutionRules`
  await ISymbolManager reads.

`packages/lsp-compliant-services/src/services/` (and `utils/`)

- All processing services that read symbols (Hover, Definition,
  Implementation, References, Completion, Diagnostic, CodeAction,
  CodeLens, GraphData, Document, WorkspaceSymbol, SignatureHelp,
  LayerEnrichment, PrerequisiteOrchestration,
  MissingArtifactResolution, …) propagate `await` and convert local
  helpers that consume those reads into `async`/`Promise`-returning
  shapes.
- `displayFQNUtils` and `missingArtifactUtils` updated similarly.

`packages/{apex-parser-ast,lsp-compliant-services}/test/`

- ~35 apex-parser-ast and 8 lsp-compliant-services test files updated
  to `await` the new async API. No coverage shape changes.

Out of scope (deferred to later stories)

- Worker runtime files (`packages/apex-ls/`), `WorkerBackendBootstrap`,
  `LSPQueueManager`, `ServiceFactory`/`ServiceConfiguration` Phase 3
  worker-topology wiring, queue runtime updates, VS Code worker
  observability scripts, docs, and the `semantic-errors` snapshot
  changes that accompany Phase 3 behavior.

Verification

- compile / lint / format: clean
- tests: 4150 passed, 0 failed, 488 pending

Made-with: Cursor
Base automatically changed from feature/W-22201103-effect-ops-services to main May 1, 2026 17:27
@kylewalke kylewalke marked this pull request as ready for review May 5, 2026 17:08
@kylewalke kylewalke requested a review from a team as a code owner May 5, 2026 17:08
@kylewalke kylewalke requested a review from madhur310 May 5, 2026 17:08
@forcedotcom forcedotcom deleted a comment from github-actions Bot May 5, 2026
@forcedotcom forcedotcom deleted a comment from github-actions Bot May 5, 2026
@kylewalke kylewalke requested a review from peternhale May 6, 2026 17:13
peternhale and others added 2 commits May 6, 2026 13:13
…ers - W-22201104 (#372)

chore: make ISymbolManager resolveWithEnrichment support async resolvers

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kylewalke kylewalke merged commit 3bec0bc into main May 7, 2026
13 of 15 checks passed
@kylewalke kylewalke deleted the feature/W-22201104-isymbolmanager-async branch May 7, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants