Avoid stub duplication on ever thread

Author: AJenbo

docs/CHANGELOG.md

@@ -46,6 +46,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- **Diagnostic and request workers no longer copy the embedded stub indexes.** Every diagnostic pass and every hover, completion, or go-to-definition request cloned the full embedded stub class, function, and constant indexes (thousands of entries each) instead of sharing them. During workspace analysis this happened twice per file, and the resulting allocation churn serialised the parallel diagnostic workers in the memory allocator. The indexes are now shared, roughly halving `analyze` wall time on large Laravel projects and removing the same overhead from every editor request.
 - **Member resolution no longer degrades on deeply nested class dependencies.** Class resolution previously cut off at internal nesting limits, silently returning incomplete member sets (missing completions, spurious unknown-member diagnostics) on projects with deeply intertwined hierarchies and virtual members. Those limits are gone: only genuine dependency cycles, such as two Eloquent models whose relationships reference each other, fall back to a partial view, and they now do so deterministically instead of depending on which class happened to resolve first on a given thread.
 - **Completion works after closing a multi-line closure argument on the same line as the next chain operator.** Typing `->` immediately after a call like `->map(function (...) { ... })->` now resolves the full receiver instead of returning no member suggestions until the operator is moved to a new line. Contributed by @calebdw.
 - **Conditional return types recognize interpolated strings as strings.** A function with a conditional return type like `($key is string ? mixed : null)` now correctly resolves the `string` branch when called with an interpolated string argument (e.g. `config("{$prefix}.host")`). Previously the interpolated string was not recognized as a string literal, causing the return type to fall through to the else branch and resolve as `null`. Contributed by @calebdw.

docs/todo/performance.md

@@ -421,6 +421,16 @@ perf report --stdio --no-children | head -80
 
 # Flamegraph (requires the `flamegraph` crate or perf-tools):
 perf script | flamegraph > /tmp/phpantom.svg
+
+# Instantaneous CPU utilisation over a run (% of one core, sampled
+# every 0.5 s), for machines where perf is unavailable
+# (kernel.perf_event_paranoid > 2):
+./target/release/phpantom_lsp analyze --project-root <dir> --no-colour \
+  >/dev/null 2>&1 & PID=$!; prev=0
+while kill -0 $PID 2>/dev/null; do
+  cur=$(awk '{print $14+$15}' /proc/$PID/stat 2>/dev/null) || break
+  [ -n "$cur" ] && echo $(( (cur - prev) * 2 )); prev=$cur; sleep 0.5
+done
 ```
 
 ### Pathological test file
@@ -964,27 +974,34 @@ current single populate thread.
 
 **Impact: Medium-High · Effort: Medium**
 
-Measured on a 32-core machine against a large Laravel project (release
-build): the `analyze` Phase 2 diagnostic pass runs at ~2 to ~13 cores,
-averaging roughly 4, despite spawning one worker per core with atomic
-work-stealing. Utilisation is lowest at the start of the pass and
-climbs as it progresses, which points at lock contention that eases as
-shared caches warm, rather than at work starvation. Likely suspects:
-`find_or_load_class` taking write locks while lazily parsing vendor
-files early in the pass, and write contention on the resolved-class
-cache and chain-resolution caches. The LSP workspace diagnostics pass
-uses the same collectors and likely has the same ceiling. Needs
-profiling to attribute the blocked time before picking a fix;
-re-measure after any change with the CPU-sampling loop in the Appendix.
-
-Projects that keep substantial code in `vendor/` (e.g. models shipped
-in a shared vendor package that the application depends on) hit this
-harder: eager population only walks classes from the indexed user
-files, so every vendor class is parsed lazily on first touch during
-the diagnostic pass. The observed CPU shape on such a project is
-bursty (alternating stalls and full-core bursts) consistent with one
-worker holding the load lock while the rest wait, then all proceeding
-until the next unparsed vendor class.
+Measured on a 32-core machine against large Laravel projects (release
+build): the `analyze` Phase 2 diagnostic pass peaks around 14 cores and
+averages fewer, despite spawning one worker per core with atomic
+work-stealing. The original worst offender (every worker deep-cloning
+the embedded stub class/function/constant indexes twice per file via
+`clone_for_diagnostic_worker`) is fixed; the indexes are Arc-shared
+now. The remaining ceiling, per a frame-pointer `perf` profile of the
+diag workers after that fix:
+
+- ~18% of worker on-CPU time is still malloc/free/memmove, now diffuse:
+  `PhpType`, `MethodInfo`, and `ParameterInfo` clones throughout the
+  type engine, plus glibc malloc arena contention (visible as
+  `arena_get2`/futex kernel time) when 32 workers allocate heavily.
+- Read-lock contention on `fqn_class_index` and `class_not_found_cache`
+  is measurable but small (~1.5% on-CPU in the rwlock slow paths).
+- Projects that keep substantial code in `vendor/` (e.g. models shipped
+  in a shared vendor package) pay extra: eager population only walks
+  classes from the indexed user files, so every vendor class is parsed
+  lazily on first touch during the diagnostic pass, serialising workers
+  on the load path early in the run.
+
+Likely next steps, in rough value order: reduce clone traffic in the
+hot type-engine paths (share via `Arc`/`Cow` instead of cloning
+`PhpType`/member vectors), include depended-upon vendor classes in
+eager population, and only then consider allocator-level fixes. The
+LSP workspace diagnostics pass uses the same collectors and has the
+same ceiling. Re-measure with `perf` (frame-pointer build) or the
+CPU-sampling loop in the Appendix after any change.
 
 ---
 

src/lib.rs

@@ -491,7 +491,7 @@ pub struct Backend {
     /// Consulted by `find_or_load_class` as a final fallback after the
     /// `uri_classes_index` and PSR-4 resolution.  Stub files are parsed lazily on
     /// first access and cached in `uri_classes_index` under `phpantom-stub://` URIs.
-    pub(crate) stub_index: RwLock<CiMap<&'static str>>,
+    pub(crate) stub_index: Arc<RwLock<CiMap<&'static str>>>,
     /// Cache of fully-resolved classes (inheritance + virtual members).
     ///
     /// Keyed by fully-qualified class name.  Populated lazily by
@@ -601,15 +601,15 @@ pub struct Backend {
     /// Filtered at startup via [`set_php_version`](Self::set_php_version) to
     /// remove stubs that do not exist in the target PHP version.
     /// Can be consulted to resolve return types of built-in function calls.
-    pub(crate) stub_function_index: RwLock<CiMap<&'static str>>,
+    pub(crate) stub_function_index: Arc<RwLock<CiMap<&'static str>>>,
     /// Embedded PHP stubs for built-in constants (e.g. `PHP_EOL`,
     /// `SORT_ASC`, …).  Maps constant name → raw PHP source code.
     ///
     /// Built once during construction via [`stubs::build_stub_constant_index`].
     /// Filtered at startup via [`set_php_version`](Self::set_php_version) to
     /// remove stubs that do not exist in the target PHP version.
     /// Can be consulted when resolving standalone constant references.
-    pub(crate) stub_constant_index: RwLock<HashMap<&'static str, &'static str>>,
+    pub(crate) stub_constant_index: Arc<RwLock<HashMap<&'static str, &'static str>>>,
     /// Diagnostic debouncing state and the pull-model diagnostic caches.
     pub(crate) diag: crate::diagnostics::state::DiagnosticState,
     /// PHPStan's dedicated background worker state (extremely slow and
@@ -815,9 +815,11 @@ impl Backend {
             phar_archives: Arc::new(RwLock::new(HashMap::new())),
             parsed_uris: Arc::new(RwLock::new(HashSet::new())),
             parse_inflight: Arc::new(resolution::ParseInflight::new()),
-            stub_index: RwLock::new(CiMap::from(stubs::build_stub_class_index())),
-            stub_function_index: RwLock::new(CiMap::from(stubs::build_stub_function_index())),
-            stub_constant_index: RwLock::new(stubs::build_stub_constant_index()),
+            stub_index: Arc::new(RwLock::new(CiMap::from(stubs::build_stub_class_index()))),
+            stub_function_index: Arc::new(RwLock::new(CiMap::from(
+                stubs::build_stub_function_index(),
+            ))),
+            stub_constant_index: Arc::new(RwLock::new(stubs::build_stub_constant_index())),
             resolved_class_cache: virtual_members::new_resolved_class_cache(),
             auth_user_type_cache: Arc::new(RwLock::new(HashMap::new())),
             laravel_aliases: Arc::new(RwLock::new(None)),
@@ -898,9 +900,9 @@ impl Backend {
             phar_archives: Arc::new(RwLock::new(HashMap::new())),
             parsed_uris: Arc::new(RwLock::new(HashSet::new())),
             parse_inflight: Arc::new(resolution::ParseInflight::new()),
-            stub_index: RwLock::new(CiMap::new()),
-            stub_function_index: RwLock::new(CiMap::new()),
-            stub_constant_index: RwLock::new(HashMap::new()),
+            stub_index: Arc::new(RwLock::new(CiMap::new())),
+            stub_function_index: Arc::new(RwLock::new(CiMap::new())),
+            stub_constant_index: Arc::new(RwLock::new(HashMap::new())),
             resolved_class_cache: virtual_members::new_resolved_class_cache(),
             auth_user_type_cache: Arc::new(RwLock::new(HashMap::new())),
             laravel_aliases: Arc::new(RwLock::new(None)),
@@ -1004,7 +1006,7 @@ impl Backend {
     pub fn new_test_with_stubs(stub_index: HashMap<&'static str, &'static str>) -> Self {
         virtual_members::phpdoc::clear_mixin_cache();
         let backend = Self {
-            stub_index: RwLock::new(CiMap::from(stub_index)),
+            stub_index: Arc::new(RwLock::new(CiMap::from(stub_index))),
             ..Self::test_defaults()
         };
         backend.set_php_version(backend.php_version());
@@ -1023,9 +1025,9 @@ impl Backend {
     ) -> Self {
         virtual_members::phpdoc::clear_mixin_cache();
         let backend = Self {
-            stub_index: RwLock::new(CiMap::from(stub_index)),
-            stub_function_index: RwLock::new(CiMap::from(stub_function_index)),
-            stub_constant_index: RwLock::new(stub_constant_index),
+            stub_index: Arc::new(RwLock::new(CiMap::from(stub_index))),
+            stub_function_index: Arc::new(RwLock::new(CiMap::from(stub_function_index))),
+            stub_constant_index: Arc::new(RwLock::new(stub_constant_index)),
             ..Self::test_defaults()
         };
         backend.set_php_version(backend.php_version());
@@ -1515,7 +1517,7 @@ impl Backend {
             phar_archives: Arc::clone(&self.phar_archives),
             parsed_uris: Arc::clone(&self.parsed_uris),
             parse_inflight: Arc::clone(&self.parse_inflight),
-            stub_index: RwLock::new(self.stub_index.read().clone()),
+            stub_index: Arc::clone(&self.stub_index),
             resolved_class_cache: Arc::clone(&self.resolved_class_cache),
             auth_user_type_cache: Arc::clone(&self.auth_user_type_cache),
             laravel_aliases: Arc::clone(&self.laravel_aliases),
@@ -1534,8 +1536,8 @@ impl Backend {
             laravel_string_key_cache: Arc::clone(&self.laravel_string_key_cache),
             schema_index: Arc::clone(&self.schema_index),
             member_completion_cache: Arc::clone(&self.member_completion_cache),
-            stub_function_index: RwLock::new(self.stub_function_index.read().clone()),
-            stub_constant_index: RwLock::new(self.stub_constant_index.read().clone()),
+            stub_function_index: Arc::clone(&self.stub_function_index),
+            stub_constant_index: Arc::clone(&self.stub_constant_index),
             diag: self.diag.clone(),
             workspace: self.workspace.clone(),
             phpstan_tool: self.phpstan_tool.clone(),