feat(scala): recognize secondary constructors as kind=constructor (WI-rupum)

jgstern-agent · jgstern-agent · commit 6773cd33aed2 · 2026-04-11T12:02:54.000-04:00
Scala secondary constructors (def this(args) = this(...)) were
being extracted as kind="method" by the function_definition handler
in scala.py. They surfaced as the top-ranked dead-code-maybe
candidates on Kafka (CachedPartition.this, KafkaConfig.this,
FullFetchContext.this, OffsetTruncationState.this, FetchManager.this
at cross_language_hits=6192) because nothing in the static call
graph textually reaches them — Scala's constructor semantics
dispatch secondary constructors via ``new ClassName(args)`` at
construction time, outside the textual call graph.

Fix: when the function_definition's identifier text is ``this`` AND
an enclosing class/object/trait is present, extract it as
kind="constructor" with full_name="&lt;Type&gt;.this". dead-code-maybe's
kind filter (``kind in ("function","method")``) now skips it
automatically. Also set is_exported=True as a secondary safety net
so the --seeds exports mode treats secondary constructors as
reachable if anything downstream looks at the flag.

Tests (3 in TestScalaSecondaryConstructor):
- secondary_constructor_kind_and_exported: two secondary ctors on
  the same class get kind="constructor" and is_exported=True.
- secondary_constructor_not_marked_method: confirms the secondary
  ctor is NOT in the method symbol list alongside real methods.
- top_level_this_not_constructor: plain top-level function is
  unaffected (no enclosing type → no special case).

Refs: WI-rupum-dabar-lodad-rubij-bapad-biris-rarob-mufur
Related: WI-tubot prospector 2026-04-11, WI-zimum Phase 1+2

Signed-off-by: jgstern-agent &lt;josh-agent@iterabloom.com&gt;
diff --git a/.ci/affected-tests.txt b/.ci/affected-tests.txt
@@ -1,9 +1,9 @@
 # Test selection manifest
-# Generated by smart-test at 2026-04-11T11:42:46-04:00
+# Generated by smart-test at 2026-04-11T12:02:48-04:00
 # Mode: targeted
 # Baseline: 94a6f0ca6c69d3de637c39e8e0667dff2382edbf
-# Changed files: 29
-# Changed source files: 10
+# Changed files: 32
+# Changed source files: 11
 # Selected tests: 200
 #
 # === CHANGED_SOURCE_FILES ===
@@ -17,6 +17,7 @@ packages/hypergumbo-core/src/hypergumbo_core/supply_chain.py
 packages/hypergumbo-lang-mainstream/src/hypergumbo_lang_mainstream/js_ts.py
 packages/hypergumbo-lang-mainstream/src/hypergumbo_lang_mainstream/kotlin.py
 packages/hypergumbo-lang-mainstream/src/hypergumbo_lang_mainstream/py.py
+packages/hypergumbo-lang-mainstream/src/hypergumbo_lang_mainstream/scala.py
 # === SELECTED_TESTS ===
 packages/hypergumbo-core/tests/BRANCHES_test_compact.py
 packages/hypergumbo-core/tests/BRANCHES_test_database_query.py
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -40,6 +40,7 @@ This changelog tracks the **tool version** (package releases). The **schema vers
 
 - **Content-based `@generated` header detection** (WI-pofin): `classify_file` now scans the first 4 KiB of text-like source files for canonical generated-code headers in addition to the existing path-pattern check. Covered markers: `// @generated`, `/* @generated`, `# @generated`, `<!-- @generated -->` (JS/TS/Go/Python/Ruby/HTML/etc.); Go-stdlib `// Code generated by <tool>. DO NOT EDIT.`; and the alternate `Autogenerated`/`AUTO-GENERATED`/`automatically generated` spellings — all required to sit at a comment-line start so a mid-file prose mention of "generated" is not flagged. Content scan is gated on extension (`_CONTENT_SCAN_EXTS`, 36 text-like source suffixes) so images, archives, and other binary files are not opened. Bounded-cost: one 4 KiB read per scanned file. Follow-up for files flagged by header but not by path — e.g., repo code-gen that emits to hand-written directories.
 - **`is_generated_file` detection for TypeScript `openapi-gen/` output** (WI-vubad): extends `GENERATED_CODE_PATTERNS` in `supply_chain.py` with `(?:^|/)openapi-gen/.*\.(?:ts|tsx|js|jsx|mjs|cjs)$`. Airflow, FastAPI, and similar Python web projects ship a generated TypeScript SDK under an `openapi-gen/` directory (typical files: `request.ts`, `CancelablePromise.ts`, `ApiClient.ts`). The WI-tubot prospector run 2026-04-11 surfaced ~200 dead-code-maybe false positives per airflow run from a single `openapi-gen/requests/core/` directory; flagging those paths as generated demotes them out of the candidate ranking. Hand-written `request.ts` outside an `openapi-gen/` directory is not affected.
+- **Scala secondary constructors extracted as `kind="constructor"`** (WI-rupum): the Scala analyzer now recognises `def this(args) = this(...)` as a secondary-constructor declaration rather than a regular method. Such symbols get `kind="constructor"` (so `dead-code-maybe`'s `kind in ("function","method")` filter skips them automatically) AND `is_exported=True` for future-proofing. Addresses the WI-tubot prospector finding where `CachedPartition.this`, `KafkaConfig.this`, `FullFetchContext.this`, `OffsetTruncationState.this`, and `FetchManager.this` dominated the top-uncategorized Kafka dead-code list at cross_language_hits=6192, purely because nothing in the static call graph reaches them — Scala's constructor semantics dispatch secondary constructors via `new ClassName(args)` at construction time, outside the textual call graph.
 - **Kotlin extension functions → `Symbol.is_exported`** (WI-fuhav): the Kotlin analyzer detects extension functions (`fun Receiver.name() { }`) by walking the `function_declaration` children in order — a `user_type` child that appears before the `identifier`/name field is the distinctive marker of an extension receiver. Extension-function symbols get `is_exported=True` (they're inherently part of the public API of their receiver type) and carry the receiver text under `meta.extension_receiver`. Addresses the WI-tubot prospector finding where spring-boot's `SpringApplicationExtensions.kt#with` and similar symbols dominated the dead-code candidate list because nothing in the static call graph reaches them. Follow-up: full receiver-typed dispatch resolution (emit edges from `receiver.extMethod()` call sites to matching extension functions in scope) is a separate larger item.
 - **TypeScript/JavaScript `export` → `Symbol.is_exported`** (WI-zimum Phase 2b, WI-nimug): the js_ts analyzer now marks `Symbol.is_exported=True` for any top-level declaration appearing under an `export_statement`. Covered syntaxes: `export function`, `export class`, `export const/let/var` (all variable declarators), `export { foo, bar }` clauses (including `export { foo as bar }` where the alias is used), `export default <named_decl>`, and `export default <identifier>`. Anonymous `export default` expressions without a name (`export default () => {}`, `export default 42`) are not flagged because there is no symbol name to join on. The module pseudo-node is never flagged. Pairs with WI-gipag (Python `__all__`) to close out WI-zimum Phase 2 for the two languages with the highest dead-code-maybe false-positive rates in the WI-tubot prospector cohort.
 - **Python `__all__` → `Symbol.is_exported`** (WI-zimum Phase 2, WI-gipag): the Python analyzer now decides `is_exported` for top-level classes and functions based on the module's `__all__` list (when present) or the leading-underscore convention (when absent). When a module defines `__all__ = ["foo", "bar"]`, only symbols whose name appears in the list are flagged exported — even if their name is public. When `__all__` is absent, any top-level (col_offset == 0) class/function whose name doesn't start with `_` is flagged exported; nested functions (col_offset > 0) are never flagged regardless of name. Supports `__all__` as a list literal, tuple literal, or annotated assignment (`__all__: list[str] = [...]`). Non-literal forms (`__all__ = other.__all__`, list comprehensions) are treated as "no __all__" and fall back to the underscore rule. Combined with the WI-zimum `--seeds exports` dead-code-maybe mode, this drops Python framework libraries (Airflow, Django, Superset — observed at 70–83 % dead rate in the WI-tubot prospector run) out of the false-positive bucket. Follow-up: TypeScript/JavaScript top-level `export` declarations (tracked separately).
diff --git a/packages/hypergumbo-lang-mainstream/src/hypergumbo_lang_mainstream/scala.py b/packages/hypergumbo-lang-mainstream/src/hypergumbo_lang_mainstream/scala.py
@@ -354,7 +354,21 @@ def _extract_symbols_from_file(
             if name_node:
                 func_name = node_text(name_node, source)
                 enclosing_type = _get_enclosing_type(node, source)
-                if enclosing_type:
+                # WI-rupum: Scala secondary constructors are parsed as
+                # ``function_definition`` with identifier text "this" —
+                # ``def this(arg) = this(...)``. These are not methods;
+                # they're constructors, invoked by ``new ClassName(arg)``.
+                # Without this special case, the WI-tubot prospector
+                # surfaced them (e.g. CachedPartition.this, KafkaConfig.this)
+                # as top-ranked dead-code candidates, because the static
+                # call graph never reaches them.
+                is_secondary_ctor = (
+                    func_name == "this" and enclosing_type is not None
+                )
+                if is_secondary_ctor:
+                    full_name = f"{enclosing_type}.this"
+                    kind = "constructor"
+                elif enclosing_type:
                     full_name = f"{enclosing_type}.{func_name}"
                     kind = "method"
                 else:
@@ -374,6 +388,12 @@ def _extract_symbols_from_file(
                     kind, norm_sig, visibility_from_modifiers(modifiers),
                 ) if norm_sig else None
 
+                # WI-rupum: secondary constructors are inherently part
+                # of the public API of their enclosing class (something
+                # calls them via ``new``) — mark is_exported=True so
+                # dead-code-maybe's --seeds exports mode treats them
+                # as reachable. The constructor kind ALSO excludes them
+                # from the dead-code candidate list at the kind filter.
                 symbol = Symbol(
                     id=make_symbol_id("scala", str(file_path), start_line, end_line, full_name, kind),
                     name=full_name,
@@ -392,6 +412,7 @@ def _extract_symbols_from_file(
                     signature=signature,
                     modifiers=modifiers,
                     meta=meta,
+                    is_exported=is_secondary_ctor,
                 )
                 analysis.symbols.append(symbol)
                 analysis.node_for_symbol[symbol.id] = node
diff --git a/packages/hypergumbo-lang-mainstream/tests/test_scala.py b/packages/hypergumbo-lang-mainstream/tests/test_scala.py
@@ -97,6 +97,75 @@ def helper(x: Int): Int = {
         assert "main" in func_names
         assert "helper" in func_names
 
+class TestScalaSecondaryConstructor:
+    """WI-rupum: Scala secondary constructors (``def this(args) = ...``)
+    must be extracted as ``kind="constructor"`` with ``is_exported=True``
+    so they don't surface as dead-code false positives.
+    """
+
+    def test_secondary_constructor_kind_and_exported(
+        self, tmp_path: Path,
+    ) -> None:
+        from hypergumbo_lang_mainstream.scala import analyze_scala
+
+        scala_file = tmp_path / "Foo.scala"
+        scala_file.write_text(
+            "class Foo(val x: Int) {\n"
+            "  def this() = this(0)\n"
+            "  def this(s: String) = this(s.toInt)\n"
+            "}\n",
+        )
+        result = analyze_scala(tmp_path)
+        ctors = [s for s in result.symbols if s.kind == "constructor"]
+        assert len(ctors) == 2
+        for ctor in ctors:
+            assert ctor.name == "Foo.this"
+            assert ctor.is_exported is True
+
+    def test_secondary_constructor_not_marked_method(
+        self, tmp_path: Path,
+    ) -> None:
+        """The secondary constructor must NOT appear among method symbols."""
+        from hypergumbo_lang_mainstream.scala import analyze_scala
+
+        scala_file = tmp_path / "Bar.scala"
+        scala_file.write_text(
+            "class Bar(val n: Int) {\n"
+            "  def this() = this(0)\n"
+            "  def helper(): Int = n + 1\n"
+            "}\n",
+        )
+        result = analyze_scala(tmp_path)
+        methods = [s for s in result.symbols if s.kind == "method"]
+        method_names = {m.name for m in methods}
+        # helper is a method, but "this" is NOT.
+        assert "Bar.helper" in method_names
+        assert "Bar.this" not in method_names
+
+    def test_top_level_this_not_constructor(self, tmp_path: Path) -> None:
+        """A function named ``this`` at top level (no enclosing class) is
+        not a secondary constructor and should stay kind='function'.
+        Scala syntactically forbids this in practice, but the extractor
+        must not treat any ``def this`` as a constructor when there's
+        no enclosing type."""
+        from hypergumbo_lang_mainstream.scala import analyze_scala
+
+        # We construct this by wrapping in an object so the tree-sitter
+        # parse is valid. The "this" method would be enclosed by the
+        # object, so enclosing_type != None and is_secondary_ctor=True.
+        # Instead, verify a plain top-level function is unaffected.
+        scala_file = tmp_path / "Top.scala"
+        scala_file.write_text(
+            "def regular(): Int = 42\n",
+        )
+        result = analyze_scala(tmp_path)
+        funcs = [s for s in result.symbols if s.kind == "function"]
+        assert any(f.name == "regular" for f in funcs)
+        # No constructor kind anywhere.
+        ctors = [s for s in result.symbols if s.kind == "constructor"]
+        assert ctors == []
+
+
 class TestScalaClassExtraction:
     """Tests for extracting Scala classes."""
 
