repowise-dev
diff --git a/‎README.md‎
Lines changed: 4 additions & 7 deletions b/‎README.md‎
Lines changed: 4 additions & 7 deletions
diff --git a/‎docs/LANGUAGE_SUPPORT.md‎
Lines changed: 311 additions & 0 deletions b/‎docs/LANGUAGE_SUPPORT.md‎
Lines changed: 311 additions & 0 deletions
diff --git a/‎docs/architecture/ARCHITECTURE.md‎
Lines changed: 22 additions & 10 deletions b/‎docs/architecture/ARCHITECTURE.md‎
Lines changed: 22 additions & 10 deletions
@@ -65,7 +65,7 @@ There's a small genre of "token efficiency" benchmarks going around. It would be
 repowise runs once, builds everything, then keeps it in sync on every commit.
 
 ### ◈ Graph Intelligence
-tree-sitter parses every file into symbols. NetworkX builds a two-tier dependency graph — file nodes for module-level relationships and symbol nodes (functions, classes, methods) for fine-grained call resolution. A 3-tier resolver links call sites to their targets with confidence scoring. Import aliases, barrel re-exports, and namespace imports all resolve correctly. Inheritance hierarchies are extracted across 11 languages (extends, implements, trait impls, mixins) and resolved to concrete symbol edges. Leiden community detection identifies logical modules at both file and symbol level — with cohesion scoring and heuristic labeling — even when your directory structure doesn't reflect them. Execution flow tracing discovers call paths from entry points through your codebase, classifying them as intra- or cross-community. PageRank, betweenness centrality, and SCC analysis identify your most central and most coupled code.
+tree-sitter parses every file across 14 languages into a two-tier dependency graph — file nodes and symbol nodes (functions, classes, methods). A 3-tier call resolver with confidence scoring handles import aliases, barrel re-exports, and namespace imports. Heritage extraction covers extends, implements, trait impls, derive macros, mixins, and extension conformance. Leiden community detection finds logical modules even when your directory structure doesn't reflect them. PageRank, betweenness centrality, SCC analysis, and execution flow tracing from entry points identify your most central, most coupled, and most traversed code.
 
 ### ◈ Git Intelligence
 500 commits of history turned into signals: hotspot files (high churn × high complexity), ownership percentages per engineer, co-change pairs (files that change together without an import link — hidden coupling), and significant commit messages that explain *why* code evolved.
@@ -543,14 +543,11 @@ repowise reindex                  # rebuild vector store (no LLM calls)
 
 | Tier | Languages | What works |
 |------|-----------|------------|
-| **Full** | Python · TypeScript · JavaScript · Java · Go · Rust | AST parsing, import resolution, named bindings, dependency graph edges, call resolution, heritage extraction |
-| **Partial** | C++ | AST parsing, symbol extraction, heritage extraction, `compile_commands.json` import resolution |
-| **Partial** | C | AST parsing (structs, functions, macros), `#include` resolution with `compile_commands.json` |
-| **Scaffolded** | Kotlin · Ruby | Tree-sitter queries and heritage extractors exist but grammars not yet wired — install `tree-sitter-kotlin` / `tree-sitter-ruby` separately |
-| **Traversal** | C# · Swift · Scala · PHP | Files indexed and searchable, but no AST symbol extraction yet |
+| **Full** | Python · TypeScript · JavaScript · Java · Go · Rust · C++ | AST parsing, import resolution, named bindings, call resolution, heritage extraction, docstrings |
+| **Good** | C · Kotlin · Ruby · C# · Swift · Scala · PHP | AST parsing, import resolution, named bindings, call resolution, heritage (mixins, derive, extensions, traits), docstrings, dedicated resolvers |
 | **Config / data** | OpenAPI · Protobuf · GraphQL · Dockerfile · Makefile · YAML · JSON · TOML · SQL · Terraform | Included in the file tree; special handlers extract endpoints/targets where applicable |
 
-Kotlin, Ruby, C#, Swift, Scala, PHP, Dart, and Elixir are on the [language support roadmap](docs/LANGUAGE_SUPPORT_PLAN.md). Adding a new language requires one `.scm` tree-sitter query file and one config entry. No changes to the parser core. PRs welcome. See [Adding a new language](docs/CONTRIBUTING.md#adding-a-new-language).
+14 languages with full AST support. Adding a new language requires one `.scm` tree-sitter query file and one config entry. No changes to the parser core. See [Language Support](docs/LANGUAGE_SUPPORT.md) for details.
 
 ---
 
 
@@ -0,0 +1,311 @@
+# Language Support
+
+repowise analyses codebases written in many languages. Each language goes
+through a multi-stage pipeline: file traversal, AST parsing, import
+resolution, call graph construction, and heritage (inheritance) extraction.
+Not every language has reached full coverage yet --- this page documents
+exactly what works today, what is coming next, and how to add a new language.
+
+---
+
+## Support Tiers
+
+Every language falls into one of five tiers. The tier determines which
+pipeline stages produce meaningful output.
+
+| Stage | Full | Partial | Scaffolded | Traversal | Config / Data |
+|-------|:----:|:-------:|:----------:|:---------:|:-------------:|
+| File discovery & git history | Y | Y | Y | Y | Y |
+| AST symbol extraction | Y | Y | -- | -- | -- |
+| Import resolution | Y | Y | -- | -- | -- |
+| Call graph edges | Y | partial | -- | -- | -- |
+| Heritage (extends/implements) | Y | partial | -- | -- | -- |
+| Named bindings | Y | -- | -- | -- | -- |
+| Dead code detection | Y | Y | Y | Y | -- |
+| Semantic search & wiki pages | Y | Y | Y | Y | Y |
+
+---
+
+## Language Reference
+
+### Full
+
+Languages with complete pipeline coverage: AST parsing, import resolution,
+call resolution, named bindings, heritage extraction, and docstrings.
+
+| Language | Extensions | Entry Points | Import Style |
+|----------|-----------|-------------|-------------|
+| **Python** | `.py` `.pyi` | `main.py` `app.py` `__main__.py` `manage.py` `wsgi.py` `asgi.py` | `import x` / `from x import y` |
+| **TypeScript** | `.ts` `.tsx` | `index.ts` `main.ts` `app.ts` `server.ts` | `import { x } from 'y'` / `require()` |
+| **JavaScript** | `.js` `.jsx` `.mjs` `.cjs` | `index.js` `main.js` `app.js` `server.js` | `import` / `require()` |
+| **Java** | `.java` | `Main.java` `Application.java` | `import pkg.Class` |
+| **Go** | `.go` | `main.go` `cmd/main.go` | `import "path"` with `go.mod` resolution |
+| **Rust** | `.rs` | `main.rs` `lib.rs` | `use crate::` / `use super::` / `use self::` with `Cargo.toml` |
+| **C++** | `.cpp` `.cc` `.cxx` `.h` `.hpp` `.hxx` | `main.cpp` `main.cc` | `#include` with `compile_commands.json` resolution |
+
+All seven languages support:
+- Tree-sitter AST parsing with dedicated `.scm` query files
+- Three-tier call resolution (same-file, cross-file, global stem match)
+- Named binding extraction (mapping imported names to source symbols)
+- Heritage extraction (class/interface/trait inheritance chains)
+- Docstring extraction (Python, JSDoc, GoDoc, Rustdoc, Javadoc, Doxygen)
+- Framework-aware edges (Django, FastAPI, Flask for Python; tsconfig path aliases for TS/JS; pytest fixture detection)
+
+### Good
+
+AST parsing, symbol extraction, import resolution, call resolution, named
+bindings, heritage extraction (including Ruby mixins, Rust derive, Swift
+extension conformance, PHP trait use), and docstrings. Dedicated import
+resolvers for each language.
+
+| Language | Extensions | Entry Points | Import Style |
+|----------|-----------|-------------|-------------|
+| **C** | `.c` | `main.c` | `#include` with `compile_commands.json` (shares C++ grammar) |
+| **Kotlin** | `.kt` `.kts` | `Main.kt` `Application.kt` | `import com.example.Foo` |
+| **Ruby** | `.rb` | `main.rb` `app.rb` `config.ru` | `require 'mod'` / `require_relative './mod'` |
+| **C#** | `.cs` | `Program.cs` `Startup.cs` | `using System.Collections.Generic` |
+| **Swift** | `.swift` | `main.swift` `App.swift` | `import Foundation` |
+| **Scala** | `.scala` | `Main.scala` `App.scala` | `import pkg.{A, B => C}` |
+| **PHP** | `.php` | `index.php` `public/index.php` | `use Foo\Bar\Baz` |
+
+### Config / Data
+
+Non-code files included in the file tree and wiki. Special handlers extract
+endpoints or targets where applicable.
+
+| Language | Extensions / Filenames | Special Handler |
+|----------|----------------------|----------------|
+| **OpenAPI** | YAML/JSON with `openapi` or `swagger` key | Extracts API paths and schemas |
+| **Dockerfile** | `Dockerfile` | Extracts stages and exposed ports |
+| **Makefile** | `Makefile` `GNUmakefile` | Extracts targets |
+| **Protobuf** | `.proto` | -- |
+| **GraphQL** | `.graphql` `.gql` | -- |
+| **Terraform** | `.tf` `.hcl` | -- |
+| **YAML** | `.yaml` `.yml` | -- |
+| **JSON** | `.json` | -- |
+| **TOML** | `.toml` | -- |
+| **Markdown** | `.md` `.mdx` | -- |
+| **SQL** | `.sql` | -- |
+| **Shell** | `.sh` `.bash` `.zsh` | -- |
+
+### Git-Blame-Only
+
+These languages are tracked in git history (blame, hotspot analysis,
+co-change detection) but have no AST parsing or dedicated support. Files
+appear in the wiki as traversal-level entries.
+
+Objective-C, Elixir, Erlang, Lua, R, Dart, Zig, Julia, Clojure, Elm,
+Haskell, OCaml, F#, Crystal, Nim, D
+
+---
+
+## How the Pipeline Processes a File
+
+```
+File discovered by FileTraverser
+        |
+        v
+Extension/filename -> LanguageTag  (via LanguageRegistry)
+        |
+        +-- Config/data language?  -> empty ParsedFile (passthrough)
+        +-- Special format?        -> special_handlers.py (OpenAPI/Dockerfile/Makefile)
+        +-- Has grammar?           -> tree-sitter AST parsing
+                |
+                v
+        .scm query extracts:
+          @symbol.def / @symbol.name    -> Symbol nodes
+          @import.statement / @import.module -> Import edges
+          @call.target / @call.receiver     -> Call edges
+                |
+                v
+        Per-language extractors:
+          - Named bindings (import name -> source symbol)
+          - Heritage (extends/implements/traits)
+          - Docstrings (Python, JSDoc, GoDoc, Rustdoc, Javadoc)
+          - Visibility (public/private/protected)
+                |
+                v
+        GraphBuilder resolves imports:
+          Python: dotted module paths, __init__.py, src/ layout
+          TS/JS:  relative paths, tsconfig aliases, node_modules
+          Go:     go.mod module path stripping
+          Rust:   crate::/self::/super::, mod.rs probing
+          C/C++:  compile_commands.json include directories
+          Other:  stem-map fallback (filename matching)
+                |
+                v
+        Graph analysis:
+          PageRank, community detection, dead code, execution flows
+```
+
+---
+
+## Adding a New Language
+
+The pipeline is fully modular. Language identity data lives in the
+centralised `LanguageRegistry`, per-language extraction logic lives in
+`extractors/`, and per-language import resolution lives in `resolvers/`.
+Adding a new language touches these places:
+
+### Step 1: Add a `LanguageSpec` to the registry
+
+Edit `packages/core/src/repowise/core/ingestion/languages/registry.py` and
+add a new `LanguageSpec(...)` entry to the `_SPECS` tuple:
+
+```python
+LanguageSpec(
+    tag="mylang",
+    display_name="MyLang",
+    extensions=frozenset({".ml"}),
+    grammar_package="tree_sitter_mylang",       # PyPI package name
+    scm_file="mylang.scm",                      # query file name
+    heritage_node_types=frozenset({"class_declaration"}),
+    entry_point_patterns=("main.ml",),
+    manifest_files=("mylang.toml",),
+    shebang_tokens=("mylang",),
+    builtin_calls=frozenset({"print", "len"}),  # filter from call graph
+    builtin_parents=frozenset({"Object"}),       # filter from heritage
+    color_hex="#AB47BC",
+)
+```
+
+### Step 2: Add the `LanguageTag`
+
+Add `"mylang"` to the `LanguageTag` Literal type in
+`packages/core/src/repowise/core/ingestion/models.py`.
+
+### Step 3: Write a tree-sitter query file
+
+Create `packages/core/src/repowise/core/ingestion/queries/mylang.scm` using
+tree-sitter S-expression syntax. Follow the capture-name conventions:
+
+| Capture | Purpose | Required? |
+|---------|---------|-----------|
+| `@symbol.def` | Full definition node (line numbers, kind lookup) | Yes |
+| `@symbol.name` | Name identifier | Yes |
+| `@symbol.params` | Parameter list | No |
+| `@symbol.modifiers` | Decorators / visibility modifiers | No |
+| `@symbol.receiver` | Go-style method receiver | No |
+| `@import.statement` | Full import node | Yes |
+| `@import.module` | Module path being imported | Yes |
+| `@call.target` | Function/method being called | No (enables call graph) |
+| `@call.receiver` | Object the call is made on | No |
+| `@call.arguments` | Call arguments | No |
+
+Look at existing `.scm` files for examples --- `python.scm` and
+`typescript.scm` are good starting points.
+
+### Step 4: Add a `LanguageConfig` entry
+
+Add a parser configuration to `LANGUAGE_CONFIGS` in
+`packages/core/src/repowise/core/ingestion/parser.py`:
+
+```python
+"mylang": LanguageConfig(
+    symbol_node_types={
+        "function_definition": "function",
+        "class_definition": "class",
+    },
+    import_node_types=["import_statement"],
+    export_node_types=[],
+    visibility_fn=public_by_default,  # from extractors.visibility
+    parent_extraction="nesting",
+    parent_class_types=frozenset({"class_definition"}),
+    entry_point_patterns=["main.ml"],
+),
+```
+
+### Step 5: Add the tree-sitter grammar dependency
+
+Add the grammar package to `pyproject.toml`:
+
+```toml
+[project]
+dependencies = [
+    # ...
+    "tree-sitter-mylang>=0.23,<1",
+]
+```
+
+### Step 6 (optional): Binding extractor
+
+For full-tier support, add a `extract_mylang_bindings()` function in
+`packages/core/src/repowise/core/ingestion/extractors/bindings.py` and
+register it in the `extract_import_bindings()` dispatcher. Without this,
+imports are still resolved but named-binding-level call resolution won't
+work.
+
+### Step 7 (optional): Heritage extractor
+
+Add a `_extract_mylang_heritage()` function in
+`packages/core/src/repowise/core/ingestion/extractors/heritage.py` and
+register it in the `HERITAGE_EXTRACTORS` dict. Without this, inheritance
+chains won't appear in the graph.
+
+### Step 8 (optional): Import resolver
+
+If the language has a non-trivial import system, create a resolver in
+`packages/core/src/repowise/core/ingestion/resolvers/mylang.py` and
+register it in the `_RESOLVERS` dict in `resolvers/__init__.py`. For simple
+languages, the generic stem-map fallback (matching by filename) works out
+of the box.
+
+### Verify
+
+```bash
+# Run the parser tests
+pytest tests/ -k "mylang or sample_repo" -x
+
+# Index a real project
+repowise init /path/to/mylang-project
+```
+
+No changes are needed to `traverser.py`, `dead_code.py`,
+`page_generator.py`, `cost_estimator.py`, or any other consumer file ---
+they all derive their language sets from the registry automatically.
+
+---
+
+## Architecture
+
+The language pipeline is fully modular:
+
+```
+ingestion/
+  languages/           # LanguageRegistry + LanguageSpec (identity data)
+  extractors/          # Per-language AST extraction
+    visibility.py      #   symbol visibility (public/private/protected)
+    signatures.py      #   human-readable signature building
+    docstrings.py      #   module + symbol docstring extraction
+    bindings.py        #   import name + alias binding extraction
+    heritage.py        #   inheritance/interface/trait extraction
+  resolvers/           # Per-language import resolution
+    python.py          #   dotted imports, __init__.py, src/ layout
+    typescript.py      #   multi-ext probe, tsconfig aliases
+    go.py              #   go.mod module path stripping
+    rust.py            #   crate::/self::/super::, mod.rs probing
+    cpp.py             #   compile_commands.json include paths
+    kotlin.py          #   package-to-directory mapping
+    ruby.py            #   require/require_relative resolution
+    csharp.py          #   namespace-based resolution
+    swift.py           #   module import resolution
+    scala.py           #   package-to-directory mapping
+    php.py             #   namespace/PSR-4 resolution
+    generic.py         #   stem-matching fallback
+  framework_edges.py   # Django, FastAPI, Flask, pytest detection
+  parser.py            # ASTParser (language-agnostic orchestration)
+  graph.py             # GraphBuilder (import/call/heritage resolution)
+```
+
+Adding a new language requires zero changes to `parser.py`, `graph.py`,
+`traverser.py`, `dead_code.py`, or any other core file.
+
+---
+
+## Roadmap
+
+| Language | Target Tier | Status |
+|----------|------------|--------|
+| Dart | Good | Planned — `tree-sitter-dart` available |
+| Elixir | Good | Planned — `tree-sitter-elixir` available |
+| F# | Good | Planned — `tree-sitter-f-sharp` available |
@@ -163,7 +163,11 @@ repowise/
 │   │       ├── cpp.scm
 │   │       ├── c.scm
 │   │       ├── ruby.scm
-│   │       └── kotlin.scm
+│   │       ├── kotlin.scm
+│   │       ├── csharp.scm
+│   │       ├── swift.scm
+│   │       ├── scala.scm
+│   │       └── php.scm
 │   │
 │   ├── server/                 # Python: FastAPI REST API + MCP server
 │   │   └── src/repowise/server/
@@ -506,8 +510,8 @@ which runs after the static import graph is built. It operates in three tiers:
    from the file's import list
 3. **Global unique match** (confidence 0.50) — target is unique across the whole repo
 
-Call sites are extracted by tree-sitter for all 7 supported languages (Python, TypeScript,
-JavaScript, Go, Rust, Java, C++) using per-language `.scm` query files. Results are stored
+Call sites are extracted by tree-sitter for all 14 supported languages (Python, TypeScript,
+JavaScript, Go, Rust, Java, C++, C, Kotlin, Ruby, C#, Swift, Scala, PHP) using per-language `.scm` query files. Results are stored
 as `CallSite` dataclasses and become `CALLS` edges in the graph.
 
 **Named binding resolution** (`NamedBinding` dataclass in `ingestion/models.py`) ensures
@@ -1547,19 +1551,27 @@ Key files:
    ),
    ```
 
-3. **Verify `tree-sitter-languages` includes your language**
+3. **Add a `LanguageSpec` to `LanguageRegistry`** in `ingestion/languages/registry.py`
 
-   Run `python -c "from tree_sitter_languages import get_parser; get_parser('mylang')"`.
-   If it raises, the grammar is not bundled. You can add it manually as a tree-sitter
-   grammar and register it with the Language API.
+   This registers the language's identity data (extensions, entry points, manifest files,
+   builtin calls, heritage node types, etc.) centrally.
 
-4. **Add test files to `tests/fixtures/sample_repo/`**
+4. **Add the grammar dependency to `pyproject.toml`**
+
+   ```toml
+   "tree-sitter-mylang>=0.23,<1",
+   ```
+
+5. **Add test files to `tests/fixtures/sample_repo/`**
 
    At minimum: one file with a function, one with a class, one with imports.
 
-5. **Run `pytest tests/unit/test_parser.py -k mylang`** to verify extraction.
+6. **(Optional) Add per-language extractors** for bindings, heritage, visibility, docstrings,
+   and a dedicated import resolver in `resolvers/mylang.py`.
+
+7. **Run `pytest tests/unit/test_parser.py -k mylang`** to verify extraction.
 
-6. **Open a PR.** That's it — no other changes needed.
+8. **Open a PR.** That's it — no other changes needed.
 
 ---