Skip to content

Commit 3ee9d5d

Browse files
committed
Drop the nim grammar and refresh language counts
Removes the heaviest vendored grammar (66 MB) along with its language wiring and test fixtures; the supported-language count moves to 158 across README, site, and manifest. The grammar manifest additionally records the canonical-source decisions for the five registry-disagreement grammars and the per-directory license files restored earlier this week.
1 parent 630bd40 commit 3ee9d5d

20 files changed

Lines changed: 1084 additions & 1644955 deletions

README.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
[![GitHub Release](https://img.shields.io/github/v/release/DeusData/codebase-memory-mcp?style=flat&color=blue)](https://github.com/DeusData/codebase-memory-mcp/releases/latest)
44
[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
55
[![CI](https://img.shields.io/github/actions/workflow/status/DeusData/codebase-memory-mcp/dry-run.yml?label=CI)](https://github.com/DeusData/codebase-memory-mcp/actions/workflows/dry-run.yml)
6-
[![Tests](https://img.shields.io/badge/tests-5577_passing-brightgreen)](https://github.com/DeusData/codebase-memory-mcp)
7-
[![Languages](https://img.shields.io/badge/languages-159-orange)](https://github.com/DeusData/codebase-memory-mcp)
6+
[![Tests](https://img.shields.io/badge/tests-5604_passing-brightgreen)](https://github.com/DeusData/codebase-memory-mcp)
7+
[![Languages](https://img.shields.io/badge/languages-158-orange)](https://github.com/DeusData/codebase-memory-mcp)
88
[![Hybrid LSP](https://img.shields.io/badge/Hybrid_LSP-9_languages-blue)](#hybrid-lsp)
99
[![Agents](https://img.shields.io/badge/agents-11-purple)](https://github.com/DeusData/codebase-memory-mcp)
1010
[![Pure C](https://img.shields.io/badge/pure_C-zero_dependencies-blue)](https://github.com/DeusData/codebase-memory-mcp)
@@ -16,7 +16,7 @@
1616

1717
**The fastest and most efficient code intelligence engine for AI coding agents.** Full-indexes an average repository in milliseconds, the Linux kernel (28M LOC, 75K files) in 3 minutes. Answers structural queries in under 1ms. Ships as a single static binary for macOS, Linux, and Windows — download, run `install`, done.
1818

19-
High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 159 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, and Rust — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 14 MCP tools. Zero dependencies. Plug and play across 11 coding agents.
19+
High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 158 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, and Rust — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 14 MCP tools. Zero dependencies. Plug and play across 11 coding agents.
2020

2121
> **Research** — The design and benchmarks behind this project are described in the preprint [*Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP*](https://arxiv.org/abs/2603.27277) (arXiv:2603.27277). Evaluated across 31 real-world repositories: 83% answer quality, 10× fewer tokens, 2.1× fewer tool calls vs. file-by-file exploration.
2222
@@ -32,7 +32,7 @@ High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-si
3232

3333
- **Extreme indexing speed** — Linux kernel (28M LOC, 75K files) in 3 minutes. RAM-first pipeline: LZ4 compression, in-memory SQLite, fused Aho-Corasick pattern matching. Memory released after indexing.
3434
- **Plug and play** — single static binary for macOS (arm64/amd64), Linux (arm64/amd64), and Windows (amd64). No Docker, no runtime dependencies, no API keys. Download → `install` → restart agent → done.
35-
- **159 languages** — vendored tree-sitter grammars compiled into the binary. Nothing to install, nothing that breaks.
35+
- **158 languages** — vendored tree-sitter grammars compiled into the binary. Nothing to install, nothing that breaks.
3636
- **120x fewer tokens** — 5 structural queries: ~3,400 tokens vs ~412,000 via file-by-file search. One graph query replaces dozens of grep/read cycles.
3737
- **11 agents, one command**`install` auto-detects Claude Code, Codex CLI, Gemini CLI, Zed, OpenCode, Antigravity, Aider, KiloCode, VS Code, OpenClaw, and Kiro — configures MCP entries, instruction files, and pre-tool hooks for each.
3838
- **Built-in graph visualization** — 3D interactive UI at `localhost:9749` (optional UI binary variant).
@@ -168,7 +168,7 @@ Removes all agent configs, skills, hooks, and instructions. Does not remove the
168168
- `SEMANTICALLY_RELATED` (vocabulary-mismatch, same-language, score ≥ 0.80)
169169

170170
### Indexing pipeline
171-
- **159 vendored tree-sitter grammars** compiled into the binary
171+
- **158 vendored tree-sitter grammars** compiled into the binary
172172
- **Generic package / module resolution** — bare specifiers like `@myorg/pkg`, `github.com/foo/bar`, `use my_crate::foo` resolved via manifest scanning (`package.json`, `go.mod`, `Cargo.toml`, `pyproject.toml`, `composer.json`, `pubspec.yaml`, `pom.xml`, `build.gradle`, `mix.exs`, `*.gemspec`)
173173
- **Infrastructure-as-code indexing** — Dockerfiles, Kubernetes manifests, Kustomize overlays as graph nodes
174174
- **[Hybrid LSP semantic type resolution](#hybrid-lsp)** for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, and Rust — a lightweight C implementation of language type-resolution algorithms, structurally inspired by and compatible with major language servers including tsserver / typescript-go, pyright, gopls, Roslyn, Eclipse JDT, and rust-analyzer (parameter binding, return-type inference, generic substitution, JSX component dispatch, JSDoc inference for plain JS files, namespace + trait + late-static-binding resolution for PHP, file-scoped namespaces + records + LINQ method syntax for C#, class-hierarchy + overload + lambda resolution for Java, extension-function + scope-function resolution for Kotlin, trait-method + UFCS resolution for Rust)
@@ -508,14 +508,14 @@ codebase-memory-mcp ships a **lightweight C implementation of language type-reso
508508

509509
**Two-layer architecture:**
510510

511-
1. **Tree-sitter pass** — fast, syntactic, runs for every one of the 159 languages. Extracts definitions, calls, imports.
511+
1. **Tree-sitter pass** — fast, syntactic, runs for every one of the 158 languages. Extracts definitions, calls, imports.
512512
2. **Hybrid LSP pass** — type-aware, runs above the tree-sitter pass per-language. Refines call edges using the import graph plus a per-file or pre-built cross-file definition registry. Languages without a Hybrid LSP pass yet fall back to textual resolution, so you always get *some* answer.
513513

514514
The result is a knowledge graph accurate enough to drive `trace_path` across packages, inheritance hierarchies, and stdlib calls — without paying for a language server process per project.
515515

516516
## Language Support
517517

518-
159 languages, all parsed via vendored tree-sitter grammars compiled into the binary. Benchmarked against 64 real open-source repositories (78 to 49K nodes):
518+
158 languages, all parsed via vendored tree-sitter grammars compiled into the binary. Benchmarked against 64 real open-source repositories (78 to 49K nodes):
519519

520520
| Tier | Score | Languages |
521521
|------|-------|-----------|
@@ -540,7 +540,7 @@ src/
540540
traces/ Runtime trace ingestion
541541
ui/ Embedded HTTP server + 3D graph visualization
542542
foundation/ Platform abstractions (threads, filesystem, logging, memory)
543-
internal/cbm/ Vendored tree-sitter grammars (159 languages) + AST extraction engine
543+
internal/cbm/ Vendored tree-sitter grammars (158 languages) + AST extraction engine
544544
```
545545

546546
## Security

THIRD_PARTY.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ The core runtime headers in `internal/cbm/vendored/common/tree_sitter/`
2626

2727
## Tree-sitter Grammars
2828

29-
159 pre-generated parsers are vendored in `internal/cbm/vendored/grammars/<lang>/`
29+
158 pre-generated parsers are vendored in `internal/cbm/vendored/grammars/<lang>/`
3030
(generated `parser.c` plus `scanner.c` where applicable, compiled statically).
3131
Each grammar is the work of its upstream authors and each grammar directory
3232
contains the upstream `LICENSE` file.
@@ -50,7 +50,6 @@ License summary:
5050
|---------|------|---------|---------|
5151
| SQLite 3 | `vendored/sqlite3/` | Public Domain | [sqlite.org](https://www.sqlite.org/) |
5252
| mimalloc | `vendored/mimalloc/` | MIT | [microsoft/mimalloc](https://github.com/microsoft/mimalloc) |
53-
| Mongoose | `vendored/mongoose/` | GPL-2.0-only (dual-licensed: GPLv2 or commercial) | [cesanta/mongoose](https://github.com/cesanta/mongoose) |
5453
| yyjson | `vendored/yyjson/` | MIT | [ibireme/yyjson](https://github.com/ibireme/yyjson) |
5554
| xxHash | `vendored/xxhash/` | BSD-2-Clause | [Cyan4973/xxHash](https://github.com/Cyan4973/xxHash) |
5655
| TRE | `vendored/tre/` | BSD-2-Clause | [laurikari/tre](https://github.com/laurikari/tre) |
@@ -60,8 +59,9 @@ License summary:
6059
| Verstable | `internal/cbm/vendored/verstable/` | MIT | [JacksonAllan/Verstable](https://github.com/JacksonAllan/Verstable) |
6160
| wyhash | `internal/cbm/vendored/wyhash/` | Unlicense (public domain) | [wangyi-fudan/wyhash](https://github.com/wangyi-fudan/wyhash) |
6261

63-
Mongoose powers the embedded graph-UI HTTP server (`src/ui/http_server.c`)
64-
and is dual-licensed by Cesanta Software Limited.
62+
The graph-UI HTTP server is a first-party implementation
63+
(`src/ui/httpd.c` + `src/ui/http_server.c`) — no third-party HTTP library
64+
is used.
6565

6666
## Embedded Model Data
6767

docs/index.html

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,15 @@
44
<meta charset="UTF-8">
55
<meta name="viewport" content="width=device-width, initial-scale=1.0">
66
<title>codebase-memory-mcp — Code Intelligence Knowledge Graph for AI Coding Agents</title>
7-
<meta name="description" content="codebase-memory-mcp is an open-source MCP server that indexes any codebase into a persistent knowledge graph so AI coding agents answer structural questions with ~120x fewer tokens. 159 languages, Hybrid LSP type resolution, local semantic vector search, code-clone detection, sub-1ms queries, Linux kernel indexed in 3 minutes. Single static C binary, zero dependencies. Works with 11 agents including Claude Code, Codex CLI, Gemini CLI, Cursor, and Zed.">
7+
<meta name="description" content="codebase-memory-mcp is an open-source MCP server that indexes any codebase into a persistent knowledge graph so AI coding agents answer structural questions with ~120x fewer tokens. 158 languages, Hybrid LSP type resolution, local semantic vector search, code-clone detection, sub-1ms queries, Linux kernel indexed in 3 minutes. Single static C binary, zero dependencies. Works with 11 agents including Claude Code, Codex CLI, Gemini CLI, Cursor, and Zed.">
88
<meta name="keywords" content="MCP server, code intelligence, knowledge graph, tree-sitter, Hybrid LSP, semantic code search, code embeddings, code clone detection, cross-repo analysis, data-flow analysis, Claude Code, Codex CLI, Gemini CLI, Cursor, Zed, code exploration, token reduction, call graph, dead code detection">
99
<meta name="author" content="DeusData">
1010
<meta name="robots" content="index, follow, max-image-preview:large, max-snippet:-1">
1111
<meta name="theme-color" content="#0d1117">
1212

1313
<!-- Open Graph -->
1414
<meta property="og:title" content="codebase-memory-mcp — Code Intelligence Knowledge Graph for AI Coding Agents">
15-
<meta property="og:description" content="Index any codebase into a persistent knowledge graph. AI agents answer structural questions with ~120x fewer tokens. 159 languages, Hybrid LSP, sub-1ms queries, single static C binary.">
15+
<meta property="og:description" content="Index any codebase into a persistent knowledge graph. AI agents answer structural questions with ~120x fewer tokens. 158 languages, Hybrid LSP, sub-1ms queries, single static C binary.">
1616
<meta property="og:type" content="website">
1717
<meta property="og:url" content="https://deusdata.github.io/codebase-memory-mcp/">
1818
<meta property="og:image" content="https://deusdata.github.io/codebase-memory-mcp/graph-ui-screenshot.png">
@@ -24,7 +24,7 @@
2424
<!-- Twitter Card -->
2525
<meta name="twitter:card" content="summary_large_image">
2626
<meta name="twitter:title" content="codebase-memory-mcp — Code Intelligence for AI Coding Agents">
27-
<meta name="twitter:description" content="Index any codebase into a knowledge graph. ~120x fewer tokens for AI code exploration. 159 languages, Hybrid LSP, sub-1ms queries, single static C binary.">
27+
<meta name="twitter:description" content="Index any codebase into a knowledge graph. ~120x fewer tokens for AI code exploration. 158 languages, Hybrid LSP, sub-1ms queries, single static C binary.">
2828
<meta name="twitter:image" content="https://deusdata.github.io/codebase-memory-mcp/graph-ui-screenshot.png">
2929

3030
<link rel="canonical" href="https://deusdata.github.io/codebase-memory-mcp/">
@@ -40,8 +40,8 @@
4040
"applicationCategory": "DeveloperApplication",
4141
"applicationSubCategory": "Model Context Protocol (MCP) server",
4242
"operatingSystem": "macOS, Linux, Windows",
43-
"softwareVersion": "0.7.0",
44-
"description": "An open-source MCP server that indexes a codebase into a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links, so AI coding agents answer structural questions with roughly 120x fewer tokens than file-by-file search. Parses 159 languages via tree-sitter with Hybrid LSP semantic type resolution for 9 language families. Ships as a single static C binary with zero runtime dependencies.",
43+
"softwareVersion": "0.8.1",
44+
"description": "An open-source MCP server that indexes a codebase into a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links, so AI coding agents answer structural questions with roughly 120x fewer tokens than file-by-file search. Parses 158 languages via tree-sitter with Hybrid LSP semantic type resolution for 9 language families. Ships as a single static C binary with zero runtime dependencies.",
4545
"url": "https://deusdata.github.io/codebase-memory-mcp/",
4646
"downloadUrl": "https://github.com/DeusData/codebase-memory-mcp/releases/latest",
4747
"codeRepository": "https://github.com/DeusData/codebase-memory-mcp",
@@ -54,7 +54,7 @@
5454
"priceCurrency": "USD"
5555
},
5656
"featureList": [
57-
"Indexes 159 programming languages via vendored tree-sitter grammars",
57+
"Indexes 158 programming languages via vendored tree-sitter grammars",
5858
"Hybrid LSP semantic type resolution for Python, TypeScript/JavaScript, PHP, C#, Go, C/C++, Java, Kotlin, and Rust",
5959
"14 MCP tools for structural search, call-path tracing, and Cypher graph queries",
6060
"Semantic vector code search via bundled nomic-embed-code embeddings (no API key, fully local)",
@@ -125,7 +125,7 @@
125125
"name": "What is codebase-memory-mcp?",
126126
"acceptedAnswer": {
127127
"@type": "Answer",
128-
"text": "codebase-memory-mcp is an open-source Model Context Protocol (MCP) server that indexes a codebase into a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. AI coding agents query that graph instead of reading files one by one, answering structural questions with roughly 120x fewer tokens. It parses 159 languages and ships as a single static C binary with no runtime dependencies."
128+
"text": "codebase-memory-mcp is an open-source Model Context Protocol (MCP) server that indexes a codebase into a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. AI coding agents query that graph instead of reading files one by one, answering structural questions with roughly 120x fewer tokens. It parses 158 languages and ships as a single static C binary with no runtime dependencies."
129129
}
130130
},
131131
{
@@ -141,7 +141,7 @@
141141
"name": "Which programming languages does codebase-memory-mcp support?",
142142
"acceptedAnswer": {
143143
"@type": "Answer",
144-
"text": "It supports 159 languages through vendored tree-sitter grammars compiled into the binary, including Python, Go, JavaScript, TypeScript, Rust, Java, C, C++, C#, PHP, Ruby, Kotlin, Swift, and many more. Nine language families — Python, TypeScript/JavaScript, PHP, C#, Go, C/C++, Java, Kotlin, and Rust — additionally get Hybrid LSP semantic type resolution."
144+
"text": "It supports 158 languages through vendored tree-sitter grammars compiled into the binary, including Python, Go, JavaScript, TypeScript, Rust, Java, C, C++, C#, PHP, Ruby, Kotlin, Swift, and many more. Nine language families — Python, TypeScript/JavaScript, PHP, C#, Go, C/C++, Java, Kotlin, and Rust — additionally get Hybrid LSP semantic type resolution."
145145
}
146146
},
147147
{
@@ -444,11 +444,11 @@ <h1>codebase-memory-mcp</h1>
444444
The fastest, most efficient code intelligence engine for AI coding agents. It indexes any
445445
repository into a persistent knowledge graph — full-indexing an average repo in seconds and the
446446
Linux kernel in 3 minutes — so your agent answers structural questions with ~120x fewer tokens.
447-
Tree-sitter parsing across 159 languages, Hybrid LSP type resolution, single static C binary.
447+
Tree-sitter parsing across 158 languages, Hybrid LSP type resolution, single static C binary.
448448
</p>
449449
<div>
450450
<div class="stat"><div class="number">~120x</div><div class="label">fewer tokens</div></div>
451-
<div class="stat"><div class="number">159</div><div class="label">languages</div></div>
451+
<div class="stat"><div class="number">158</div><div class="label">languages</div></div>
452452
<div class="stat"><div class="number">3 min</div><div class="label">Linux kernel index</div></div>
453453
<div class="stat"><div class="number">11</div><div class="label">agents supported</div></div>
454454
</div>
@@ -483,7 +483,7 @@ <h2>What is codebase-memory-mcp?</h2>
483483
Protocol (MCP)</a> server that indexes a codebase into a persistent knowledge graph of functions,
484484
classes, call chains, HTTP routes, and cross-service links. Instead of reading files one at a time,
485485
an AI coding agent queries the graph — answering structural questions with roughly 120x fewer
486-
tokens. It parses 159 languages and ships as a single static C binary with zero runtime dependencies.
486+
tokens. It parses 158 languages and ships as a single static C binary with zero runtime dependencies.
487487
</p>
488488
<p class="muted">
489489
It is a structural-analysis backend, not a chatbot: there is no embedded LLM and no API key. Your
@@ -566,7 +566,7 @@ <h3>Languages with full Hybrid LSP</h3>
566566
</tbody>
567567
</table>
568568
<p class="muted">
569-
The two-layer pipeline runs a fast syntactic tree-sitter pass for every one of the 159 languages,
569+
The two-layer pipeline runs a fast syntactic tree-sitter pass for every one of the 158 languages,
570570
then a type-aware Hybrid LSP pass on top for the families above. Languages without a Hybrid LSP pass
571571
yet fall back to textual resolution, so you always get an answer.
572572
</p>
@@ -659,7 +659,7 @@ <h2>How fast is it?</h2>
659659
<h2>Features</h2>
660660
<div class="features">
661661
<div class="feature">
662-
<h3>159 languages</h3>
662+
<h3>158 languages</h3>
663663
<p>Python, Go, JS, TS, TSX, Rust, Java, C++, C#, C, PHP, Ruby, Kotlin, Scala, Zig, Elixir, Haskell, OCaml, Swift, Dart, Lean 4, and many more via vendored tree-sitter grammars compiled into the binary.</p>
664664
</div>
665665
<div class="feature">

0 commit comments

Comments
 (0)