Skip to content

Commit 902d98a

Browse files
committed
Update docs
Signed-off-by: Rahul Krishna <rkrsn@ibm.com>
1 parent dc9303e commit 902d98a

20 files changed

Lines changed: 2403 additions & 539 deletions

src/content/docs/backends.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ flowchart LR
2020
This separation lets a single `analysis` API span languages. The SDK does not parse code itself; it invokes the appropriate backend, then maps the backend JSON onto typed models (`JApplication`, `PyModule`, and others). A backend that emits the canonical JSON makes its language available to the SDK with only binding work required. See [Add a language backend](/contributing/add-language-backend/).
2121

2222
<Aside type="note" title="Backends are managed by the SDK">
23-
The SDK manages the backend automatically: it downloads the Java JAR and provisions the Python analyzer in a virtualenv. These pages document the backend layer for readers who want to understand or contribute to it.
23+
The SDK manages the backend automatically: each `codeanalyzer-*` backend binary ships with its packaged PyPI dependency (the JVM-free Java backend runs via `python -m codeanalyzer_java`; the TypeScript binary ships in `codeanalyzer-typescript`), and the Python analyzer is provisioned in a virtualenv. There is no separate JAR download or `analysis_backend_path` override. These pages document the backend layer for readers who want to understand or contribute to it.
2424
</Aside>
2525

2626
This is the **backend** layer. For the SDKs that consume it, see the [SDKs overview](/reference/sdks/).

src/content/docs/backends/codeanalyzer-java.mdx

Lines changed: 34 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ codeanalyzer-java is a **standalone JAR tool** that takes a Java project (or a s
1818
- **Comments & Javadoc**: Extracted and positioned for each entity
1919
- **Entry points**: Main methods and framework-identified entry points (REST endpoints, Struts actions)
2020

21-
The Python SDK invokes this JAR, parses the JSON, and wraps it in `JavaAnalysis`, so callers interact with an `analysis` object rather than the backend directly.
21+
The Python SDK invokes this backend, parses the JSON, and wraps it in `JavaAnalysis`, so callers interact with an `analysis` object rather than the backend directly. The backend is a JVM-free binary that ships with the packaged `codeanalyzer-java` PyPI dependency and is run via `python -m codeanalyzer_java`; there is no separate JAR download or `analysis_backend_path` override.
2222

2323
## Architecture
2424

@@ -329,10 +329,10 @@ java -jar codeanalyzer-2.3.7.jar \
329329

330330
## How the Python SDK uses it
331331

332-
The Python SDK (`JavaAnalysis`) automates JAR invocation:
332+
The Python SDK (`JavaAnalysis`) automates backend invocation:
333333

334-
1. **JAR discovery**: If `analysis_backend_path` is not provided, searches the package resources for `codeanalyzer-*.jar` (bundled)
335-
2. **Invocation**: Calls `java -jar codeanalyzer-*.jar -i <project> -a <level> -o <tmpdir>` (or reads from stdout)
334+
1. **Backend discovery**: The JVM-free backend ships with the packaged `codeanalyzer-java` PyPI dependency and is run via `python -m codeanalyzer_java`. There is no `analysis_backend_path` override anymore.
335+
2. **Invocation**: Calls the backend with `-i <project> -a <level> -o <cache_dir>` (or reads from stdout)
336336
3. **JSON parsing**: Deserializes the output into Pydantic models (`JApplication`, `JType`, `JCallable`, etc.)
337337
4. **Facade**: Wraps the models in `JavaAnalysis`, exposing query methods like `get_classes()`, `get_call_graph()`, `get_callers()`
338338

@@ -341,29 +341,49 @@ from cldk import CLDK
341341
from cldk.analysis import AnalysisLevel
342342

343343
# Behind the scenes:
344-
# 1. CLDK detects language="java" -> uses JCodeanalyzer backend
345-
# 2. JCodeanalyzer finds/downloads codeanalyzer-2.3.7.jar
346-
# 3. Invokes: java -jar ... -i commons-cli -a 2 -o /tmp/...
347-
# 4. Parses JSON -> JApplication (symbol_table + call_graph)
348-
# 5. Returns JavaAnalysis
344+
# 1. CLDK.java(...) selects the codeanalyzer-java backend (default CodeAnalyzerConfig)
345+
# 2. Runs the packaged backend: python -m codeanalyzer_java -i commons-cli -a 2 -o <cache_dir>
346+
# 3. Parses JSON -> JApplication (symbol_table + call_graph)
347+
# 4. Returns JavaAnalysis
349348

350-
analysis = CLDK(language="java").analysis(
349+
analysis = CLDK.java(
351350
project_path="commons-cli",
352351
analysis_level=AnalysisLevel.call_graph,
353-
analysis_backend_path="/path/to/jar/dir" # Optional: custom JAR location
354352
)
355353
```
356354

357-
To point at a custom JAR location (for example, a locally built version):
355+
The old `CLDK(language="java").analysis(...)` form still works but is **deprecated**; prefer the `CLDK.java(...)` factory shown above. The `from cldk import CLDK` import is unchanged.
356+
357+
To control where analysis artifacts are cached, pass a backend config:
358358

359359
```python
360-
analysis = CLDK(language="java").analysis(
360+
from cldk import CLDK
361+
from cldk.analysis import AnalysisLevel
362+
from cldk.analysis.commons.backend_config import CodeAnalyzerConfig
363+
364+
analysis = CLDK.java(
361365
project_path="my_project",
362366
analysis_level=AnalysisLevel.call_graph,
363-
analysis_backend_path="/path/containing/codeanalyzer-custom.jar"
367+
backend=CodeAnalyzerConfig(cache_dir="/tmp/analysis-cache"),
364368
)
365369
```
366370

371+
### Keyword arguments
372+
373+
`CLDK.java(...)` accepts `project_path`, `analysis_level`, `target_files`, and `eager`. The backend is selected by the **type** of the `backend=` config; for Java the only backend is the in-memory codeanalyzer (`CodeAnalyzerConfig`), so omit `backend=` for the default. Pass `backend=CodeAnalyzerConfig(cache_dir=...)` only to override where artifacts are cached. Caching is **on by default**; the cache root defaults to `<project>/.codeanalyzer` and artifacts are written under a language-keyed subdirectory (`<cache_dir>/java/`).
374+
375+
Java also accepts `source_code=...` for analyzing a single Java source string, but this is **deprecated**; prefer `project_path`.
376+
377+
```python
378+
analysis = CLDK.java(source_code="public class Test {}") # deprecated; prefer project_path
379+
```
380+
381+
Import the config objects from:
382+
383+
```python
384+
from cldk.analysis.commons.backend_config import CodeAnalyzerConfig
385+
```
386+
367387
## Building & testing
368388

369389
To build codeanalyzer-java from source:

src/content/docs/backends/codeanalyzer-python.mdx

Lines changed: 50 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -174,7 +174,7 @@ codeanalyzer --input /path/to/my_pkg [OPTIONS]
174174
```
175175

176176
<Aside type="note">
177-
The CLI is intended for backend developers and direct analysis runs. CLDK SDK users invoke the backend indirectly through `CLDK(language="python").analysis(...)`, which manages the virtualenv and invokes the CLI internally.
177+
The CLI is intended for backend developers and direct analysis runs. CLDK SDK users invoke the backend indirectly through `CLDK.python(...)`, which manages the virtualenv and runs the analysis internally.
178178
</Aside>
179179

180180
### Options
@@ -233,7 +233,7 @@ Analyzes only `src/handlers.py`.
233233

234234
## How the SDK consumes it
235235

236-
A call to `CLDK(language="python").analysis(project_path="my_pkg")` in the Python SDK proceeds as follows:
236+
A call to `CLDK.python(project_path="my_pkg")` in the Python SDK proceeds as follows:
237237

238238
1. **Virtualenv provisioning**: CLDK detects or installs `codeanalyzer-python` into a managed virtualenv in the cache directory (default: `<project_dir>/.codeanalyzer/venv`).
239239

@@ -253,11 +253,12 @@ A call to `CLDK(language="python").analysis(project_path="my_pkg")` in the Pytho
253253
```python
254254
from cldk import CLDK
255255
from cldk.analysis import AnalysisLevel
256+
from cldk.analysis.commons.backend_config import PyCodeAnalyzerConfig
256257

257-
analysis = CLDK(language="python").analysis(
258+
analysis = CLDK.python(
258259
project_path="my_pkg",
259260
analysis_level=AnalysisLevel.call_graph,
260-
use_codeql=True, # optional; merges CodeQL edges
261+
backend=PyCodeAnalyzerConfig(use_codeql=True), # optional; merges CodeQL edges
261262
)
262263

263264
# Query the symbol table
@@ -281,14 +282,55 @@ cat results/analysis.json | jq '.symbol_table | keys'
281282
</TabItem>
282283
</Tabs>
283284

285+
### Choosing a backend
286+
287+
The backend is selected by the **type** of the `backend=` config passed to `CLDK.python(...)`:
288+
289+
- **In-memory codeanalyzer** (default): omit `backend=`, or pass `backend=PyCodeAnalyzerConfig(...)`. The Python-only call-graph knobs `use_codeql=...` and `use_ray=...` live on this config, as does `cache_dir=...`.
290+
- **Read-only Neo4j**: pass `backend=Neo4jConnectionConfig(...)` to query a graph populated out of band (no local analysis is run).
291+
292+
```python
293+
from cldk import CLDK
294+
from cldk.analysis.commons.backend_config import (
295+
PyCodeAnalyzerConfig,
296+
Neo4jConnectionConfig,
297+
)
298+
299+
# In-memory backend with Ray + custom cache directory
300+
analysis = CLDK.python(
301+
project_path="my_pkg",
302+
backend=PyCodeAnalyzerConfig(
303+
use_codeql=True,
304+
use_ray=True,
305+
cache_dir="/tmp/analysis-cache",
306+
),
307+
)
308+
309+
# Read-only Neo4j backend
310+
analysis = CLDK.python(
311+
project_path="my_pkg",
312+
backend=Neo4jConnectionConfig(
313+
uri="bolt://localhost:7687",
314+
username="neo4j",
315+
password="neo4j",
316+
database=None,
317+
application_name="my_pkg",
318+
),
319+
)
320+
```
321+
322+
`Neo4jConnectionConfig` is importable from `cldk.analysis.commons.backend_config` (and also from `cldk.analysis.python.neo4j`).
323+
324+
`CLDK.python(...)` keeps the `project_path`, `analysis_level`, `target_files`, and `eager` keyword arguments. The old `CLDK(language="python").analysis(...)` form still works but is **deprecated**; prefer `CLDK.python(...)`. The `from cldk import CLDK` import is unchanged.
325+
284326
### Caching and virtualenv management
285327

286-
- **Cache location**: Stored in `cache_dir/.codeanalyzer/` (default: `<project_dir>/.codeanalyzer/`).
287-
- **Virtualenv**: Auto-created at `.codeanalyzer/<project_name>/virtualenv/`. The backend installs dependencies from `requirements.txt`, `pyproject.toml`, `setup.py`, `Pipfile`, etc.
328+
- **Cache location**: A single language-keyed `cache_dir` (default: `<project_dir>/.codeanalyzer`); Python artifacts live under `<cache_dir>/python/`. Set it via `backend=PyCodeAnalyzerConfig(cache_dir=...)`.
329+
- **Virtualenv**: Auto-created under the cache directory. The backend installs dependencies from `requirements.txt`, `pyproject.toml`, `setup.py`, `Pipfile`, etc.
288330
- **Analysis cache**: Indexed by file content hash; unchanged files reuse cached results.
289-
- **CodeQL database**: Stored in `.codeanalyzer/codeql/` if `--codeql` is enabled; downloaded on first use.
331+
- **CodeQL database**: Stored under the cache directory if CodeQL is enabled; downloaded on first use.
290332

291-
To force a clean rebuild, pass `--eager` (or `eager_analysis=True` in the SDK), or delete the cache directory.
333+
To force a clean rebuild, pass `eager=True` to `CLDK.python(...)` (or `--eager` on the CLI), or delete the cache directory.
292334

293335
## Design principles
294336

src/content/docs/backends/codeanalyzer-ts.mdx

Lines changed: 40 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ The `cants` CLI is available from **PyPI** and **Homebrew**. Both distribute a p
5656
</TabItem>
5757
</Tabs>
5858

59-
The CLDK Python SDK depends on the PyPI package `codeanalyzer-typescript` to locate this backend; the package exposes `bin_path()`, which returns the path to the bundled binary.
59+
The CLDK Python SDK depends on the PyPI package `codeanalyzer-typescript` to locate this backend; the package exposes `bin_path()`, which returns the path to the bundled binary. There is no `analysis_backend_path` argument. To override the binary out of band (for example, a locally built `cants`), set the `$CODEANALYZER_TS_BIN` environment variable to its path; this is the sole override.
6060

6161
<Aside type="note">
6262
The distributable is named **`codeanalyzer-typescript`** (the PyPI package and the Homebrew formula), but the command it installs on `PATH` is **`cants`**, the analyzer's binary name. Run `cants`, not `codeanalyzer-typescript`.
@@ -297,13 +297,13 @@ TypeScript includes several language-specific constructs that codeanalyzer-ts mo
297297

298298
## Python SDK integration
299299

300-
TypeScript support in the Python SDK is **in development**. Once available, TypeScript projects will be analyzable through the same `CLDK` analysis API as Python and Java:
300+
TypeScript projects are analyzable through the same `CLDK` analysis API as Python and Java, via the `CLDK.typescript(...)` factory:
301301

302302
```python
303303
from cldk import CLDK
304304
from cldk.analysis import AnalysisLevel
305305

306-
analysis = CLDK(language="typescript").analysis(
306+
analysis = CLDK.typescript(
307307
project_path="/path/to/ts/project",
308308
analysis_level=AnalysisLevel.call_graph,
309309
)
@@ -313,9 +313,42 @@ print(analysis.get_classes()) # Dict[str, TSClass]
313313
graph = analysis.get_call_graph() # networkx.DiGraph
314314
```
315315

316-
<Aside type="note">
317-
The codeanalyzer-ts backend is complete and production-ready. The Python SDK integration is in progress on the `add-typescript-support` branch.
318-
</Aside>
316+
The old `CLDK(language="typescript").analysis(...)` form still works but is **deprecated**; prefer `CLDK.typescript(...)`. The `from cldk import CLDK` import is unchanged.
317+
318+
### Choosing a backend
319+
320+
`CLDK.typescript(...)` keeps the `project_path`, `analysis_level`, `target_files`, and `eager` keyword arguments. The backend is selected by the **type** of the `backend=` config:
321+
322+
- **In-memory codeanalyzer** (default): omit `backend=`, or pass `backend=CodeAnalyzerConfig(cache_dir=...)` to override where artifacts are cached.
323+
- **Read-only Neo4j**: pass `backend=Neo4jConnectionConfig(...)` to query a graph populated out of band.
324+
325+
```python
326+
from cldk import CLDK
327+
from cldk.analysis.commons.backend_config import (
328+
CodeAnalyzerConfig,
329+
Neo4jConnectionConfig,
330+
)
331+
332+
# In-memory backend with a custom cache directory
333+
analysis = CLDK.typescript(
334+
project_path="/path/to/ts/project",
335+
backend=CodeAnalyzerConfig(cache_dir="/tmp/analysis-cache"),
336+
)
337+
338+
# Read-only Neo4j backend
339+
analysis = CLDK.typescript(
340+
project_path="/path/to/ts/project",
341+
backend=Neo4jConnectionConfig(
342+
uri="bolt://localhost:7687",
343+
username="neo4j",
344+
password="neo4j",
345+
database=None,
346+
application_name="my_project",
347+
),
348+
)
349+
```
350+
351+
`Neo4jConnectionConfig` is importable from `cldk.analysis.commons.backend_config` (and also from `cldk.analysis.typescript.neo4j`).
319352

320353
## Schema invariants
321354

@@ -330,7 +363,7 @@ This ensures every call-graph edge points to a real symbol-table entry.
330363

331364
## Caching and incremental analysis
332365

333-
The analyzer maintains a cache under `<project>/.codeanalyzer/` (or `-c <dir>`), stamped with the analyzer version. The cache is invalidated on:
366+
The analyzer maintains a cache under a single language-keyed `cache_dir` (default: `<project>/.codeanalyzer`), with TypeScript artifacts under `<cache_dir>/typescript/`, stamped with the analyzer version. Caching is **on by default**. From the SDK, set the cache root via `backend=CodeAnalyzerConfig(cache_dir=...)`; on the CLI, use `-c <dir>`. The cache is invalidated on:
334367
- Analyzer version change
335368
- Any source file modification (content hash mismatch)
336369
- `--eager` flag (forces clean rebuild)

src/content/docs/cocoa.mdx

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ from cldk import CLDK
4141
from cldk.analysis import AnalysisLevel
4242
import networkx as nx
4343
44-
analysis = CLDK(language="java").analysis(
44+
analysis = CLDK.java(
4545
project_path=".",
4646
analysis_level=AnalysisLevel.call_graph, # needed for callers/reachability
4747
)
@@ -168,7 +168,7 @@ The trace is reconstructed from call-graph edges.
168168
from cldk.analysis import AnalysisLevel
169169
import networkx as nx
170170
171-
analysis = CLDK(language="java").analysis(
171+
analysis = CLDK.java(
172172
project_path=".", analysis_level=AnalysisLevel.call_graph,
173173
)
174174
# ... query analysis (get_callers / get_callees / get_call_graph) and print() ...
@@ -203,7 +203,7 @@ The trace is reconstructed from call-graph edges.
203203
from cldk import CLDK
204204
from cldk.analysis import AnalysisLevel
205205
206-
analysis = CLDK(language="java").analysis(
206+
analysis = CLDK.java(
207207
project_path=".", analysis_level=AnalysisLevel.call_graph,
208208
)
209209
callers = analysis.get_callers("<CLASS>", "<METHOD>")
@@ -230,7 +230,7 @@ The trace is reconstructed from call-graph edges.
230230
from cldk.analysis import AnalysisLevel
231231
import networkx as nx
232232
233-
analysis = CLDK(language="java").analysis(
233+
analysis = CLDK.java(
234234
project_path=".", analysis_level=AnalysisLevel.call_graph,
235235
)
236236
cg = analysis.get_call_graph()

0 commit comments

Comments
 (0)