Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion ts-parser/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,6 @@ dist/

temp/

output.json
output.json

output/
68 changes: 53 additions & 15 deletions ts-parser/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
# TypeScript Parser for ABCoder

A TypeScript AST parser that extracts method calls, variable references, and dependencies.
A TypeScript AST parser that extracts method calls, variable references, and dependencies with advanced monorepo support and intelligent parsing strategies.

Usage:
## Features

- 🚀 **Monorepo Support**: Intelligent detection and parsing of monorepo projects
- ⚡ **Smart Parsing Strategy**: Automatic selection between single-process and cluster-based parsing
- 📦 **Multiple Monorepo Formats**: Support for Edex, pnpm workspaces, Lerna
- 🎯 **Flexible Output Modes**: Combined or separate repo output for monorepo packages

## Usage

Build: `npm run build`

Expand All @@ -11,31 +18,63 @@ Run: `node dist/index.js parse [options] <directory>`
Parse a TypeScript repository and generate UNIAST JSON

Arguments:
directory Directory to parse
directory Directory to parse

## Examples

### Basic Usage

- **Parse a single TypeScript project** : `node dist/index.js parse ./my-project`

- **Parse with pretty output** : `node dist/index.js parse ./my-project --pretty`

- **Parse monorepo with separate package outputs** : `node dist/index.js parse ./my-monorepo --monorepo-mode separate`

- **Parse monorepo (combined output)**: `node dist/index.js parse ./my-monorepo`
- **Parse monorepo (separate output for each package)**: `node dist/index.js parse ./my-monorepo --monorepo-mode separate`

- **Custom output path** : `node dist/index.js parse ./my-project -o ./output/result.json `

| Option | Description |
|--------|-------------|
| -o, --output <file> | Output file path (default: "output.json") |
| -t, --tsconfig <file> | Path to tsconfig.json file, if you provide a relative path, it will be relative to **the directory of the input file** (default: "tsconfig.json") |
| --no-dist | Ignore dist folder and its contents |
| --pretty | Pretty print JSON output |
| --src <dirs> | Directory paths to include (comma-separated) |
| -h, --help | display help for command |
## Options

| Option | Description |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| -o, --output <file> | Output file path (default: "output.json") |
| -t, --tsconfig <file> | Path to tsconfig.json file, if you provide a relative path, it will be relative to **the directory of the input file** (default: "tsconfig.json") |
| --no-dist | Ignore dist folder and its contents |
| --pretty | Pretty print JSON output |
| --src <dirs> | Directory paths to include (comma-separated) |
| --monorepo-mode <mode> | Monorepo output mode: "combined" (entire repository) or "separate" (each package) (default: "combined") |
| -h, --help | display help for command |

See `./index.ts` for more information.

## Monorepo Support

The parser automatically detects and supports various monorepo configurations:

- **Eden Monorepo**: Supports both `packages` format and `workspaces` format
- **pnpm Workspaces**: Reads `pnpm-workspace.yaml` configuration
- **Lerna**: Detects `lerna.json` configuration

### Parsing Strategies

The parser intelligently selects the optimal parsing strategy based on project size:

- **Single Process Mode**: For small to medium projects
- **Cluster Mode**: For large projects with parallel processing across multiple CPU cores

## Notes

1. MUST correctly specify the location of the current project's `tsconfig.json`.
1. MUST correctly specify the location of the current project's `tsconfig.json`.

2. If you provide a relative path to argument `--tsconfig`, it will be relative to **the directory of the input file**.

3. Before usage, please configure the dependencies for your TypeScript project, such as running npm install and setting up cross-package dependencies in monorepo.

4. If the repository you're analyzing is too large, you may need to adjust Node.js's maximum memory allocation.
4. For large monorepo projects, the parser will automatically use cluster-based processing to improve performance.

5. If the repository you're analyzing is too large, you may need to adjust Node.js's maximum memory allocation.

## Terminology

Expand All @@ -46,10 +85,9 @@ See `./index.ts` for more information.

This terminology mapping is used consistently throughout the parser to align with the UniAST specification, but it may initially seem counterintuitive to developers familiar with JavaScript/TypeScript conventions.


## Some known issues

- When there is a circular dependency, the parser will choose one of the dependencies as the main dependency.
- The parser does not handle dynamic imports.
- The parser does not handle TypeScript decorators.
- For external symbol which has no `.d.ts` declaration file, the parser will not be able to resolve the symbol.
- For external symbol which has no `.d.ts` declaration file, the parser will not be able to resolve the symbol.
19 changes: 19 additions & 0 deletions ts-parser/jest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
roots: ['<rootDir>/src'],
testMatch: ['**/__tests__/**/*.ts', '**/?(*.)+(spec|test).ts'],
transform: {
'^.+\\.ts$': 'ts-jest',
},
collectCoverageFrom: [
'src/**/*.ts',
'!src/**/*.d.ts',
'!src/**/*.test.ts',
'!src/**/*.spec.ts',
],
moduleFileExtensions: ['ts', 'js', 'json'],
testPathIgnorePatterns: ['/node_modules/', '/dist/', '/temp/'],
coverageDirectory: 'coverage',
coverageReporters: ['text', 'lcov', 'html'],
};
5 changes: 4 additions & 1 deletion ts-parser/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ program
.option('--no-dist', 'Ignore dist folder and its contents', false)
.option('--pretty', 'Pretty print JSON output', false)
.option('--src <dirs>', 'Directory paths to include (comma-separated)', (value) => value.split(','))
.option('--monorepo-mode <mode>', '"combined"(output entrie monorep repository) "separate"(output each app)', 'combined')
.action(async (directory, options) => {
try {
const repoPath = path.resolve(directory);
Expand All @@ -36,9 +37,11 @@ program
const repository = await parser.parseRepository(repoPath, {
loadExternalSymbols: false,
noDist: options.noDist,
srcPatterns: options.src
srcPatterns: options.src,
monorepoMode: options.monorepoMode as 'combined' | 'separate'
});

// Output the repository JSON file
const outputPath = path.resolve(options.output);
const jsonOutput = options.pretty
? JSON.stringify(repository, null, 2)
Expand Down
Loading
Loading