Skip to content

Commit 36e95bb

Browse files
feat(compiler): Add JavaScript/TypeScript IDL code generation (#3394)
## Summary Implements TypeScript code generation for Fory IDL within the fory-compiler, converting FDL (Fory Definition Language) schema files into pure TypeScript type definitions. Zero runtime dependencies, with comprehensive test coverage (12/12 tests passing), supporting messages, enums, unions, and all primitive types. ## Changes ### Core Implementation - **compiler/fory_compiler/generators/typescript.py** - TypeScript code generator extending BaseGenerator (365 lines) - Generates type-safe TypeScript interfaces, enums, and discriminated unions - Supports nested types, collections, and optional fields - Proper type mapping for all 25 FDL primitive kinds - Field name conversion (snake_case → camelCase) - Registration helper function generation for Fory serialization integration - **compiler/fory_compiler/generators/__init__.py** - Registration of TypeScriptGenerator in the compiler ecosystem - **compiler/fory_compiler/cli.py** - Added --typescript_out CLI argument for TypeScript code generation - **compiler/fory_compiler/tests/test_typescript_codegen.py** - 12 golden codegen tests covering: - Enum and message generation - Nested types (messages and enums) - Discriminated unions - All primitive type mappings - Collection types (arrays, maps) - Field naming conventions - File structure and licensing - Zero runtime dependencies validation ### Features - **Message-to-Interface Generation**: Auto-converts FDL messages to TypeScript interfaces - **Enum Support**: Generates TypeScript enums with proper value stripping and type IDs - **Discriminated Unions**: Creates union types with discriminator enums for type safety - **Type Mappings**: Full support for all FDL primitives (bool→boolean, int32→number, int64→bigint|number, float/double→number, etc.) - **Nested Types**: Supports nested messages and enums within parent types - **Collection Types**: Arrays (repeated fields) and maps with type-safe generics - **Zero Runtime Dependencies**: Pure TypeScript type definitions, no gRPC or external imports - **Field Naming**: Automatic conversion to camelCase per TypeScript conventions - **Package/Module Handling**: Uses last segment of package name for module name and registration functions - **License Headers**: All generated files include Apache 2.0 license headers - **Registration Helpers**: Generates registration functions for Fory serialization framework integration ### AI Assistance Checklist - [x] Substantial AI assistance was used in this PR (yes) - [x] I included the standardized AI Usage Disclosure block below - [x] I can explain and defend all important changes without AI help - [x] I reviewed AI-assisted code changes line by line before submission - [x] I ran adequate human verification and recorded evidence (local/CI checks, pass/fail summary, and review confirmation) - [x] I added/updated tests and specs where required - [x] I validated protocol/performance impacts with evidence when applicable - [x] I verified licensing and provenance compliance AI Usage Disclosure - substantial_ai_assistance: yes - scope: <design drafting | code drafting> - affected_files_or_subsystems: <high-level paths/modules> - human_verification: <checks run locally or in CI + pass/fail summary + contributor reviewed results> - performance_verification: <N/A> - provenance_license_confirmation: <Apache-2.0-compatible provenance confirmed; no incompatible third-party code introduced> ### AI Review Evidence #### General AI Review Initial Review • Identified issues in decimal type mapping, array type precedence, tag id drop, and traversalContainer doesn't traverse union cases Fixes Applied • Updated DECIMAL mapping to string to avoid precision loss • fix array type precedence issue.
• tag id drop prevention for id == 0
• Aligned map type documentation with generated output Final Review (Fresh Session) • Re reviewed updated changes
• All issues resolved
• All tests pass. No actionable issues remain #### Fory-Specific AI Review Scope • Reviewed serialization and deserialization logic
• Verified typeInfo handling, struct embedding, and reference tracking • Checked alignment with Java reference implementation
• Validated cross language compatibility and code generation Result • No actionable issues found
 • Implementation aligns with Fory wire format and runtime behavior Fixes #3280 --------- Co-authored-by: Shawn Yang <shawn.ck.yang@gmail.com>
1 parent f129b21 commit 36e95bb

File tree

37 files changed

+3035
-259
lines changed

37 files changed

+3035
-259
lines changed

.github/workflows/ci.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -494,6 +494,7 @@ jobs:
494494
run: |
495495
cd javascript
496496
npm install
497+
npm run build
497498
- name: Run JavaScript Xlang Test
498499
env:
499500
FORY_JAVASCRIPT_JAVA_CI: "1"
@@ -502,6 +503,8 @@ jobs:
502503
mvn -T16 --no-transfer-progress clean install -DskipTests -Dmaven.javadoc.skip=true -Dmaven.source.skip=true
503504
cd fory-core
504505
mvn --no-transfer-progress test -Dtest=org.apache.fory.xlang.JavaScriptXlangTest -DforkCount=0
506+
- name: Run JavaScript IDL Tests
507+
run: ./integration_tests/idl_tests/run_javascript_tests.sh
505508

506509
rust:
507510
name: Rust CI

compiler/README.md

Lines changed: 39 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ The FDL compiler generates cross-language serialization code from schema definit
44

55
## Features
66

7-
- **Multi-language code generation**: Java, Python, Go, Rust, C++, C#
7+
- **Multi-language code generation**: Java, Python, Go, Rust, C++, C#, JavaScript, and Swift
88
- **Rich type system**: Primitives, enums, messages, lists, maps
99
- **Cross-language serialization**: Generated code works seamlessly with Apache Fory
1010
- **Type ID and namespace support**: Both numeric IDs and name-based type registration
@@ -64,16 +64,16 @@ message Cat [id=103] {
6464
foryc schema.fdl --output ./generated
6565

6666
# Generate for specific languages
67-
foryc schema.fdl --lang java,python,csharp --output ./generated
67+
foryc schema.fdl --lang java,python,csharp,javascript --output ./generated
6868

6969
# Override package name
7070
foryc schema.fdl --package myapp.models --output ./generated
7171

7272
# Language-specific output directories (protoc-style)
73-
foryc schema.fdl --java_out=./src/main/java --python_out=./python/src --csharp_out=./csharp/src/Generated
73+
foryc schema.fdl --java_out=./src/main/java --python_out=./python/src --csharp_out=./csharp/src/Generated --javascript_out=./javascript
7474

7575
# Combine with other options
76-
foryc schema.fdl --java_out=./gen --go_out=./gen/go --csharp_out=./gen/csharp -I ./proto
76+
foryc schema.fdl --java_out=./gen --go_out=./gen/go --csharp_out=./gen/csharp --javascript_out=./gen/js -I ./proto
7777
```
7878

7979
### 3. Use Generated Code
@@ -185,19 +185,19 @@ message Config { ... } // Registered as "package.Config"
185185

186186
### Primitive Types
187187

188-
| FDL Type | Java | Python | Go | Rust | C++ | C# |
189-
| ----------- | ----------- | ------------------- | ----------- | ----------------------- | ---------------------- | ---------------- |
190-
| `bool` | `boolean` | `bool` | `bool` | `bool` | `bool` | `bool` |
191-
| `int8` | `byte` | `pyfory.int8` | `int8` | `i8` | `int8_t` | `sbyte` |
192-
| `int16` | `short` | `pyfory.int16` | `int16` | `i16` | `int16_t` | `short` |
193-
| `int32` | `int` | `pyfory.int32` | `int32` | `i32` | `int32_t` | `int` |
194-
| `int64` | `long` | `pyfory.int64` | `int64` | `i64` | `int64_t` | `long` |
195-
| `float32` | `float` | `pyfory.float32` | `float32` | `f32` | `float` | `float` |
196-
| `float64` | `double` | `pyfory.float64` | `float64` | `f64` | `double` | `double` |
197-
| `string` | `String` | `str` | `string` | `String` | `std::string` | `string` |
198-
| `bytes` | `byte[]` | `bytes` | `[]byte` | `Vec<u8>` | `std::vector<uint8_t>` | `byte[]` |
199-
| `date` | `LocalDate` | `datetime.date` | `time.Time` | `chrono::NaiveDate` | `fory::Date` | `DateOnly` |
200-
| `timestamp` | `Instant` | `datetime.datetime` | `time.Time` | `chrono::NaiveDateTime` | `fory::Timestamp` | `DateTimeOffset` |
188+
| FDL Type | Java | Python | Go | Rust | C++ | C# | JavaScript |
189+
| ----------- | ----------- | ------------------- | ----------- | ----------------------- | ---------------------- | ---------------- | ------------------ |
190+
| `bool` | `boolean` | `bool` | `bool` | `bool` | `bool` | `bool` | `boolean` |
191+
| `int8` | `byte` | `pyfory.int8` | `int8` | `i8` | `int8_t` | `sbyte` | `number` |
192+
| `int16` | `short` | `pyfory.int16` | `int16` | `i16` | `int16_t` | `short` | `number` |
193+
| `int32` | `int` | `pyfory.int32` | `int32` | `i32` | `int32_t` | `int` | `number` |
194+
| `int64` | `long` | `pyfory.int64` | `int64` | `i64` | `int64_t` | `long` | `bigint \| number` |
195+
| `float32` | `float` | `pyfory.float32` | `float32` | `f32` | `float` | `float` | `number` |
196+
| `float64` | `double` | `pyfory.float64` | `float64` | `f64` | `double` | `double` | `number` |
197+
| `string` | `String` | `str` | `string` | `String` | `std::string` | `string` | `string` |
198+
| `bytes` | `byte[]` | `bytes` | `[]byte` | `Vec<u8>` | `std::vector<uint8_t>` | `byte[]` | `Uint8Array` |
199+
| `date` | `LocalDate` | `datetime.date` | `time.Time` | `chrono::NaiveDate` | `fory::Date` | `DateOnly` | `Date` |
200+
| `timestamp` | `Instant` | `datetime.datetime` | `time.Time` | `chrono::NaiveDateTime` | `fory::Timestamp` | `DateTimeOffset` | `Date` |
201201

202202
### Collection Types
203203

@@ -285,7 +285,8 @@ fory_compiler/
285285
├── go.py # Go struct generator
286286
├── rust.py # Rust struct generator
287287
├── cpp.py # C++ struct generator
288-
└── csharp.py # C# class generator
288+
├── csharp.py # C# class generator
289+
└── javascript.py # JavaScript interface generator
289290
```
290291

291292
### FDL Frontend
@@ -422,6 +423,25 @@ cd integration_tests/idl_tests
422423
./run_csharp_tests.sh
423424
```
424425

426+
### JavaScript
427+
428+
Generates interfaces with:
429+
430+
- `export interface` declarations for messages
431+
- `export enum` declarations for enums
432+
- Discriminated unions with case enums
433+
- Registration helper function
434+
435+
```javascript
436+
export interface Cat {
437+
friend?: Dog | null;
438+
name?: string | null;
439+
tags: string[];
440+
scores: Map<string, number>;
441+
lives: number;
442+
}
443+
```
444+
425445
## CLI Reference
426446

427447
```
@@ -431,7 +451,7 @@ Arguments:
431451
FILES FDL files to compile
432452
433453
Options:
434-
--lang TEXT Target languages (java,python,cpp,rust,go,csharp or "all")
454+
--lang TEXT Target languages (java,python,cpp,rust,go,csharp,javascript or "all")
435455
Default: all
436456
--output, -o PATH Output directory
437457
Default: ./generated

compiler/fory_compiler/cli.py

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -264,7 +264,7 @@ def parse_args(args: Optional[List[str]] = None) -> argparse.Namespace:
264264
"--lang",
265265
type=str,
266266
default="all",
267-
help="Comma-separated list of target languages (java,python,cpp,rust,go,csharp,swift). Default: all",
267+
help="Comma-separated list of target languages (java,python,cpp,rust,go,csharp,javascript,swift). Default: all",
268268
)
269269

270270
parser.add_argument(
@@ -343,6 +343,14 @@ def parse_args(args: Optional[List[str]] = None) -> argparse.Namespace:
343343
help="Generate C# code in DST_DIR",
344344
)
345345

346+
parser.add_argument(
347+
"--javascript_out",
348+
type=Path,
349+
default=None,
350+
metavar="DST_DIR",
351+
help="Generate JavaScript code in DST_DIR",
352+
)
353+
346354
parser.add_argument(
347355
"--swift_out",
348356
type=Path,
@@ -650,6 +658,7 @@ def cmd_compile(args: argparse.Namespace) -> int:
650658
"go": args.go_out,
651659
"rust": args.rust_out,
652660
"csharp": args.csharp_out,
661+
"javascript": args.javascript_out,
653662
"swift": args.swift_out,
654663
}
655664

compiler/fory_compiler/generators/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
from fory_compiler.generators.rust import RustGenerator
2525
from fory_compiler.generators.go import GoGenerator
2626
from fory_compiler.generators.csharp import CSharpGenerator
27+
from fory_compiler.generators.javascript import JavaScriptGenerator
2728
from fory_compiler.generators.swift import SwiftGenerator
2829

2930
GENERATORS = {
@@ -33,6 +34,7 @@
3334
"rust": RustGenerator,
3435
"go": GoGenerator,
3536
"csharp": CSharpGenerator,
37+
"javascript": JavaScriptGenerator,
3638
"swift": SwiftGenerator,
3739
}
3840

@@ -44,6 +46,7 @@
4446
"RustGenerator",
4547
"GoGenerator",
4648
"CSharpGenerator",
49+
"JavaScriptGenerator",
4750
"SwiftGenerator",
4851
"GENERATORS",
4952
]

0 commit comments

Comments
 (0)