Skip to content

Commit 8d2f7a1

Browse files
committed
- TypeScript-to-ASL compiler now supports JSONata as the default query language
- Standard JS methods compile directly: string manipulation, math, type conversion, object utilities - Higher-order array functions (map, filter, reduce, find, some, every) with pure callbacks compile to JSONat lambdas - Full arithmetic support: *, /, %, - work natively (previously JSONPath-only limitation) - JSONPath still fully supported via --query-language jsonpath flag, no regressions - Clear error messages when JSONata-only features are used in JSONPath mode - All documentation and examples updated for dual-mode support
1 parent 677cca6 commit 8d2f7a1

30 files changed

Lines changed: 3289 additions & 291 deletions

README.md

Lines changed: 58 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,15 @@
33
[![CI](https://github.com/DevNamedZed/simplesteps/actions/workflows/ci.yml/badge.svg)](https://github.com/DevNamedZed/simplesteps/actions/workflows/ci.yml)
44
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
55

6-
TypeScript-to-ASL compiler for AWS Step Functions. Define workflows as typed async functions, compile to Amazon States Language with full data flow inference, and deploy with CDK or the CLI.
6+
TypeScript-to-ASL compiler for AWS Step Functions. Define workflows as typed async functions, compile to Amazon States Language with full data flow inference, and deploy with CDK or the CLI. Supports both JSONata (default) and JSONPath query languages.
77

88
[Playground](https://devnamedzed.github.io/simplesteps/) | [Getting Started](docs/getting-started.md) | [CDK Integration](docs/cdk-integration.md) | [Language Reference](docs/language-reference.md)
99

1010
## Motivation
1111

1212
At my previous company we had dozens of Step Functions deployed with CDK. They were incredibly hard to read and maintain. The CDK approach defines a program by stringing together a tree of construct objects — `.next().next().next()` chains with raw JSONPath and `sfn.CustomState` workarounds. That's exactly the kind of work a compiler should handle.
1313

14-
SimpleSteps compiles typed async functions to ASL state machines. The compiler performs whole-program data flow analysis and derives all JSONPath expressions (`Parameters`, `ResultPath`, `InputPath`, `ResultSelector`) from variable usage. Service bindings are resolved at compile time, with CDK token substitution at synth time.
14+
SimpleSteps compiles typed async functions to ASL state machines. The compiler performs whole-program data flow analysis and derives all data flow fields from variable usage — you never write path expressions. In JSONata mode (the default), standard JavaScript methods like `Math.round()`, `str.toUpperCase()`, and `arr.filter()` compile directly to JSONata built-ins. Service bindings are resolved at compile time, with CDK token substitution at synth time.
1515

1616
**Input:**
1717

@@ -31,7 +31,31 @@ export const helloWorld = Steps.createFunction(
3131
);
3232
```
3333

34-
**Output:**
34+
**Output (JSONata — default):**
35+
36+
```json
37+
{
38+
"QueryLanguage": "JSONata",
39+
"StartAt": "Invoke_helloFn",
40+
"States": {
41+
"Invoke_helloFn": {
42+
"Type": "Task",
43+
"Resource": "arn:aws:lambda:us-east-1:123456789:function:Hello",
44+
"Arguments": { "name": "{% $states.input.name %}" },
45+
"Assign": { "result": "{% $states.result %}" },
46+
"Next": "Return_Result"
47+
},
48+
"Return_Result": {
49+
"Type": "Pass",
50+
"Arguments": { "greeting": "{% $result.greeting %}" },
51+
"End": true
52+
}
53+
}
54+
}
55+
```
56+
57+
<details>
58+
<summary>JSONPath output (--query-language jsonpath)</summary>
3559

3660
```json
3761
{
@@ -53,6 +77,8 @@ export const helloWorld = Steps.createFunction(
5377
}
5478
```
5579

80+
</details>
81+
5682
## Quick Start
5783

5884
### CDK
@@ -125,6 +151,9 @@ Workflows can also be defined in [separate files](docs/cdk-integration.md) using
125151
```bash
126152
npm install @simplesteps/core
127153
npx simplesteps compile workflow.ts -o output/
154+
155+
# Use JSONPath mode instead of JSONata (default)
156+
npx simplesteps compile workflow.ts -o output/ --query-language jsonpath
128157
```
129158

130159
## Language Support
@@ -145,14 +174,34 @@ npx simplesteps compile workflow.ts -o output/
145174
| `return value` | Succeed / End state |
146175
| `try { ... } catch (e) { ... }` | Catch rules |
147176
| `.call(input, { retry, timeoutSeconds, heartbeatSeconds })` | Retry / Timeout / Heartbeat |
148-
| `` `Hello ${name}` `` | `States.Format` |
149-
| `a + b` (numbers) | `States.MathAdd` |
150-
| `str.split(',')` | `States.StringSplit` |
151-
| `JSON.parse(str)` | `States.StringToJson` |
152-
| `Steps.uuid()` / `crypto.randomUUID()` | `States.UUID` |
177+
| `` `Hello ${name}` `` | `States.Format` / string concatenation |
178+
| `a + b`, `a * b`, `a - b`, `a / b`, `a % b` | Native arithmetic (JSONata) / `States.MathAdd` (JSONPath) |
179+
| `str.split(',')` | `$split()` / `States.StringSplit` |
180+
| `JSON.parse(str)` | `$eval()` / `States.StringToJson` |
181+
| `Steps.uuid()` / `crypto.randomUUID()` | `$uuid()` / `States.UUID` |
182+
183+
### JSONata-Only Methods (default mode)
184+
185+
| TypeScript | JSONata |
186+
|---|---|
187+
| `str.toUpperCase()`, `.toLowerCase()`, `.trim()` | `$uppercase`, `$lowercase`, `$trim` |
188+
| `str.substring()`, `.replace()`, `.charAt()`, `.repeat()` | `$substring`, `$replace`, `$pad` |
189+
| `str.startsWith()`, `.endsWith()`, `.padStart()`, `.padEnd()` | Composed expressions |
190+
| `Math.floor/ceil/round/abs/pow/sqrt/min/max/random` | `$floor`, `$ceil`, `$round`, `$abs`, `$power`, `$sqrt`, `$min`, `$max`, `$random` |
191+
| `Number()`, `String()`, `Boolean()`, `typeof` | `$number`, `$string`, `$boolean`, `$type` |
192+
| `Object.keys()`, `Object.values()` | `$keys`, `$lookup` |
193+
| `Date.now()`, `Array.isArray()` | `$millis`, `$type(x) = 'array'` |
194+
| `arr.join()`, `.reverse()`, `.sort()`, `.concat()` | `$join`, `$reverse`, `$sort`, `$append` |
195+
| `arr.map(v => expr)` | `$map(arr, function($v) { expr })` |
196+
| `arr.filter(v => pred)` | `$filter(arr, function($v) { pred })` |
197+
| `arr.reduce((a, v) => e, init)` | `$reduce(arr, function($a, $v) { e }, init)` |
198+
| `arr.find()`, `.some()`, `.every()` | Composed from `$filter` + `$count` |
153199

154200
## Compiler Features
155201

202+
- **Dual query language support** — JSONata (default) and JSONPath, switchable via `--query-language`
203+
- **55+ JS method → JSONata mappings** — string, math, array, type conversion, and higher-order functions compile directly
204+
- **Lambda expression analysis** — pure callbacks in `.map()`, `.filter()`, `.reduce()` auto-compile to JSONata higher-order functions
156205
- **Whole-program data flow analysis** with constant propagation lattice across modules
157206
- **Cross-file import resolution** with demand-driven analysis and cycle detection
158207
- **Pure function inlining** for compile-time constant derivation
@@ -180,7 +229,7 @@ npx simplesteps compile workflow.ts -o output/
180229
## Examples
181230

182231
- [Starter projects](examples/starters/) — CLI, library API, and CDK templates
183-
- [Showcase](examples/showcase/)30+ examples covering every language feature
232+
- [Showcase](examples/showcase/)38 examples covering every language feature, including JSONata methods
184233

185234
## License
186235

docs/cli.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ simplesteps compile <file-or-tsconfig> [options]
1515
| Flag | Description | Default |
1616
|---|---|---|
1717
| `-o, --output <dir>` | Write `.asl.json` files to this directory | stdout |
18+
| `--query-language <jsonata\|jsonpath>` | ASL query language | `jsonata` |
1819
| `--indent <N>` | JSON indentation spaces | `2` |
1920
| `-v, --verbose` | Verbose output | off |
2021
| `-h, --help` | Show help | |
@@ -33,6 +34,9 @@ npx simplesteps compile tsconfig.json -o dist/
3334

3435
# Custom indentation
3536
npx simplesteps compile src/workflow.ts --indent 4
37+
38+
# Compile with JSONPath instead of JSONata (default)
39+
npx simplesteps compile src/workflow.ts --query-language jsonpath
3640
```
3741

3842
## Input Detection

docs/getting-started.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,9 +123,12 @@ export const orderWorkflow = Steps.createFunction(
123123

124124
```bash
125125
npx simplesteps compile workflows/order.ts -o output/
126+
127+
# Use JSONPath instead of JSONata (the default)
128+
npx simplesteps compile workflows/order.ts -o output/ --query-language jsonpath
126129
```
127130

128-
This produces `output/orderWorkflow.asl.json` — useful for reviewing the generated state machine before deploying.
131+
This produces `output/orderWorkflow.asl.json` — useful for reviewing the generated state machine before deploying. The default query language is JSONata, which supports richer expressions (arithmetic, string methods, Math functions, higher-order array functions). Use `--query-language jsonpath` if you need the original ASL query language.
129132

130133
## Next Steps
131134

docs/intrinsic-functions.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ count += 5; // States.MathAdd($.count, 5)
5353
count -= 3; // States.MathAdd($.count, -3)
5454
```
5555

56-
Multiplication, division, and modulo are **not supported** -- ASL only provides `States.MathAdd`.
56+
In **JSONPath mode**, multiplication, division, and modulo are not supported ASL only provides `States.MathAdd`. In **JSONata mode** (the default), all arithmetic operators work natively: `*`, `/`, `%`, `-`, `+`.
5757

5858
## JSON (`States.StringToJson` / `States.JsonToString`)
5959

docs/language-reference.md

Lines changed: 64 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,62 @@ Every TypeScript construct supported by SimpleSteps and its ASL mapping.
2323
| `if (e instanceof TimeoutError)` | Typed error matching |
2424
| `.call(input, { retry, timeoutSeconds, heartbeatSeconds })` | Retry / Timeout / Heartbeat |
2525

26+
## Query Language
27+
28+
SimpleSteps supports two ASL query languages. JSONata is the default and recommended mode.
29+
30+
### JSONata (default)
31+
32+
JSONata provides native arithmetic, string manipulation, type conversion, higher-order array functions, and more — most JavaScript patterns compile directly:
33+
34+
```typescript
35+
// Arithmetic — all operators work
36+
const total = price * quantity;
37+
const tax = total * 0.08;
38+
const discounted = total - discount;
39+
40+
// String methods → $uppercase, $lowercase, $trim, $substring, $pad, $replace
41+
const upper = name.toUpperCase();
42+
const trimmed = raw.trim();
43+
const first5 = text.substring(0, 5);
44+
const padded = id.padStart(10, '0');
45+
46+
// Math functions → $floor, $ceil, $round, $abs, $power, $sqrt, $min, $max, $random
47+
const rounded = Math.round(amount);
48+
const clamped = Math.min(value, 100);
49+
50+
// Type conversion → $number, $string, $boolean, $type, $millis
51+
const num = Number(input.text);
52+
const keys = Object.keys(config);
53+
const priceType = typeof input.price;
54+
const now = Date.now();
55+
const isArr = Array.isArray(input.items);
56+
57+
// Array methods → $join, $reverse, $sort, $append
58+
const sorted = items.sort();
59+
const csv = items.join(', ');
60+
61+
// Higher-order functions → $map, $filter, $reduce
62+
const names = items.map(item => item.name);
63+
const active = items.filter(item => item.active);
64+
const total = items.reduce((sum, item) => sum + item.price, 0);
65+
const found = items.find(item => item.id === targetId);
66+
const hasActive = items.some(item => item.active);
67+
const allValid = items.every(item => item.valid);
68+
```
69+
70+
Higher-order functions require pure expression callbacks (no `await`). For callbacks with service calls, use `Steps.map()` instead (see below).
71+
72+
### JSONPath
73+
74+
The original ASL query language. Use `queryLanguage: 'JSONPath'` to opt in:
75+
76+
```typescript
77+
compile({ sourceFiles: ['workflow.ts'], queryLanguage: 'JSONPath' });
78+
```
79+
80+
JSONPath mode has more restrictions — no arithmetic beyond addition, no string/Math methods. See [Limitations](./limitations.md) for details.
81+
2682
## Entry Points
2783

2884
### `Steps.createFunction()`
@@ -257,17 +313,16 @@ Substeps can contain any supported control flow: `if/else`, `try/catch`, loops,
257313

258314
See [Limitations](./limitations.md#substeps) for constraints.
259315

260-
## Automatic Data Flow (No JSONPath)
316+
## Automatic Data Flow
261317

262-
One of SimpleSteps' key design goals: **you never write JSONPath or data flow fields**. The compiler derives all of them from your variable usage.
318+
One of SimpleSteps' key design goals: **you never write path expressions or data flow fields**. The compiler derives all of them from your variable usage.
263319

264-
| ASL Field | Derived From |
265-
|---|---|
266-
| `Parameters` | Service call arguments — the compiler maps each argument to a `"field.$": "$.variable"` reference |
267-
| `ResultPath` | Variable assignment — `const x = await svc.call(...)` stores the result at `$.x` |
268-
| `InputPath` | Variable references in service call arguments — the compiler determines what data the state needs |
269-
| `ResultSelector` | Automatic `.Payload` extraction for Lambda results (so you access `result.field`, not `result.Payload.field`) |
270-
| `OutputPath` | *Not yet implemented* — planned optimization for payload size reduction via variable liveness analysis |
320+
In **JSONata mode** (default), the compiler emits `Arguments` with `{% %}` expressions. In **JSONPath mode**, it emits `Parameters` with `"field.$": "$.variable"` references. Either way, you write plain TypeScript:
321+
322+
| Your Code | JSONata ASL | JSONPath ASL |
323+
|---|---|---|
324+
| `await svc.call({ id: x.id })` | `Arguments: { id: "{% $x.id %}" }` | `Parameters: { "id.$": "$.x.id" }` |
325+
| `const x = await svc.call(...)` | `Assign: { x: "{% $states.result %}" }` | `ResultPath: "$.x"` |
271326

272327
In CDK, you'd write `sfn.JsonPath.stringAt('$.order.total')` and manually set `outputPath: '$.Payload'`. In SimpleSteps, you write `order.total` and the compiler handles the rest.
273328

docs/library-api.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ interface CompileOptions {
3939
sourceFiles?: string[]; // Specific files to compile
4040
cwd?: string; // Working directory (default: process.cwd())
4141
substitutions?: Record<string, unknown>; // Deploy-time value overrides
42+
queryLanguage?: 'JSONata' | 'JSONPath'; // ASL query language (default: 'JSONata')
4243
}
4344
```
4445

0 commit comments

Comments
 (0)