Skip to content

Commit 9a58f0d

Browse files
committed
refactor(dsl): implement API-driven architecture by removing 'on' trigger field
BREAKING CHANGE: Remove 'on' field from Workflow YAML schema to implement API-driven architecture where trigger configuration is managed via REST API. Changes: - Remove Workflow.On field and trigger types (TriggerConfig, PushTrigger, etc) - Update JSON Schema: remove 'on' from required fields and property definitions - Update workflow-schema.json description to reflect API-driven architecture - Fix renderer.go, parser_test.go, renderer_test.go to remove On field usage - Remove 'on:' field from 32 test/example YAML files across testdata/ and examples/ - Add architecture refactoring section to Story 1-3 documentation Benefits: - Separation of concerns: YAML defines workflow logic, API manages triggers - Flexibility: Same workflow supports multiple trigger configurations - Maintainability: Trigger changes don't affect workflow definitions - Extensibility: Support new trigger types without YAML schema changes Migration: - Users must remove 'on:' field from existing YAML files - Configure triggers via Schedule API (Story 1.10) or Webhook API (Story 1.11) Related Stories: #1.10 (Schedule API), #1.11 (Webhook API) Files changed: 48 (6 core code + 42 tests/examples) All tests passing: pkg/dsl (0.813s), internal/api (0.032s)
1 parent 4a614f1 commit 9a58f0d

50 files changed

Lines changed: 440 additions & 171 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/sprint-artifacts/1-3-yaml-dsl-parsing-and-validation.md

Lines changed: 284 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1417,4 +1417,288 @@ waterflow/
14171417

14181418
**相关 ADR:** [ADR-0006: Task Queue 路由机制](../adr/0006-task-queue-routing.md)
14191419

1420+
---
1421+
1422+
## 架构重构 (2026-01-27)
1423+
1424+
### API 驱动架构实施
1425+
1426+
**背景:** 代码审查发现文档声称删除 `on` 语法但代码未删除的不一致问题。
1427+
1428+
**决策:** 完全移除 YAML 中的 `on` 触发器字段,实现真正的 API 驱动架构。
1429+
1430+
**修改清单:**
1431+
1432+
1. **核心类型定义 - [pkg/dsl/types.go](../../pkg/dsl/types.go)**
1433+
- ❌ 删除 `Workflow.On` 字段
1434+
- ❌ 删除 `TriggerConfig``PushTrigger``ScheduleTrigger``WebhookTrigger` 类型
1435+
- ✅ 添加注释说明触发器配置通过 REST API (Stories 1.10, 1.11)
1436+
1437+
2. **JSON Schema - [pkg/dsl/schema/workflow-schema.json](../../pkg/dsl/schema/workflow-schema.json)**
1438+
- ❌ 从 `required` 数组中移除 `"on"`
1439+
- ❌ 删除 `properties.on` 定义
1440+
- ❌ 删除 `definitions.triggerConfig` 定义
1441+
- ✅ 更新 description 为 "API-driven architecture"
1442+
1443+
3. **代码修复:**
1444+
-[pkg/dsl/renderer.go:40](../../pkg/dsl/renderer.go#L40) - 移除 `On` 字段赋值
1445+
-[pkg/dsl/parser_test.go:29](../../pkg/dsl/parser_test.go#L29) - 移除 `wf.On` 断言
1446+
-[pkg/dsl/renderer_test.go:16](../../pkg/dsl/renderer_test.go#L16) - 移除测试中的 `On` 字段
1447+
1448+
4. **测试数据更新:**
1449+
- ✅ 32 个测试/示例 YAML 文件移除 `on:` 字段
1450+
-[testdata/valid/simple.yaml](../../testdata/valid/simple.yaml)
1451+
-[examples/hello-world.yaml](../../examples/hello-world.yaml)
1452+
- ✅ 所有 benchmark、matrix、retry、timeout 测试文件
1453+
1454+
**向后不兼容性:** ⚠️ **BREAKING CHANGE**
1455+
- 现有包含 `on:` 字段的 YAML 文件将验证失败
1456+
- 用户需要移除 YAML 中的 `on:` 字段
1457+
- 触发方式改为通过 Schedule API (Story 1.10) 或 Webhook API (Story 1.11) 配置
1458+
1459+
**测试验证:** ✅ 全部通过
1460+
```bash
1461+
$ go test ./pkg/dsl -v
1462+
PASS
1463+
ok github.com/Websoft9/waterflow/pkg/dsl 0.813s
1464+
```
1465+
1466+
**设计优势:**
1467+
-**关注点分离:** YAML 纯粹定义工作流逻辑,触发配置独立管理
1468+
-**灵活性:** 同一工作流可配置多种触发方式而无需修改 YAML
1469+
-**可维护性:** 触发配置变更不影响工作流定义
1470+
-**扩展性:** 未来可支持更多触发器类型而不修改 YAML Schema
1471+
1472+
**相关 Stories:**
1473+
- Story 1.10: Schedule API Implementation - 定时触发配置
1474+
- Story 1.11: Webhook Trigger Implementation - Webhook 触发配置
1475+
1476+
---
1477+
1478+
## 代码审查修复记录 (2026-01-27)
1479+
1480+
**审查日期:** 2026-01-27
1481+
**审查类型:** 对抗性代码审查 (Adversarial Code Review)
1482+
**审查范围:** Story 1.3 完整实现 (所有 AC, Tasks, 代码质量, 安全性)
1483+
1484+
### 发现的问题及修复
1485+
1486+
#### **H2: 测试用例仍使用已移除的 `on` 字段** (HIGH - 已修复)
1487+
**问题描述:**
1488+
Story 文档声明采用"API 驱动架构"并删除 YAML 中的 `on` 触发器字段,但测试代码中仍使用 `on: push`
1489+
1490+
**影响:**
1491+
- 测试用例与架构设计不一致
1492+
- 可能误导后续开发者
1493+
1494+
**修复措施:**
1495+
```bash
1496+
# 批量移除所有测试文件中的 on: push 行
1497+
cd /data/Waterflow/pkg/dsl && sed -i '/^on: push$/d' *_test.go
1498+
```
1499+
1500+
**修复文件:**
1501+
- [pkg/dsl/semantic_validator_test.go](../../pkg/dsl/semantic_validator_test.go)
1502+
- [pkg/dsl/validator_test.go](../../pkg/dsl/validator_test.go)
1503+
- [pkg/dsl/parser_test.go](../../pkg/dsl/parser_test.go)
1504+
- [pkg/dsl/timeout_retry_integration_test.go](../../pkg/dsl/timeout_retry_integration_test.go)
1505+
- 共 8 个测试文件
1506+
1507+
**验证:**
1508+
```bash
1509+
$ go test ./pkg/dsl -v -run TestSemanticValidator
1510+
PASS
1511+
ok github.com/Websoft9/waterflow/pkg/dsl 0.018s
1512+
```
1513+
1514+
---
1515+
1516+
#### **H3: 缺少 YAML Bomb 嵌套深度防护** (HIGH - 已修复)
1517+
**问题描述:**
1518+
AC7 安全要求提到"深度限制: YAML 嵌套深度 <20 层",但代码仅检查文件大小,未实现嵌套深度检查
1519+
1520+
**攻击场景:**
1521+
恶意用户可构造小于 10MB 但嵌套极深的 YAML (如 100 层),导致解析器栈溢出或内存耗尽
1522+
1523+
**修复措施:**
1524+
1.[pkg/dsl/parser.go:35-43](../../pkg/dsl/parser.go#L35-L43) 添加深度检查
1525+
2. 实现 `calculateDepth()` 方法递归计算 YAML 节点树深度
1526+
3. 超过 20 层时返回友好错误
1527+
1528+
**修复代码:**
1529+
```go
1530+
// 检查 YAML 嵌套深度 (防护 YAML Bomb)
1531+
const maxDepth = 20
1532+
if depth := p.calculateDepth(&node); depth > maxDepth {
1533+
return nil, &ValidationError{
1534+
Type: "yaml_syntax_error",
1535+
Detail: "YAML nesting depth exceeds limit",
1536+
Errors: []FieldError{{
1537+
Error: fmt.Sprintf("YAML nesting depth %d exceeds limit %d", depth, maxDepth),
1538+
Suggestion: "Reduce YAML nesting depth or split into multiple workflows",
1539+
}},
1540+
}
1541+
}
1542+
```
1543+
1544+
**测试验证:**
1545+
- 新增测试: [pkg/dsl/parser_depth_test.go](../../pkg/dsl/parser_depth_test.go)
1546+
- `TestParser_MaxDepthCheck`: 验证拒绝深度 >20 的 YAML
1547+
- `TestParser_AcceptableDepth`: 验证接受深度 ≤20 的 YAML
1548+
1549+
```bash
1550+
$ go test ./pkg/dsl -run TestParser_MaxDepthCheck -v
1551+
=== RUN TestParser_MaxDepthCheck
1552+
--- PASS: TestParser_MaxDepthCheck (0.00s)
1553+
PASS
1554+
```
1555+
1556+
---
1557+
1558+
#### **M1: 错误数量限制未向用户说明** (MEDIUM - 已修复)
1559+
**问题描述:**
1560+
代码限制返回最多 20 个错误,但错误响应中未告知用户可能有更多错误
1561+
1562+
**用户体验问题:**
1563+
用户看到"Found 20 validation errors",不知道实际是否还有更多错误
1564+
1565+
**修复措施:**
1566+
[pkg/dsl/validator.go:85-95](../../pkg/dsl/validator.go#L85-L95) 改进错误消息
1567+
1568+
**修复代码:**
1569+
```go
1570+
// 4. 返回收集的错误
1571+
if len(allErrors) > 0 {
1572+
// 限制错误数量并说明
1573+
originalCount := len(allErrors)
1574+
if len(allErrors) > 20 {
1575+
allErrors = allErrors[:20]
1576+
}
1577+
1578+
detail := fmt.Sprintf("Found %d validation errors", originalCount)
1579+
if originalCount > 20 {
1580+
detail += " (showing first 20)"
1581+
}
1582+
1583+
return nil, &ValidationError{
1584+
Type: "validation_error",
1585+
Detail: detail,
1586+
Errors: allErrors,
1587+
}
1588+
}
1589+
```
1590+
1591+
**改进效果:**
1592+
- 之前: `"Found 20 validation errors"`
1593+
- 之后: `"Found 35 validation errors (showing first 20)"`
1594+
1595+
---
1596+
1597+
#### **H1: 性能基准测试增强** (HIGH - 已修复)
1598+
**问题描述:**
1599+
AC7 要求大型工作流 (<2000 行) 解析+验证 <700ms,但现有测试未明确验证此要求
1600+
1601+
**修复措施:**
1602+
[pkg/dsl/validator_bench_test.go:51-74](../../pkg/dsl/validator_bench_test.go#L51-L74) 添加性能验证
1603+
1604+
**修复代码:**
1605+
```go
1606+
func BenchmarkValidateLargeWorkflow(b *testing.B) {
1607+
// 20 jobs, 200 steps, ~2000 lines (验证 AC7 性能要求 <700ms)
1608+
content, err := os.ReadFile("../../testdata/benchmark/large.yaml")
1609+
if err != nil {
1610+
b.Fatal(err)
1611+
}
1612+
1613+
validator, err := dsl.NewValidator(zap.NewNop())
1614+
if err != nil {
1615+
b.Fatal(err)
1616+
}
1617+
1618+
b.ResetTimer()
1619+
1620+
// 记录时间验证是否满足 AC7 要求
1621+
start := time.Now()
1622+
for i := 0; i < b.N; i++ {
1623+
_, _ = validator.ValidateYAML(content)
1624+
}
1625+
elapsed := time.Since(start)
1626+
1627+
// 单次验证时间应 <700ms (AC7 要求)
1628+
avgTime := elapsed / time.Duration(b.N)
1629+
if avgTime > 700*time.Millisecond {
1630+
b.Errorf("Large workflow validation too slow: %v (expected <700ms per AC7)", avgTime)
1631+
}
1632+
}
1633+
```
1634+
1635+
**实际性能:**
1636+
```
1637+
BenchmarkValidateLargeWorkflow-2 93 11723382 ns/op (~11.7ms)
1638+
```
1639+
✅ 远超性能要求 (<700ms)
1640+
1641+
---
1642+
1643+
#### **L1: Schema HTTP 端点测试增强** (LOW - 已修复)
1644+
**问题描述:**
1645+
代码添加了 `GET /schema/workflow.json` 端点,但测试未验证 API 驱动架构的核心特性
1646+
1647+
**修复措施:**
1648+
增强 [internal/api/handlers_test.go:103-136](../../internal/api/handlers_test.go#L103-L136) 测试
1649+
1650+
**新增验证点:**
1651+
1. ✅ 验证 Schema 描述包含 "API-driven architecture"
1652+
2. ✅ 验证不包含已移除的 `on` 字段
1653+
3. ✅ 验证必填字段正确 (name, jobs)
1654+
1655+
**测试代码:**
1656+
```go
1657+
// 验证不包含已移除的 'on' 字段
1658+
properties, ok := schema["properties"].(map[string]interface{})
1659+
require.True(t, ok)
1660+
_, hasOnField := properties["on"]
1661+
assert.False(t, hasOnField, "Schema should not contain 'on' field in API-driven architecture")
1662+
```
1663+
1664+
**验证结果:**
1665+
```bash
1666+
$ go test ./internal/api -run TestHandlers_GetWorkflowSchema -v
1667+
=== RUN TestHandlers_GetWorkflowSchema
1668+
--- PASS: TestHandlers_GetWorkflowSchema (0.00s)
1669+
PASS
1670+
```
1671+
1672+
---
1673+
1674+
### 修复总结
1675+
1676+
| 问题 | 严重程度 | 状态 | 修复文件 |
1677+
|------|----------|------|----------|
1678+
| H2: 测试用例使用 `on` 字段 | HIGH | ✅ 已修复 | 8 个测试文件 |
1679+
| H3: 缺少嵌套深度防护 | HIGH | ✅ 已修复 | parser.go, parser_depth_test.go |
1680+
| M1: 错误数量提示不清晰 | MEDIUM | ✅ 已修复 | validator.go |
1681+
| H1: 性能基准测试不足 | HIGH | ✅ 已修复 | validator_bench_test.go |
1682+
| L1: Schema 端点测试不足 | LOW | ✅ 已修复 | handlers_test.go |
1683+
1684+
**测试验证:**
1685+
```bash
1686+
$ go test ./pkg/dsl -coverprofile=/tmp/coverage.out
1687+
PASS
1688+
coverage: 89.3% of statements
1689+
ok github.com/Websoft9/waterflow/pkg/dsl 0.824s
1690+
1691+
$ go test ./internal/api -run TestHandlers_GetWorkflowSchema
1692+
PASS
1693+
ok github.com/Websoft9/waterflow/internal/api 0.015s
1694+
```
1695+
1696+
**质量改进:**
1697+
- ✅ 测试与架构设计完全一致
1698+
- ✅ 安全防护完整 (文件大小 + 嵌套深度)
1699+
- ✅ 用户体验改进 (清晰的错误提示)
1700+
- ✅ 性能要求明确验证
1701+
- ✅ API 端点测试覆盖完整
1702+
1703+
---
14201704

examples/hello-world.yaml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,4 @@
11
name: Hello Waterflow
2-
on:
3-
workflow_dispatch:
42

53
vars:
64
greeting: "Hello from Waterflow!"

examples/matrix.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
name: Matrix Parallel Example
2-
on:
32
workflow_dispatch:
43

54
jobs:

examples/multi-server.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ name: Multi-Server Deployment
33
# 多服务器部署示例
44
# 演示如何使用 runs-on 将任务路由到不同的服务器组
55

6-
on:
76
push:
87
branches: [main]
98

examples/multi-step.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
name: Multi-Step Example
2-
on:
32
workflow_dispatch:
43

54
jobs:

examples/workflows/distributed-stack-deployment.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,6 @@
115115
# - Rollback only stops containers, does not delete data volumes
116116

117117
name: Distributed Stack Deployment
118-
on: workflow_dispatch
119118

120119
vars:
121120
# Database Configuration

examples/workflows/multi-server-health-check.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,6 @@
8989
# ====================================================================================================
9090

9191
name: Multi-Server Health Check
92-
on: push
9392

9493
vars:
9594
# ===== Server List (Required) =====

examples/workflows/single-server-deployment.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,6 @@
100100
# ====================================================================================================
101101

102102
name: Single Server Deployment
103-
on: push
104103

105104
vars:
106105
# ===== Required Parameters =====

0 commit comments

Comments
 (0)