Skip to content

Commit 5f28c2d

Browse files
voyvodkavoyvodka
committed
feat(endpoints): add payload transformation schema and API (ADR-003 Phase 1)
Per ADR-003, Phase 1 stores transformation configuration on endpoints without applying it during delivery. Schema additions on the endpoints table: - transform_expression (varchar(4096), nullable) — JMESPath expression to apply - transform_enabled (boolean, default false) — kill switch per endpoint - transform_validated_at (timestamptz, nullable) — last successful validation time Migration generated via dotnet ef migrations add includes Up/Down operations and updates ModelSnapshot for downstream diffs. API surface: CreateEndpointRequest / UpdateEndpointRequest and the dashboard equivalents accept optional TransformExpression and TransformEnabled fields. EndpointResponseDto, the EndpointListItem repository projection, and the dashboard endpoint create/update inline responses expose all three fields. FluentValidation rules enforce the 4096-character cap, and the UpdateEndpointRequest "at least one field" guard now treats Transform fields as valid inputs. Setting a non-empty TransformExpression on update resets TransformValidatedAt to null so a fresh validation pass is required before the expression is trusted. Pipeline integration (HttpDeliveryService applying the expression with JmesPath.Net + 100 ms timeout + fail-open) and the dashboard expression editor remain in ADR-003 Phase 2 and Phase 3. The optional /transform/validate endpoint will land alongside the JmesPath.Net dependency in Phase 2.
1 parent 4532185 commit 5f28c2d

11 files changed

Lines changed: 799 additions & 5 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
77

88
## [Unreleased]
99

10+
### Added
11+
- **Payload transformation schema and API (ADR-003 Phase 1):** endpoints now accept `transformExpression` (JMESPath, max 4096 chars), `transformEnabled` (kill switch, default `false`), and a server-managed `transformValidatedAt` timestamp on create/update. Both the public Bearer-key API (`POST /api/v1/endpoints`, `PUT /api/v1/endpoints/{id}`) and the dashboard endpoints (`POST /api/v1/dashboard/endpoints`, `PUT /api/v1/dashboard/endpoints/{id}`) carry the new fields, and `EndpointResponseDto` exposes them on read. Stored only — pipeline integration (delivery-time application with `JmesPath.Net` + 100 ms timeout + fail-open) and the dashboard editor land in ADR-003 Phase 2 and Phase 3 respectively.
12+
- **Security automations:** CodeQL workflow (csharp + javascript-typescript, push/PR/Mondays at 06:30 UTC), Dependency Review action on PRs (high-severity fail + GPL/LGPL/AGPL/EUPL/SSPL deny-list), and Dependabot config covering NuGet, npm, GitHub Actions, and Docker base images. Five repo labels (`dependencies`, `nuget`, `npm`, `ci`, `docker`) created to support the Dependabot config.
13+
1014
### Changed
1115
- **Frontend toolchain:** migrated dashboard package manager from Yarn to [Bun](https://bun.sh/) 1.2+. `yarn.lock` replaced with `bun.lock` (text format introduced in Bun 1.2); CI, Dockerfile, contributor docs, and PR template now reference `bun` commands. No runtime behavior changes.
1216

‎src/WebhookEngine.API/Contracts/ApiResponseDtos.cs‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@ public sealed class EndpointResponseDto
2525
public JsonElement CustomHeadersJson { get; init; }
2626
public string? SecretOverride { get; init; }
2727
public JsonElement MetadataJson { get; init; }
28+
public string? TransformExpression { get; init; }
29+
public bool TransformEnabled { get; init; }
30+
public DateTime? TransformValidatedAt { get; init; }
2831
public List<Guid> FilterEventTypes { get; init; } = [];
2932
public DateTime CreatedAt { get; init; }
3033
public DateTime UpdatedAt { get; init; }
@@ -89,6 +92,9 @@ public static EndpointResponseDto ToDto(this EndpointEntity endpoint)
8992
CustomHeadersJson = JsonValueParser.ParseOrEmptyObject(endpoint.CustomHeadersJson),
9093
SecretOverride = endpoint.SecretOverride,
9194
MetadataJson = JsonValueParser.ParseOrEmptyObject(endpoint.MetadataJson),
95+
TransformExpression = endpoint.TransformExpression,
96+
TransformEnabled = endpoint.TransformEnabled,
97+
TransformValidatedAt = endpoint.TransformValidatedAt,
9298
FilterEventTypes = endpoint.EventTypes.Select(et => et.Id).ToList(),
9399
CreatedAt = endpoint.CreatedAt,
94100
UpdatedAt = endpoint.UpdatedAt
@@ -107,6 +113,9 @@ public static EndpointResponseDto ToDto(this EndpointListItem endpoint)
107113
CustomHeadersJson = JsonValueParser.ParseOrEmptyObject(endpoint.CustomHeadersJson),
108114
SecretOverride = endpoint.SecretOverride,
109115
MetadataJson = JsonValueParser.ParseOrEmptyObject(endpoint.MetadataJson),
116+
TransformExpression = endpoint.TransformExpression,
117+
TransformEnabled = endpoint.TransformEnabled,
118+
TransformValidatedAt = endpoint.TransformValidatedAt,
110119
FilterEventTypes = endpoint.EventTypeIds,
111120
CreatedAt = endpoint.CreatedAt,
112121
UpdatedAt = endpoint.UpdatedAt

‎src/WebhookEngine.API/Controllers/DashboardEndpointController.cs‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,9 @@ public async Task<IActionResult> CreateEndpoint([FromBody] DashboardCreateEndpoi
8383
Description = request.Description,
8484
SecretOverride = string.IsNullOrWhiteSpace(request.SecretOverride) ? null : request.SecretOverride,
8585
CustomHeadersJson = System.Text.Json.JsonSerializer.Serialize(request.CustomHeaders ?? new Dictionary<string, string>()),
86-
MetadataJson = System.Text.Json.JsonSerializer.Serialize(request.Metadata ?? new Dictionary<string, string>())
86+
MetadataJson = System.Text.Json.JsonSerializer.Serialize(request.Metadata ?? new Dictionary<string, string>()),
87+
TransformExpression = string.IsNullOrWhiteSpace(request.TransformExpression) ? null : request.TransformExpression,
88+
TransformEnabled = request.TransformEnabled ?? false
8789
};
8890

8991
if (request.FilterEventTypes is not null && request.FilterEventTypes.Count > 0)
@@ -117,6 +119,9 @@ public async Task<IActionResult> CreateEndpoint([FromBody] DashboardCreateEndpoi
117119
circuitState = created?.Health?.CircuitState.ToString().ToLowerInvariant() ?? "closed",
118120
eventTypes = created?.EventTypes.Select(et => et.Name).ToList() ?? [],
119121
eventTypeIds = created?.EventTypes.Select(et => et.Id).ToList() ?? [],
122+
transformExpression = endpoint.TransformExpression,
123+
transformEnabled = endpoint.TransformEnabled,
124+
transformValidatedAt = endpoint.TransformValidatedAt,
120125
createdAt = endpoint.CreatedAt,
121126
updatedAt = endpoint.UpdatedAt
122127
}));
@@ -147,6 +152,16 @@ public async Task<IActionResult> UpdateEndpoint(Guid endpointId, [FromBody] Dash
147152
if (request.Metadata is not null)
148153
endpoint.MetadataJson = System.Text.Json.JsonSerializer.Serialize(request.Metadata);
149154

155+
if (request.TransformExpression is not null)
156+
{
157+
// Empty/whitespace clears the expression; any new expression resets validation timestamp.
158+
endpoint.TransformExpression = string.IsNullOrWhiteSpace(request.TransformExpression) ? null : request.TransformExpression;
159+
endpoint.TransformValidatedAt = null;
160+
}
161+
162+
if (request.TransformEnabled is not null)
163+
endpoint.TransformEnabled = request.TransformEnabled.Value;
164+
150165
if (request.FilterEventTypes is not null)
151166
{
152167
endpoint.EventTypes.Clear();
@@ -180,6 +195,9 @@ public async Task<IActionResult> UpdateEndpoint(Guid endpointId, [FromBody] Dash
180195
circuitState = updated?.Health?.CircuitState.ToString().ToLowerInvariant() ?? "closed",
181196
eventTypes = updated?.EventTypes.Select(et => et.Name).ToList() ?? [],
182197
eventTypeIds = updated?.EventTypes.Select(et => et.Id).ToList() ?? [],
198+
transformExpression = endpoint.TransformExpression,
199+
transformEnabled = endpoint.TransformEnabled,
200+
transformValidatedAt = endpoint.TransformValidatedAt,
183201
createdAt = endpoint.CreatedAt,
184202
updatedAt = endpoint.UpdatedAt
185203
}));
@@ -375,6 +393,8 @@ public class DashboardCreateEndpointRequest
375393
public Dictionary<string, string>? CustomHeaders { get; set; }
376394
public Dictionary<string, string>? Metadata { get; set; }
377395
public string? SecretOverride { get; set; }
396+
public string? TransformExpression { get; set; }
397+
public bool? TransformEnabled { get; set; }
378398
}
379399

380400
public class DashboardUpdateEndpointRequest
@@ -385,6 +405,8 @@ public class DashboardUpdateEndpointRequest
385405
public Dictionary<string, string>? CustomHeaders { get; set; }
386406
public Dictionary<string, string>? Metadata { get; set; }
387407
public string? SecretOverride { get; set; }
408+
public string? TransformExpression { get; set; }
409+
public bool? TransformEnabled { get; set; }
388410
}
389411

390412
public class DashboardCreateEventTypeRequest

‎src/WebhookEngine.API/Controllers/EndpointsController.cs‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,9 @@ public async Task<IActionResult> Create([FromBody] CreateEndpointRequest request
3737
Url = request.Url,
3838
Description = request.Description,
3939
CustomHeadersJson = System.Text.Json.JsonSerializer.Serialize(request.CustomHeaders ?? new Dictionary<string, string>()),
40-
MetadataJson = System.Text.Json.JsonSerializer.Serialize(request.Metadata ?? new Dictionary<string, string>())
40+
MetadataJson = System.Text.Json.JsonSerializer.Serialize(request.Metadata ?? new Dictionary<string, string>()),
41+
TransformExpression = string.IsNullOrWhiteSpace(request.TransformExpression) ? null : request.TransformExpression,
42+
TransformEnabled = request.TransformEnabled ?? false
4143
};
4244

4345
if (request.FilterEventTypes is not null && request.FilterEventTypes.Count > 0)
@@ -114,6 +116,16 @@ public async Task<IActionResult> Update(Guid endpointId, [FromBody] UpdateEndpoi
114116
if (request.SecretOverride is not null)
115117
endpoint.SecretOverride = string.IsNullOrWhiteSpace(request.SecretOverride) ? null : request.SecretOverride;
116118

119+
if (request.TransformExpression is not null)
120+
{
121+
// Treat empty/whitespace as a clear, otherwise store the new expression and reset validation timestamp.
122+
endpoint.TransformExpression = string.IsNullOrWhiteSpace(request.TransformExpression) ? null : request.TransformExpression;
123+
endpoint.TransformValidatedAt = null;
124+
}
125+
126+
if (request.TransformEnabled is not null)
127+
endpoint.TransformEnabled = request.TransformEnabled.Value;
128+
117129
if (request.FilterEventTypes is not null)
118130
{
119131
endpoint.EventTypes.Clear();
@@ -236,6 +248,8 @@ public class CreateEndpointRequest
236248
public List<Guid>? FilterEventTypes { get; set; }
237249
public Dictionary<string, string>? CustomHeaders { get; set; }
238250
public Dictionary<string, string>? Metadata { get; set; }
251+
public string? TransformExpression { get; set; }
252+
public bool? TransformEnabled { get; set; }
239253
}
240254

241255
public class UpdateEndpointRequest
@@ -246,4 +260,6 @@ public class UpdateEndpointRequest
246260
public Dictionary<string, string>? CustomHeaders { get; set; }
247261
public Dictionary<string, string>? Metadata { get; set; }
248262
public string? SecretOverride { get; set; }
263+
public string? TransformExpression { get; set; }
264+
public bool? TransformEnabled { get; set; }
249265
}

‎src/WebhookEngine.API/Validators/RequestValidators.cs‎

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,11 @@ public CreateEndpointRequestValidator()
7676
RuleFor(x => x.Description)
7777
.MaximumLength(500)
7878
.When(x => x.Description is not null);
79+
80+
RuleFor(x => x.TransformExpression)
81+
.MaximumLength(4096)
82+
.When(x => x.TransformExpression is not null)
83+
.WithMessage("TransformExpression must not exceed 4096 characters.");
7984
}
8085

8186
private static bool BeValidHttpsUrl(string? url)
@@ -103,13 +108,20 @@ public UpdateEndpointRequestValidator()
103108
.MaximumLength(500)
104109
.When(x => x.Description is not null);
105110

111+
RuleFor(x => x.TransformExpression)
112+
.MaximumLength(4096)
113+
.When(x => x.TransformExpression is not null)
114+
.WithMessage("TransformExpression must not exceed 4096 characters.");
115+
106116
RuleFor(x => x)
107117
.Must(x => x.Url is not null
108118
|| x.Description is not null
109119
|| x.FilterEventTypes is not null
110120
|| x.CustomHeaders is not null
111121
|| x.Metadata is not null
112-
|| x.SecretOverride is not null)
122+
|| x.SecretOverride is not null
123+
|| x.TransformExpression is not null
124+
|| x.TransformEnabled is not null)
113125
.WithMessage("At least one field must be provided.");
114126
}
115127

@@ -140,6 +152,11 @@ public DashboardCreateEndpointRequestValidator()
140152
RuleFor(x => x.Description)
141153
.MaximumLength(500)
142154
.When(x => x.Description is not null);
155+
156+
RuleFor(x => x.TransformExpression)
157+
.MaximumLength(4096)
158+
.When(x => x.TransformExpression is not null)
159+
.WithMessage("TransformExpression must not exceed 4096 characters.");
143160
}
144161

145162
private static bool BeValidHttpsUrl(string? url)
@@ -167,13 +184,20 @@ public DashboardUpdateEndpointRequestValidator()
167184
.MaximumLength(500)
168185
.When(x => x.Description is not null);
169186

187+
RuleFor(x => x.TransformExpression)
188+
.MaximumLength(4096)
189+
.When(x => x.TransformExpression is not null)
190+
.WithMessage("TransformExpression must not exceed 4096 characters.");
191+
170192
RuleFor(x => x)
171193
.Must(x => x.Url is not null
172194
|| x.Description is not null
173195
|| x.FilterEventTypes is not null
174196
|| x.CustomHeaders is not null
175197
|| x.Metadata is not null
176-
|| x.SecretOverride is not null)
198+
|| x.SecretOverride is not null
199+
|| x.TransformExpression is not null
200+
|| x.TransformEnabled is not null)
177201
.WithMessage("At least one field must be provided.");
178202
}
179203

‎src/WebhookEngine.Core/Entities/Endpoint.cs‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,14 @@ public class Endpoint
1212
public string CustomHeadersJson { get; set; } = "{}";
1313
public string? SecretOverride { get; set; }
1414
public string MetadataJson { get; set; } = "{}";
15+
16+
// Payload transformation (ADR-003) — declarative JMESPath reshape applied
17+
// before delivery. Stored only in Phase 1; pipeline integration arrives in
18+
// Phase 2 (HttpDeliveryService) and the dashboard editor in Phase 3.
19+
public string? TransformExpression { get; set; }
20+
public bool TransformEnabled { get; set; }
21+
public DateTime? TransformValidatedAt { get; set; }
22+
1523
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
1624
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
1725

‎src/WebhookEngine.Infrastructure/Data/WebhookDbContext.cs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,9 @@ protected override void OnModelCreating(ModelBuilder modelBuilder)
7575
entity.Property(e => e.CustomHeadersJson).HasColumnName("custom_headers").HasColumnType("jsonb").HasDefaultValueSql("'{}'");
7676
entity.Property(e => e.SecretOverride).HasColumnName("secret_override").HasMaxLength(64);
7777
entity.Property(e => e.MetadataJson).HasColumnName("metadata").HasColumnType("jsonb").HasDefaultValueSql("'{}'");
78+
entity.Property(e => e.TransformExpression).HasColumnName("transform_expression").HasMaxLength(4096);
79+
entity.Property(e => e.TransformEnabled).HasColumnName("transform_enabled").HasDefaultValue(false);
80+
entity.Property(e => e.TransformValidatedAt).HasColumnName("transform_validated_at");
7881
entity.Property(e => e.CreatedAt).HasColumnName("created_at").HasDefaultValueSql("NOW()");
7982
entity.Property(e => e.UpdatedAt).HasColumnName("updated_at").HasDefaultValueSql("NOW()");
8083

0 commit comments

Comments
 (0)