You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/adr/005-filesystem-serdes.md
+49-49Lines changed: 49 additions & 49 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,55 +25,6 @@ There are a few Java-specific constraints:
25
25
- Filesystem-backed storage is optional and storage-specific. It should not add filesystem-oriented public surface area to the core SDK artifact.
26
26
- Filesystem persistence is not automatically durable. Lambda `/tmp` is not valid for replay across environments. Mounted S3 Files may have delayed synchronization and can lose recent writes if the runtime crashes before the mount flushes. EFS or an explicitly accepted S3 Files durability tradeoff should be required for production use.
27
27
28
-
## Shared Design Constraints
29
-
30
-
These constraints apply to both approaches below.
31
-
32
-
### Stable payload identity
33
-
34
-
Both approaches need a stable payload identity that can be used to address external storage. The identity must include the durable execution ARN, operation identity, payload kind, and enough operation metadata to distinguish result, input, callback, wait-for-condition state, and exception payloads.
35
-
36
-
`entityId` is the primary stable key for external storage. It must be unique within a durable execution and include the payload kind so one operation can safely store multiple values:
| Invoke result |`operation/<operation-id>/result`|
47
-
| Callback result |`operation/<operation-id>/result`|
48
-
| Child context result |`operation/<operation-id>/result`|
49
-
| Map result |`operation/<operation-id>/result`|
50
-
| WaitForCondition state |`operation/<operation-id>/state`|
51
-
52
-
Do not include the checkpoint token or raw user payload in the context.
53
-
54
-
### Extra package pattern
55
-
56
-
Payload offloading implementations should live outside the core SDK artifact when they target a specific storage mechanism.
57
-
58
-
Use the `aws-durable-execution-sdk-java-extra-xxx` artifact pattern. The filesystem payload package name depends on which approach is chosen; the repository should not publish both a filesystem SerDes package and a filesystem offloader package for the same feature.
59
-
60
-
| Feature | Artifact ID | Java package |
61
-
|---------|-------------|--------------|
62
-
| Filesystem payload storage, Approach A |`aws-durable-execution-sdk-java-extra-filesystem-serdes`|`software.amazon.lambda.durable.extra.filesystem`|
63
-
| Filesystem payload storage, Approach B |`aws-durable-execution-sdk-java-extra-filesystem-offloader`|`software.amazon.lambda.durable.extra.filesystem`|
Extra modules should be independently documented, tested, and versioned with the repository release. They may depend on the core SDK and normal support libraries, but the core SDK should expose stable extension points without knowing about any specific extra package. For filesystem payload storage, create exactly one extra module after choosing Approach A or Approach B.
68
-
69
-
### Protocol SerDes boundary
70
-
71
-
Do not make `DurableInputOutputSerDes` customizable as part of this ADR. It serializes the Lambda Durable Functions backend protocol envelope, not user payloads. Routing that envelope through external payload storage would risk storing checkpoint tokens or protocol data externally and would require the backend to understand file pointers.
72
-
73
-
User input and output payloads should still use the configured user payload mechanism when they are extracted from or written to the execution operation. The internal `DurableExecutionInput` and `DurableExecutionOutput` envelope remains handled by `DurableInputOutputSerDes`.
74
-
75
-
If protocol customization is needed later, introduce a separate `ProtocolSerDes` configuration surface with a clear warning that it must produce the exact backend wire format. Do not reuse the user payload `SerDes` for that purpose.
76
-
77
28
## Approach A: Reuse SerDes for Offload
78
29
79
30
### Summary
@@ -532,3 +483,52 @@ Deferred:
532
483
- A fully async Java SerDes or payload pipeline contract.
533
484
- A separate, explicitly dangerous protocol-envelope customization API.
534
485
- File cleanup, retention policies, and lifecycle management for offloaded payloads.
486
+
487
+
## Shared Design Constraints
488
+
489
+
These constraints apply to both approaches above.
490
+
491
+
### Stable payload identity
492
+
493
+
Both approaches need a stable payload identity that can be used to address external storage. The identity must include the durable execution ARN, operation identity, payload kind, and enough operation metadata to distinguish result, input, callback, wait-for-condition state, and exception payloads.
494
+
495
+
`entityId` is the primary stable key for external storage. It must be unique within a durable execution and include the payload kind so one operation can safely store multiple values:
| Invoke result |`operation/<operation-id>/result`|
506
+
| Callback result |`operation/<operation-id>/result`|
507
+
| Child context result |`operation/<operation-id>/result`|
508
+
| Map result |`operation/<operation-id>/result`|
509
+
| WaitForCondition state |`operation/<operation-id>/state`|
510
+
511
+
Do not include the checkpoint token or raw user payload in the context.
512
+
513
+
### Extra package pattern
514
+
515
+
Payload offloading implementations should live outside the core SDK artifact when they target a specific storage mechanism.
516
+
517
+
Use the `aws-durable-execution-sdk-java-extra-xxx` artifact pattern. The filesystem payload package name depends on which approach is chosen; the repository should not publish both a filesystem SerDes package and a filesystem offloader package for the same feature.
518
+
519
+
| Feature | Artifact ID | Java package |
520
+
|---------|-------------|--------------|
521
+
| Filesystem payload storage, Approach A |`aws-durable-execution-sdk-java-extra-filesystem-serdes`|`software.amazon.lambda.durable.extra.filesystem`|
522
+
| Filesystem payload storage, Approach B |`aws-durable-execution-sdk-java-extra-filesystem-offloader`|`software.amazon.lambda.durable.extra.filesystem`|
Extra modules should be independently documented, tested, and versioned with the repository release. They may depend on the core SDK and normal support libraries, but the core SDK should expose stable extension points without knowing about any specific extra package. For filesystem payload storage, create exactly one extra module after choosing Approach A or Approach B.
527
+
528
+
### Protocol SerDes boundary
529
+
530
+
Do not make `DurableInputOutputSerDes` customizable as part of this ADR. It serializes the Lambda Durable Functions backend protocol envelope, not user payloads. Routing that envelope through external payload storage would risk storing checkpoint tokens or protocol data externally and would require the backend to understand file pointers.
531
+
532
+
User input and output payloads should still use the configured user payload mechanism when they are extracted from or written to the execution operation. The internal `DurableExecutionInput` and `DurableExecutionOutput` envelope remains handled by `DurableInputOutputSerDes`.
533
+
534
+
If protocol customization is needed later, introduce a separate `ProtocolSerDes` configuration surface with a clear warning that it must produce the exact backend wire format. Do not reuse the user payload `SerDes` for that purpose.
0 commit comments