Skip to content

Commit 93d1e50

Browse files
Bueller87Bueller87
authored
docs(blog): Your Workflow History Is Storing More Than You Think (#352)
1 parent 7f9a5bc commit 93d1e50

7 files changed

Lines changed: 562 additions & 3 deletions

File tree

‎.github/workflows/link-check-pr.yml‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,21 @@ jobs:
5454
with:
5555
# --scheme http --scheme https restricts lychee to absolute
5656
# HTTP(S) URLs only (i.e., external links). See header comment.
57-
args: --no-progress --scheme http --scheme https --config lychee.toml ${{ steps.changed.outputs.all_changed_files }}
57+
#
58+
# --root-dir is required so lychee can resolve root-relative
59+
# paths like /docs/... or /img/... that are common in MDX
60+
# (href="/docs/foo", src="/img/bar.svg"). Without it, lychee
61+
# errors at URI construction *before* the --scheme filter runs,
62+
# causing false-positive failures on internal links. With
63+
# --root-dir set, those links become file:// URIs and are then
64+
# correctly filtered out by --scheme http --scheme https.
65+
#
66+
# Must be an absolute path; github.workspace resolves to the
67+
# runner's absolute checkout dir (e.g. /home/runner/work/...).
68+
# Do NOT swap in --base-url here — that would rewrite root-
69+
# relative paths into http(s):// URIs and lychee would actually
70+
# hit them over the network against the canonical site.
71+
args: --no-progress --scheme http --scheme https --root-dir ${{ github.workspace }} --config lychee.toml ${{ steps.changed.outputs.all_changed_files }}
5872
fail: true
5973
env:
6074
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎blog/2026-04-20-shard-manager-incident/2026-04-20-shard-manager-incident.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ them in a day, with six lines of code total.
4949

5050
To understand how this happened, we first need to understand the new
5151
architecture.
52-
52+
<!-- truncate -->
5353
### Shard Manager Architecture
5454

5555
Cadence has two sharded services,

‎blog/2026-06-03-workflow-history-data-converter/2026-06-03-workflow-history-data-converter.mdx‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
---
2+
title: "Your Workflow History Is Storing More Than You Think"
3+
description: "Cadence writes every workflow input, activity parameter, signal, and query response into history as JSON. Learn how a custom DataConverter adds compression, encryption, or claim-check offload without changing your workflow code."
4+
keywords:
5+
- cadence data converter
6+
- workflow history payload
7+
- payload compression
8+
- payload encryption
9+
- claim check pattern
10+
- workflow history size
11+
- PII history
12+
date: 2026-06-03T09:00
13+
authors: kevinb
14+
tags:
15+
- announcement
16+
- deep-dive
17+
---
18+
19+
import PatternCards from './PatternCards';
20+
import PayloadFlow from './PayloadFlow';
21+
22+
23+
24+
That customer order you passed into your fulfillment activity (the one with the email address, the shipping details, and the internal pricing fields) is sitting in your workflow history as plaintext JSON right now. Anyone with read access to Cadence history can see it. So can anyone with access to your history storage backend.
25+
![Data pipeline: workflow history exposed vs. protected with DataConverter](./data-pipeline-exposed-vs-protected.png)
26+
This is not a bug. It is how Cadence works by design, and it is the right default for most workloads. But three problems follow from it in production, and most teams hit at least one of them before they know the solution exists.
27+
28+
<!-- truncate -->
29+
30+
## Are your workflows affected?
31+
32+
<PatternCards />
33+
34+
If none of those apply, the default JSON converter is probably fine.
35+
36+
## See it in action
37+
38+
Select a pattern and send a payload through to see what lands in history.
39+
40+
<PayloadFlow />
41+
42+
Ready to implement? The **[DataConverter Guide](/docs/concepts/data-converter)** has wiring examples, interface signatures, and production considerations for all three patterns.
43+
44+
## What DataConverter does not see
45+
46+
The converter intercepts payloads crossing the history boundary. It does not touch:
47+
48+
- **Search attributes:** indexed and queryable, but outside the converter path
49+
- **Memo:** uses the default JSON converter unless you wrap that path separately
50+
- **Application logs and metrics:** separate disclosure surfaces entirely
51+
52+
Encrypting your payloads does not encrypt these. If any of them carry sensitive data, that is a separate data-governance problem. Week 3 of this series maps out the full picture.
53+
54+
## What's next in this series
55+
56+
**Week 2: Bypass the 2 MB Limit Without Shrinking Your Workflow**
57+
58+
A deep dive into the claim-check pattern: how to keep only a reference in history, why blob keys must be deterministic (SHA-256, not UUID), and how to run the Go and Java samples against a live cluster with local-FS, S3, GCS, or MinIO as the backing store.
59+
60+
**Week 3: Encrypt Cadence History Payloads (And Know What You Didn't Encrypt)**
61+
62+
Key management, the `CADENCE_ENCRYPTION_KEY` environment variable, and a practical map of what AES-256-GCM encryption protects in Cadence versus what it leaves exposed. If you handle regulated data, this is the post to bookmark.
63+
64+
## Get started
65+
66+
- [DataConverter Guide](/docs/concepts/data-converter)
67+
- [Go samples](https://github.com/cadence-workflow/cadence-samples/tree/master/new_samples/data)
68+
- [Java samples](https://github.com/cadence-workflow/cadence-java-samples)
69+
- [Community and support](/community/contact-us)

‎blog/2026-06-03-workflow-history-data-converter/PatternCards/index.tsx‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
import React, { useState } from "react";
2+
3+
const PATTERNS = [
4+
{
5+
id: "compression",
6+
label: "Compression",
7+
icon: "🗜️",
8+
color: "#2563eb",
9+
colorRgb: "37, 99, 235",
10+
symptom: "Storage or bandwidth costs climbing on repetitive JSON",
11+
what: "gzip after JSON encode — typically 60–80% smaller. No workflow code changes. The easiest pattern to start with.",
12+
tradeoff: "Reduces and delays the 2 MB cap, but does not remove it.",
13+
docHref: "/docs/concepts/data-converter#compression",
14+
},
15+
{
16+
id: "claimcheck",
17+
label: "Claim-check",
18+
icon: "🔗",
19+
color: "#16a34a",
20+
colorRgb: "22, 163, 74",
21+
symptom: "Payloads rejected at the ~2 MB limit with no useful error",
22+
what: "Offloads payloads above a threshold to an external blob store (S3, GCS, MinIO, local FS). Only a tiny reference travels through Cadence history.",
23+
tradeoff: "The only pattern that fully removes the size constraint.",
24+
docHref: "/docs/concepts/data-converter#claim-check",
25+
},
26+
{
27+
id: "encryption",
28+
label: "Encryption",
29+
icon: "🔐",
30+
color: "#dc2626",
31+
colorRgb: "220, 38, 38",
32+
symptom: "PII, PHI, or credentials visible in workflow history or the Cadence UI",
33+
what: "AES-256-GCM wraps every payload before it reaches history. Without your key, operators see nothing readable.",
34+
tradeoff: "Covers history payloads only — not search attributes, memo, or logs.",
35+
docHref: "/docs/concepts/data-converter#encryption",
36+
},
37+
];
38+
39+
export default function PatternCards() {
40+
const [openId, setOpenId] = useState<string | null>(null);
41+
42+
return (
43+
<div style={{ display: "flex", flexDirection: "column", gap: 12, marginBottom: 24 }}>
44+
{PATTERNS.map((p) => {
45+
const isOpen = openId === p.id;
46+
return (
47+
<div
48+
key={p.id}
49+
role="button"
50+
tabIndex={0}
51+
onClick={() => setOpenId(isOpen ? null : p.id)}
52+
onKeyDown={(e) => {
53+
if (e.key === "Enter" || e.key === " ") {
54+
e.preventDefault();
55+
setOpenId(isOpen ? null : p.id);
56+
}
57+
}}
58+
aria-expanded={isOpen}
59+
style={{
60+
border: `2px solid ${isOpen ? p.color : "var(--ifm-color-emphasis-300)"}`,
61+
borderRadius: 10,
62+
padding: "14px 18px",
63+
cursor: "pointer",
64+
background: isOpen
65+
? `rgba(${p.colorRgb}, 0.08)`
66+
: "var(--ifm-card-background-color, var(--ifm-background-surface-color))",
67+
transition: "all 0.2s",
68+
userSelect: "none",
69+
}}
70+
>
71+
{/* Header row */}
72+
<div style={{ display: "flex", alignItems: "center", gap: 10 }}>
73+
<span style={{ fontSize: 22 }}>{p.icon}</span>
74+
<a
75+
href={p.docHref}
76+
onClick={(e) => e.stopPropagation()}
77+
style={{
78+
fontWeight: 700,
79+
fontSize: 15,
80+
color: isOpen ? p.color : "var(--ifm-color-content)",
81+
transition: "color 0.2s",
82+
textDecoration: "none",
83+
}}
84+
onMouseEnter={(e) => (e.currentTarget.style.textDecoration = "underline")}
85+
onMouseLeave={(e) => (e.currentTarget.style.textDecoration = "none")}
86+
>
87+
{p.label}
88+
</a>
89+
<span
90+
style={{
91+
marginLeft: "auto",
92+
fontSize: 13,
93+
color: "var(--ifm-color-content)",
94+
flex: 1,
95+
textAlign: "right",
96+
paddingLeft: 12,
97+
}}
98+
>
99+
{p.symptom}
100+
</span>
101+
<span style={{ marginLeft: 10, color: "var(--ifm-color-content-secondary)", fontSize: 14 }}>
102+
{isOpen ? "▲" : "▼"}
103+
</span>
104+
</div>
105+
106+
{/* Expanded content */}
107+
{isOpen && (
108+
<div
109+
style={{
110+
marginTop: 12,
111+
paddingTop: 12,
112+
borderTop: `1px solid rgba(${p.colorRgb}, 0.25)`,
113+
}}
114+
>
115+
<p
116+
style={{
117+
margin: "0 0 8px",
118+
fontSize: 14,
119+
color: "var(--ifm-color-content)",
120+
lineHeight: 1.6,
121+
}}
122+
>
123+
{p.what}
124+
</p>
125+
<p
126+
style={{
127+
margin: "0 0 12px",
128+
fontSize: 13,
129+
color: "var(--ifm-color-content-secondary)",
130+
fontStyle: "italic",
131+
lineHeight: 1.5,
132+
}}
133+
>
134+
{p.tradeoff}
135+
</p>
136+
<a
137+
href={p.docHref}
138+
onClick={(e) => e.stopPropagation()}
139+
style={{
140+
display: "inline-block",
141+
padding: "6px 14px",
142+
borderRadius: 6,
143+
background: p.color,
144+
color: "white",
145+
fontWeight: 600,
146+
fontSize: 13,
147+
textDecoration: "none",
148+
}}
149+
>
150+
Read the docs →
151+
</a>
152+
</div>
153+
)}
154+
</div>
155+
);
156+
})}
157+
</div>
158+
);
159+
}

0 commit comments

Comments
 (0)