Commit 6360e20
fix(lib)!: preserve symlinks in TerraformAsset walkers (#321)
### Description
Closes #320.
`TerraformAsset` walked source trees with `fs.statSync`, which follows
symlinks. On symlinked trees (e.g. pnpm's `node_modules`) this meant:
- **Archive bloat**: a target reachable through N symlink paths was
copied N times into the zip (15 MB → 34 MB on a real Lambda bundle for
byte-identical logical content).
- **No symlink fidelity**: archives contained zero symlink entries —
links became full copies.
- **Hard synth crash**: circular symlinks (legitimate in pnpm trees) hit
`ELOOP`. Notably this crashed for **every** asset type, not just
`ARCHIVE`, because `hashPath` runs in the `TerraformAsset` constructor.
This is a fork regression, not inherited from cdktf: upstream archives
with `archiver`, which walks with `lstat` and emits real symlink entries
(cycles are structurally impossible there). The regression entered when
`archiver` was replaced by yazl in #95 — yazl has no symlink awareness
at all (`addFile` stats-and-follows) — and survived the yazl → fflate
rewrite in #148, which kept the hand-rolled `statSync` walk.
### The fix
All three walkers in `packages/cdktn/src/private/fs.ts` now use
`lstatSync` and treat symlinks first-class, restoring `archiver`/cdktf
parity:
- **`archiveSync`** emits real symlink entries: the `readlink` target as
STORED entry data with `S_IFLNK | perms` in the unix external attributes
and version-made-by host = Unix — exactly what `archiver` produces and
what Info-ZIP `unzip`, Go `archive/zip`, and the AWS Lambda runtime
(verified live on `nodejs22.x` in #320) recreate as symlinks. Never
recursing through links makes cycles unreachable by construction. fflate
0.8.2 supports this natively via per-file `[data, { os, attrs, level }]`
tuples (attrs must be caller-pre-shifted, `(mode << 16) >>> 0`).
- **`hashPath`** hashes a symlink by its target path instead of
following it — retargeting a link still changes the asset hash, but
shared targets are no longer double-counted and cycles no longer crash
the constructor.
- **`copySync`** (`AssetType.DIRECTORY`) recreates symlinks with
`fs.symlinkSync`.
Two latent zip-metadata gaps fixed along the way (both also
`archiver`/cdktf parity):
- Regular-file unix modes are now preserved (executable bits were
silently stripped — matters for Lambda binaries and `.bin` scripts).
- Entry mtimes are pinned to a fixed 1980 date, making archives
byte-reproducible across synths (fflate defaults `mtime` to
`Date.now()`, so every synth previously produced different zip bytes —
perpetual drift for anyone hashing the archive, e.g.
`filebase64sha256`). The pin uses the local-time `Date` constructor
deliberately: fflate encodes DOS dates from local getters and rejects
years < 1980, so a UTC midnight date would underflow to 1979 in
timezones west of UTC.
Prior art considered and rejected: `terraform-provider-archive` always
dereferences and has no cycle detection — it accumulated the opt-in
`exclude_symlink_directories` band-aid (v2.4.0,
hashicorp/terraform-provider-archive#183; follow-up defect fixed in
v2.4.2, #298) and still `ELOOP`s on cycles. Reverting to `archiver`
would reintroduce the `execSync` child-process bridge that #148 removed
(its API is async-only).
### Hash compatibility (updated after review — downscoped)
Review surfaced a domain collision in the first cut of the `hashPath`
change (file bytes and symlink-target bytes shared one unframed stream).
Per the requested downscope this PR now carries only the
compatibility-preserving resolution (reviewer's option 3):
- the legacy content hash is kept byte-identical for symlink-free trees
(proven by a test that recomputes the historical md5 independently);
- symlink metadata (relative path + target) is framed into a separate
digest and combined under a tagged outer hash **only when symlinks
exist** — so hashes still change only for symlink-bearing trees, which
today either bloat 2-3× or crash outright.
The canonical entry-framed scheme and its `canonicalAssetHashes` feature
flag moved to the stack: design issue #322 → implementation #323 → docs
#324.
`TerraformModuleAsset` stays in scope per review: it shares the
corrected copier (its private copier sent directory symlinks into
`copyFileSync` → `EISDIR`) with a regression test.
### Testing
- `packages/cdktn/test/archive-symlink.test.ts`: 10 tests — the issue's
repros (file/dir symlink preservation via system-`unzip` round-trip +
`lstat`, dedup of shared targets, `ELOOP` on cycles for both
`archiveSync` and `hashPath`, hash-changes-on-retarget), `copySync`
symlink recreation, executable-bit preservation, and byte-identical
archives across runs. 5 of the 6 core repros fail on `main`.
- Full `cdktn` package suite green (453 passed; the one pre-existing
`matchers.test.ts` → `toPlanSuccessfully` failure shells out to a real
`terraform plan` and fails identically without this change).
- Issue #320's literal repro script against the built lib: `zipinfo`
shows `lrwxr-xr-x ... stor` entries for `link-a`/`link-b`, one payload
copy (538-byte zip vs 3× 200 KiB before), and the cyclic tree archives
to a 120-byte zip instead of aborting synth.
### Checklist
- [x] I have updated the PR title to match [CDKTN's style
guide](https://github.com/open-constructs/cdk-terrain/blob/main/CONTRIBUTING.md#pull-requests-1)
- [x] I have run the linter on my code locally
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation if
applicable
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works if applicable
- [x] New and existing unit tests pass locally with my changes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>1 parent a899b7c commit 6360e20
4 files changed
Lines changed: 386 additions & 38 deletions
File tree
- packages/cdktn
- src
- private
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
| 7 | + | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
11 | 33 | | |
12 | 34 | | |
13 | 35 | | |
| |||
21 | 43 | | |
22 | 44 | | |
23 | 45 | | |
24 | | - | |
25 | | - | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
26 | 50 | | |
27 | | - | |
28 | | - | |
| 51 | + | |
29 | 52 | | |
30 | 53 | | |
31 | 54 | | |
| |||
51 | 74 | | |
52 | 75 | | |
53 | 76 | | |
54 | | - | |
| 77 | + | |
55 | 78 | | |
56 | 79 | | |
57 | 80 | | |
58 | 81 | | |
59 | | - | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
60 | 96 | | |
61 | 97 | | |
62 | | - | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
63 | 105 | | |
64 | 106 | | |
65 | 107 | | |
66 | 108 | | |
67 | | - | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
68 | 113 | | |
69 | 114 | | |
70 | 115 | | |
71 | 116 | | |
72 | 117 | | |
73 | 118 | | |
74 | 119 | | |
75 | | - | |
76 | | - | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
77 | 128 | | |
78 | 129 | | |
79 | 130 | | |
80 | 131 | | |
81 | | - | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
82 | 135 | | |
83 | 136 | | |
84 | | - | |
| 137 | + | |
| 138 | + | |
85 | 139 | | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
86 | 143 | | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
91 | 151 | | |
92 | 152 | | |
93 | | - | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
94 | 157 | | |
95 | 158 | | |
96 | 159 | | |
97 | 160 | | |
98 | | - | |
99 | | - | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
100 | 170 | | |
101 | 171 | | |
102 | 172 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
127 | 127 | | |
128 | 128 | | |
129 | 129 | | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
0 commit comments