Skip to content

Commit f177fb4

Browse files
avrabeclaude
andauthored
fix(fuse): add --reproducible for byte-stable output (#325) (#327)
* fix(fuse): add --reproducible for byte-stable output (#325) meld's fused output was byte-nondeterministic: identical input yielded a different sha256 each run, breaking reproducible-build / supply-chain attestation (scry/loom/witness/sigil key on the fused bytes). Root cause (found by structural diff, not the issue's HashMap guess): the ONLY nondeterministic bytes are in the `wsc.transformation.attestation` custom section — a random `attestation_id` UUID + a wall-clock `timestamp`. Everything else is already deterministic (`--no-attestation` is byte-identical across runs; the section's own `output.hash` is stable). Add a `--reproducible` flag (FuserConfig.reproducible). When set, the attestation id is derived from the output content (`entropy_from_hex(output_hash)` → `generate_uuid_from`) and the timestamp from the ecosystem-standard `SOURCE_DATE_EPOCH` (default epoch 0) via `chrono_timestamp_from`, instead of a random UUID + system clock — so identical input yields an identical artifact. Both attestation builders are covered: the default `FusionAttestationBuilder` path and the `feature = "attestation"` `wsc` path. Verified end-to-end: `--reproducible` → identical sha256 across runs; `SOURCE_DATE_EPOCH=0` → `1970-01-01T00:00:00Z`; control (no flag) still varies. Regression test `test_reproducible_attestation_is_byte_stable` pins it (and asserts the control differs, so the flag can't silently become a no-op). The new FuserConfig field is threaded through the existing explicit test literals (reproducible: false). Refs #325. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(fuse): scope #325 reproducibility test to the default build path The Coverage job (cargo test --all-features) exposed a real gap: under the optional `attestation` (wsc) feature, the emitted section carries `tool_parameters` / `metadata` as `wsc_attestation` `HashMap` fields whose serde serialization order is process-random — meld can't control it from this side (the field types live in the upstream crate). So `--reproducible` makes the shipped default build (`features = []`, `FusionAttestationBuilder`, no such maps) fully byte-stable, but not the optional wsc path. Gate `test_reproducible_attestation_is_byte_stable` to `#[cfg(not(feature = "attestation"))]` — it asserts the guarantee meld actually provides (the default build) and documents the wsc-path limitation (needs an upstream sorted/BTreeMap serialization fix). The default-path fix and its verification are unchanged. Refs #325. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent be66dc7 commit f177fb4

21 files changed

Lines changed: 219 additions & 5 deletions

meld-cli/src/main.rs

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,13 @@ enum Commands {
7676
#[arg(long)]
7777
no_attestation: bool,
7878

79+
/// Emit a byte-reproducible artifact (#325): derive the attestation
80+
/// id from the output content and take the timestamp from
81+
/// `SOURCE_DATE_EPOCH` (default epoch 0) instead of a random UUID +
82+
/// wall clock, so identical input yields an identical sha256.
83+
#[arg(long)]
84+
reproducible: bool,
85+
7986
/// Disable the `component-provenance` custom section (#192).
8087
/// The section maps each fused-module function index back to
8188
/// its originating component + function index; downstream
@@ -163,6 +170,7 @@ fn main() -> Result<()> {
163170
address_rebase,
164171
stats,
165172
no_attestation,
173+
reproducible,
166174
no_component_provenance,
167175
dwarf,
168176
preserve_names,
@@ -178,6 +186,7 @@ fn main() -> Result<()> {
178186
address_rebase,
179187
stats,
180188
no_attestation,
189+
reproducible,
181190
no_component_provenance,
182191
dwarf,
183192
preserve_names,
@@ -235,6 +244,7 @@ fn fuse_command(
235244
address_rebase: bool,
236245
show_stats: bool,
237246
no_attestation: bool,
247+
reproducible: bool,
238248
no_component_provenance: bool,
239249
dwarf: String,
240250
preserve_names: bool,
@@ -333,6 +343,7 @@ fn fuse_command(
333343
let config = FuserConfig {
334344
memory_strategy,
335345
attestation: !no_attestation,
346+
reproducible,
336347
component_provenance: !no_component_provenance,
337348
address_rebasing: address_rebase,
338349
preserve_names,

meld-core/benches/fusion_benchmarks.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ fn bench_config() -> FuserConfig {
7272
FuserConfig {
7373
memory_strategy: MemoryStrategy::MultiMemory,
7474
attestation: false,
75+
reproducible: false,
7576
component_provenance: false,
7677
address_rebasing: false,
7778
preserve_names: false,

meld-core/src/attestation.rs

Lines changed: 49 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,7 @@ pub struct FusionAttestationBuilder {
123123
tool_version: String,
124124
tool_hash: Option<String>,
125125
memory_strategy: String,
126+
reproducible: bool,
126127
}
127128

128129
impl FusionAttestationBuilder {
@@ -134,9 +135,18 @@ impl FusionAttestationBuilder {
134135
tool_version: tool_version.into(),
135136
tool_hash: None,
136137
memory_strategy: "shared".to_string(),
138+
reproducible: false,
137139
}
138140
}
139141

142+
/// Emit a byte-reproducible attestation (#325): derive the id from the
143+
/// output content and take the timestamp from `SOURCE_DATE_EPOCH`
144+
/// (default epoch 0) instead of a random UUID + wall clock.
145+
pub fn reproducible(mut self, reproducible: bool) -> Self {
146+
self.reproducible = reproducible;
147+
self
148+
}
149+
140150
/// Set the tool hash for reproducibility
141151
pub fn tool_hash(mut self, hash: impl Into<String>) -> Self {
142152
self.tool_hash = Some(hash.into());
@@ -211,8 +221,16 @@ impl FusionAttestationBuilder {
211221
/// Build the attestation
212222
pub fn build(self, output_bytes: &[u8], stats: &FusionStats) -> FusionAttestation {
213223
let output_hash = compute_sha256(output_bytes);
214-
let timestamp = chrono_timestamp();
215-
let attestation_id = generate_uuid();
224+
// #325: reproducible mode derives the id from the output content and
225+
// the timestamp from SOURCE_DATE_EPOCH so the artifact is byte-stable.
226+
let (timestamp, attestation_id) = if self.reproducible {
227+
(
228+
chrono_timestamp_from(source_date_epoch()),
229+
generate_uuid_from(entropy_from_hex(&output_hash)),
230+
)
231+
} else {
232+
(chrono_timestamp(), generate_uuid())
233+
};
216234

217235
let size_reduction = if stats.input_size > 0 {
218236
let diff = stats.input_size as i128 - stats.output_size as i128;
@@ -328,6 +346,35 @@ pub(crate) fn generate_uuid_from(entropy: u128) -> String {
328346
)
329347
}
330348

349+
/// Seconds-since-epoch for a reproducible attestation timestamp (#325).
350+
///
351+
/// Honors the ecosystem-standard `SOURCE_DATE_EPOCH` environment variable
352+
/// (a decimal seconds-since-Unix-epoch string); when it is unset or
353+
/// unparseable, falls back to `0` (`1970-01-01T00:00:00Z`) so a reproducible
354+
/// build is byte-stable regardless of the wall clock. Only consulted on the
355+
/// `--reproducible` path; normal builds keep the real `chrono_timestamp()`.
356+
pub(crate) fn source_date_epoch() -> u64 {
357+
std::env::var("SOURCE_DATE_EPOCH")
358+
.ok()
359+
.and_then(|s| s.trim().parse::<u64>().ok())
360+
.unwrap_or(0)
361+
}
362+
363+
/// Derive deterministic UUID entropy from a content hash (#325).
364+
///
365+
/// Takes the fused output's SHA-256 hex string and folds its first 32 hex
366+
/// digits into a `u128`, so the attestation id is a stable function of the
367+
/// artifact content rather than the wall clock. Short/odd input degrades
368+
/// gracefully to whatever parses (never panics).
369+
pub(crate) fn entropy_from_hex(hash_hex: &str) -> u128 {
370+
let head: String = hash_hex
371+
.chars()
372+
.filter(|c| c.is_ascii_hexdigit())
373+
.take(32)
374+
.collect();
375+
u128::from_str_radix(&head, 16).unwrap_or(0)
376+
}
377+
331378
/// Get current timestamp in ISO 8601 format using the system clock.
332379
///
333380
/// Thin wrapper over [`chrono_timestamp_from`] sourcing seconds-since-epoch

meld-core/src/lib.rs

Lines changed: 110 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,12 @@ pub struct FuserConfig {
8484
/// Whether to generate attestation data
8585
pub attestation: bool,
8686

87+
/// Whether to emit a byte-reproducible attestation (#325): derive the
88+
/// attestation id from the output content and source the timestamp from
89+
/// `SOURCE_DATE_EPOCH` (default epoch 0) instead of a random UUID + the
90+
/// wall clock, so identical input yields an identical fused artifact.
91+
pub reproducible: bool,
92+
8793
/// Whether to emit the `component-provenance` custom section
8894
/// (issue #192). When enabled (the default), every defined
8995
/// function in the fused module gets a back-pointer entry
@@ -144,6 +150,7 @@ impl Default for FuserConfig {
144150
Self {
145151
memory_strategy: MemoryStrategy::Auto,
146152
attestation: true,
153+
reproducible: false,
147154
component_provenance: true,
148155
address_rebasing: false,
149156
preserve_names: false,
@@ -1832,7 +1839,8 @@ impl Fuser {
18321839
stats: &FusionStats,
18331840
) -> attestation::FusionAttestation {
18341841
let mut builder = FusionAttestationBuilder::new("meld", env!("CARGO_PKG_VERSION"))
1835-
.memory_strategy(self.memory_strategy_label());
1842+
.memory_strategy(self.memory_strategy_label())
1843+
.reproducible(self.config.reproducible);
18361844

18371845
for (index, component) in self.components.iter().enumerate() {
18381846
let name = component
@@ -1863,8 +1871,20 @@ impl Fuser {
18631871
};
18641872

18651873
let output_hash = attestation::compute_sha256(output_bytes);
1866-
let timestamp = attestation::chrono_timestamp();
1867-
let attestation_id = attestation::generate_uuid();
1874+
// #325: in reproducible mode, derive the id from the output content and
1875+
// the timestamp from SOURCE_DATE_EPOCH so the fused artifact is
1876+
// byte-stable across runs; otherwise keep the wall-clock/random id.
1877+
let (timestamp, attestation_id) = if self.config.reproducible {
1878+
(
1879+
attestation::chrono_timestamp_from(attestation::source_date_epoch()),
1880+
attestation::generate_uuid_from(attestation::entropy_from_hex(&output_hash)),
1881+
)
1882+
} else {
1883+
(
1884+
attestation::chrono_timestamp(),
1885+
attestation::generate_uuid(),
1886+
)
1887+
};
18681888

18691889
let mut inputs = Vec::new();
18701890
for (index, component) in self.components.iter().enumerate() {
@@ -2822,6 +2842,93 @@ mod tests {
28222842
}
28232843
}
28242844

2845+
/// #325: with `reproducible: true`, the fused artifact is byte-identical
2846+
/// across runs — the attestation id is derived from the output content and
2847+
/// the timestamp from `SOURCE_DATE_EPOCH` (default epoch 0), so the
2848+
/// random-UUID + wall-clock non-determinism is removed. The control
2849+
/// (reproducible off) must differ, proving the flag is what fixes it.
2850+
///
2851+
/// Scoped to the default build (the shipped configuration). Under the
2852+
/// optional `attestation` (wsc) feature the emitted section additionally
2853+
/// carries `tool_parameters` / `metadata` as `wsc_attestation` `HashMap`
2854+
/// fields, whose serialization order meld cannot control from this side —
2855+
/// full byte-reproducibility on that path needs an upstream fix (sorted /
2856+
/// BTreeMap serialization). The default `FusionAttestationBuilder` path has
2857+
/// no such maps and is fully reproducible.
2858+
#[cfg(not(feature = "attestation"))]
2859+
#[test]
2860+
fn test_reproducible_attestation_is_byte_stable() {
2861+
use wasm_encoder::{
2862+
CodeSection, Component, ExportKind, ExportSection, Function, FunctionSection,
2863+
Instruction, MemorySection, MemoryType, Module as EncoderModule, ModuleSection,
2864+
TypeSection, ValType,
2865+
};
2866+
fn build_minimal_component() -> Vec<u8> {
2867+
let mut types = TypeSection::new();
2868+
types.ty().function([], [ValType::I32]);
2869+
let mut functions = FunctionSection::new();
2870+
functions.function(0);
2871+
let mut memory = MemorySection::new();
2872+
memory.memory(MemoryType {
2873+
minimum: 1,
2874+
maximum: None,
2875+
memory64: false,
2876+
shared: false,
2877+
page_size_log2: None,
2878+
});
2879+
let mut exports = ExportSection::new();
2880+
exports.export("run", ExportKind::Func, 0);
2881+
exports.export("memory", ExportKind::Memory, 0);
2882+
let mut code = CodeSection::new();
2883+
let mut func = Function::new([]);
2884+
func.instruction(&Instruction::I32Const(42));
2885+
func.instruction(&Instruction::End);
2886+
code.function(&func);
2887+
let mut module = EncoderModule::new();
2888+
module
2889+
.section(&types)
2890+
.section(&functions)
2891+
.section(&memory)
2892+
.section(&exports)
2893+
.section(&code);
2894+
let mut component = Component::new();
2895+
component.section(&ModuleSection(&module));
2896+
component.finish()
2897+
}
2898+
let component_bytes = build_minimal_component();
2899+
2900+
let fuse_once = |reproducible: bool| -> Vec<u8> {
2901+
let config = FuserConfig {
2902+
attestation: true,
2903+
reproducible,
2904+
..FuserConfig::default()
2905+
};
2906+
let mut fuser = Fuser::new(config);
2907+
fuser
2908+
.add_component(&component_bytes)
2909+
.expect("add_component failed");
2910+
fuser.fuse().expect("fuse failed")
2911+
};
2912+
2913+
// Reproducible: identical bytes across independent runs.
2914+
let a = fuse_once(true);
2915+
let b = fuse_once(true);
2916+
assert_eq!(
2917+
a, b,
2918+
"#325: reproducible fusion must be byte-identical across runs"
2919+
);
2920+
2921+
// Control: without reproducible, the attestation's random UUID makes
2922+
// the output differ — otherwise the flag would be a no-op.
2923+
let c = fuse_once(false);
2924+
let d = fuse_once(false);
2925+
assert_ne!(
2926+
c, d,
2927+
"#325: non-reproducible fusion is expected to differ (random attestation id); \
2928+
if this ever holds, the reproducible flag is testing nothing"
2929+
);
2930+
}
2931+
28252932
/// SR-20 / SC-8: Fail-fast when a core module (not a component) is passed
28262933
/// to `add_component()`.
28272934
///

meld-core/tests/adapter_dwarf_e2e.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ fn fuse_remap() -> Vec<u8> {
7676
memory_strategy: MemoryStrategy::MultiMemory,
7777
dwarf_handling: DwarfHandling::Remap,
7878
attestation: false,
79+
reproducible: false,
7980
..Default::default()
8081
});
8182
fuser.add_component_named(&a, Some("a")).unwrap();

meld-core/tests/adapter_safety.rs

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -431,6 +431,7 @@ fn test_sr12_adapter_generation_for_string_param() {
431431
let config = FuserConfig {
432432
memory_strategy: MemoryStrategy::MultiMemory,
433433
attestation: false,
434+
reproducible: false,
434435
component_provenance: false,
435436
address_rebasing: false,
436437
preserve_names: false,
@@ -819,6 +820,7 @@ fn test_sr13_cabi_realloc_targets_correct_memory() {
819820
let config = FuserConfig {
820821
memory_strategy: MemoryStrategy::MultiMemory,
821822
attestation: false,
823+
reproducible: false,
822824
component_provenance: false,
823825
address_rebasing: false,
824826
preserve_names: false,
@@ -1235,6 +1237,7 @@ fn test_sr15_list_copy_length() {
12351237
let config = FuserConfig {
12361238
memory_strategy: MemoryStrategy::MultiMemory,
12371239
attestation: false,
1240+
reproducible: false,
12381241
component_provenance: false,
12391242
address_rebasing: false,
12401243
preserve_names: false,
@@ -1731,6 +1734,7 @@ fn test_sr16_inner_pointer_fixup_list_string() {
17311734
let config = FuserConfig {
17321735
memory_strategy: MemoryStrategy::MultiMemory,
17331736
attestation: false,
1737+
reproducible: false,
17341738
component_provenance: false,
17351739
address_rebasing: false,
17361740
preserve_names: false,
@@ -2031,6 +2035,7 @@ fn test_sr17_utf8_to_utf16_string_transcoding() {
20312035
let config = FuserConfig {
20322036
memory_strategy: MemoryStrategy::MultiMemory,
20332037
attestation: false,
2038+
reproducible: false,
20342039
component_provenance: false,
20352040
address_rebasing: false,
20362041
preserve_names: false,
@@ -2120,6 +2125,7 @@ fn test_sr17_utf8_to_utf16_supplementary_plane_transcoding() {
21202125
let config = FuserConfig {
21212126
memory_strategy: MemoryStrategy::MultiMemory,
21222127
attestation: false,
2128+
reproducible: false,
21232129
component_provenance: false,
21242130
address_rebasing: false,
21252131
preserve_names: false,
@@ -2645,6 +2651,7 @@ fn test_sr17_utf16_to_utf8_supplementary_plane_transcoding() {
26452651
let config = FuserConfig {
26462652
memory_strategy: MemoryStrategy::MultiMemory,
26472653
attestation: false,
2654+
reproducible: false,
26482655
component_provenance: false,
26492656
address_rebasing: false,
26502657
preserve_names: false,
@@ -2719,6 +2726,7 @@ fn test_sr17_utf16_to_utf8_lone_high_surrogate_replacement() {
27192726
let config = FuserConfig {
27202727
memory_strategy: MemoryStrategy::MultiMemory,
27212728
attestation: false,
2729+
reproducible: false,
27222730
component_provenance: false,
27232731
address_rebasing: false,
27242732
preserve_names: false,
@@ -2783,6 +2791,7 @@ fn test_sr17_utf16_to_utf8_midstring_lone_surrogate_replacement() {
27832791
let config = FuserConfig {
27842792
memory_strategy: MemoryStrategy::MultiMemory,
27852793
attestation: false,
2794+
reproducible: false,
27862795
component_provenance: false,
27872796
address_rebasing: false,
27882797
preserve_names: false,
@@ -2872,6 +2881,7 @@ fn test_sr17_utf16_to_utf8_malformed_surrogate_matrix() {
28722881
let config = FuserConfig {
28732882
memory_strategy: MemoryStrategy::MultiMemory,
28742883
attestation: false,
2884+
reproducible: false,
28752885
component_provenance: false,
28762886
address_rebasing: false,
28772887
preserve_names: false,
@@ -3007,6 +3017,7 @@ fn ls_p_20_test_sr17_utf8_to_utf16_malformed_matrix() {
30073017
let config = FuserConfig {
30083018
memory_strategy: MemoryStrategy::MultiMemory,
30093019
attestation: false,
3020+
reproducible: false,
30103021
component_provenance: false,
30113022
address_rebasing: false,
30123023
preserve_names: false,
@@ -3082,6 +3093,7 @@ fn test_sr17_latin1_to_utf16_transcoding() {
30823093
let config = FuserConfig {
30833094
memory_strategy: MemoryStrategy::MultiMemory,
30843095
attestation: false,
3096+
reproducible: false,
30853097
component_provenance: false,
30863098
address_rebasing: false,
30873099
preserve_names: false,
@@ -3565,6 +3577,7 @@ fn fuse_run_i32(caller: &[u8], callee: &[u8], label: &str) -> i32 {
35653577
let config = FuserConfig {
35663578
memory_strategy: MemoryStrategy::MultiMemory,
35673579
attestation: false,
3580+
reproducible: false,
35683581
component_provenance: false,
35693582
address_rebasing: false,
35703583
preserve_names: false,

0 commit comments

Comments
 (0)