Skip to content

Commit 35d52ce

Browse files
docs: bound stable diffusion rediffuse source labels (#291)
1 parent 19c9062 commit 35d52ce

3 files changed

Lines changed: 57 additions & 1 deletion

File tree

ROADMAP.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,29 @@
22

33
> Last updated: 2026-05-25
44
5+
## 2026-05-25 Stable Diffusion ReDiffuse source-label 边界审计
6+
7+
最新决策:徐驰转交的 Stable Diffusion ReDiffuse `5000` 行包继续保留为
8+
Research 侧候选证据,但不能升级为第二资产,也不能释放 GPU/下载去补同一路线。
9+
原因不是分数包不可用:现有 probe 仍能从 `result.csv` 重放
10+
`AUC = 0.71031888``ASR = 0.6846``TPR@1%FPR = 0.0736`
11+
`TPR@0.1%FPR = 0.0100`。真正的边界问题是标签语义:
12+
`source` 列与标签完全重合,`LAION-5B member subset` 覆盖全部 `2500`
13+
member 行,`COCO2017-val non-member subset` 覆盖全部 `2500` nonmember 行,
14+
仅用 `source` 就得到 `AUC = 1.000000`
15+
16+
CPU-only 机制审计还确认:`caption` 去重后分组 AUC 仍为 `0.707006`
17+
`271` 个重复 caption 组没有混合标签,文件名没有重复。这说明现有分数有非平凡
18+
重放价值,但该包更准确的定位是 Stable Diffusion 跨来源隐私压力测试,而不是严格的
19+
同分布 per-sample membership portability 证据。
20+
21+
因此不请求 `coco_data`,不下载 Stable Diffusion v1.4 权重,不重跑 `2500 / 2500`
22+
pipeline,不围绕该包新增 CLI/validator/长文档。当前 slots:
23+
`active_gpu_question = none``next_gpu_candidate = none`
24+
`CPU sidecar = none selected after Stable Diffusion ReDiffuse source-label boundary audit`
25+
See
26+
[docs/evidence/stable-diffusion-rediffuse-collaborator-artifact-20260517.md](docs/evidence/stable-diffusion-rediffuse-collaborator-artifact-20260517.md)
27+
528
## 2026-05-25 Feature-Packet 消费边界修正
629

730
最新决策:不把 Tracing the Roots 直接升级为现有 Platform/Runtime

docs/evidence/reproduction-status.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Smoke tests and dry runs are engineering validation, not benchmark claims.
7070
| Black-box `H2 response-strength` | candidate-only | Positive-but-bounded DDPM/CIFAR10 candidate: frozen cutoff-0.50 lowpass follow-up passed, and raw H2 recovered strict-tail signal on the fresh packet. SD/CelebA text-to-image transfer is blocked by protocol mismatch. The frozen SD/CelebA image-to-image micro-packet is runnable, but H2 logistic does not beat the same-cache simple distance comparator, so H2 is not promoted beyond candidate-only. A separate simple-distance line now has bounded single-asset evidence: first 10/10 packet `AUC = 0.92`, non-overlapping 10/10 packet `AUC = 0.99` with 9/10 TP at 0 FP, and non-overlapping 25/25 admission packet `AUC = 0.8768`, `ASR = 0.84`, 11/25 TP at 0 FP. This is not a conditional-diffusion generalization or a `recon` product replacement. See [black-box-response-strength-preflight.md](black-box-response-strength-preflight.md), [h2-lowpass-followup-contract.md](h2-lowpass-followup-contract.md), [h2-cross-asset-contract-preflight.md](h2-cross-asset-contract-preflight.md), [h2-image-to-image-contract.md](h2-image-to-image-contract.md), [h2-img2img-micro-result.md](h2-img2img-micro-result.md), [h2-img2img-simple-distance-review.md](h2-img2img-simple-distance-review.md), [h2-img2img-simple-distance-stability-result.md](h2-img2img-simple-distance-stability-result.md), and [h2-img2img-simple-distance-admission-result.md](h2-img2img-simple-distance-admission-result.md). |
7171
| Black-box mid-frequency same-noise residual | `candidate-only` | Distinct paper-backed observable gap: unlike H2/H3 response-cache frequency filters, this line requires `x_t`, `tilde_x_t`, timestep, noise provenance, and residual scores at the same noise level. The frozen `64/64` sign-check on the collaborator 750k checkpoint produced `AUC = 0.733398`, `ASR = 0.710938`, and finite `4/64` zero-FP recovery. The seed-only repeat retained signal with `AUC = 0.719238`, `ASR = 0.6875`, and finite `3/64` zero-FP recovery. A CPU comparator audit shows low-frequency and full-band residual comparators are at least as strong as the frozen mid-band score on AUC, so the line is candidate-stable-but-bounded but not a proven mid-frequency-specific mechanism. Same-contract GPU expansion is closed. See [midfreq-residual-comparator-audit-20260512.md](midfreq-residual-comparator-audit-20260512.md), [midfreq-residual-stability-result-20260512.md](midfreq-residual-stability-result-20260512.md), [midfreq-residual-stability-decision-20260512.md](midfreq-residual-stability-decision-20260512.md), [midfreq-residual-signcheck-20260512.md](midfreq-residual-signcheck-20260512.md), [midfreq-same-noise-residual-preflight-20260512.md](midfreq-same-noise-residual-preflight-20260512.md), [midfreq-residual-scorer-contract-20260512.md](midfreq-residual-scorer-contract-20260512.md), [midfreq-residual-collector-contract-20260512.md](midfreq-residual-collector-contract-20260512.md), [midfreq-residual-tiny-runner-contract-20260512.md](midfreq-residual-tiny-runner-contract-20260512.md), and [midfreq-residual-real-asset-preflight-20260512.md](midfreq-residual-real-asset-preflight-20260512.md). |
7272
| Gray-box `PIA` | `evidence-ready` | Strongest admitted local DDPM/CIFAR10 gray-box line. PIA baseline exposes `epsilon-trajectory consistency`; stochastic dropout is a provisional defended comparator that weakens but does not eliminate the signal. The review is bounded to repeated-query adaptive checks with `adaptive repeats=3`; low-FPR values are finite empirical strict-tail points, not calibrated sub-percent FPR. Paper-aligned release provenance remains blocked. See [pia-stochastic-dropout-truth-hardening-review.md](pia-stochastic-dropout-truth-hardening-review.md). |
73-
| Gray-box `ReDiffuse` | `hold-split-manifest-only` | Candidate baseline-alignment line. The collaborator 750k bundle and checkpoint are runnable, a 64/64 direct-distance compatibility packet exists, and the existing PIA 800k checkpoint is runtime-probe compatible, but prior exact replay showed only modest AUC with weak strict-tail evidence and was not admitted. The official OpenReview supplement now improves provenance by providing exact DDPM train/eval split index manifests for CIFAR10/CIFAR100/STL10/Tiny-IN, but it does not release target checkpoints, generated response/feature caches, score packets, ROC CSVs, or metric artifacts. Do not train DDPM/DiT/Stable Diffusion targets or rerun same-family attack scripts by default. See [rediffuse-openreview-split-manifest-audit-20260515.md](rediffuse-openreview-split-manifest-audit-20260515.md), [rediffuse-collaborator-integration-report.md](rediffuse-collaborator-integration-report.md), [rediffuse-800k-runtime-probe.md](rediffuse-800k-runtime-probe.md), [rediffuse-resnet-parity-packet.md](rediffuse-resnet-parity-packet.md), [rediffuse-direct-distance-boundary-review.md](rediffuse-direct-distance-boundary-review.md), [rediffuse-checkpoint-portability-gate.md](rediffuse-checkpoint-portability-gate.md), [rediffuse-resnet-contract-scout.md](rediffuse-resnet-contract-scout.md), [rediffuse-exact-replay-preflight.md](rediffuse-exact-replay-preflight.md), and [rediffuse-exact-replay-packet.md](rediffuse-exact-replay-packet.md). |
73+
| Gray-box `ReDiffuse` | `hold-split-manifest-only` | Candidate baseline-alignment line. The collaborator 750k bundle and checkpoint are runnable, a 64/64 direct-distance compatibility packet exists, and the existing PIA 800k checkpoint is runtime-probe compatible, but prior exact replay showed only modest AUC with weak strict-tail evidence and was not admitted. The official OpenReview supplement now improves provenance by providing exact DDPM train/eval split index manifests for CIFAR10/CIFAR100/STL10/Tiny-IN, but it does not release target checkpoints, generated response/feature caches, score packets, ROC CSVs, or metric artifacts. The collaborator Stable Diffusion ReDiffuse `5000`-row packet remains replayable (`AUC = 0.71031888`), but its member/nonmember labels are perfectly aligned with `LAION-5B member subset` versus `COCO2017-val non-member subset`, so it is a cross-source stress-test candidate rather than a same-distribution second asset. Do not train DDPM/DiT/Stable Diffusion targets, request `coco_data`, download Stable Diffusion weights, or rerun same-family attack scripts by default. See [stable-diffusion-rediffuse-collaborator-artifact-20260517.md](stable-diffusion-rediffuse-collaborator-artifact-20260517.md), [rediffuse-openreview-split-manifest-audit-20260515.md](rediffuse-openreview-split-manifest-audit-20260515.md), [rediffuse-collaborator-integration-report.md](rediffuse-collaborator-integration-report.md), [rediffuse-800k-runtime-probe.md](rediffuse-800k-runtime-probe.md), [rediffuse-resnet-parity-packet.md](rediffuse-resnet-parity-packet.md), [rediffuse-direct-distance-boundary-review.md](rediffuse-direct-distance-boundary-review.md), [rediffuse-checkpoint-portability-gate.md](rediffuse-checkpoint-portability-gate.md), [rediffuse-resnet-contract-scout.md](rediffuse-resnet-contract-scout.md), [rediffuse-exact-replay-preflight.md](rediffuse-exact-replay-preflight.md), and [rediffuse-exact-replay-packet.md](rediffuse-exact-replay-packet.md). |
7474
| Gray-box `Tracing the Roots` | `positive-provenance-limited` | OpenReview supplementary material exposes a small CIFAR10 diffusion-trajectory feature packet with fixed `1000/1000` train and `1000/1000` eval member/external tensors plus replay code. The bounded local replay gives `AUC = 0.815826`, `accuracy = 0.737500`, `TPR@1%FPR = 0.134000`, and `TPR@0.1%FPR = 0.038000`. A machine-readable candidate-only card now records the feature tensor hashes, live OpenReview/arXiv recheck, blocked claims, and reopen conditions. It is not admitted because the supplement lacks raw target checkpoint identity, raw sample IDs, and image query-response artifacts, and arXiv `2411.07449v3` source does not add a regeneration manifest. Do not expand timestep, feature-family, seed, classifier, optimizer, or regularization matrices without raw provenance/regeneration assets or a feature-packet consumer-boundary decision. See [tracing-roots-feature-packet-mia-20260515.md](tracing-roots-feature-packet-mia-20260515.md) and [../product-bridge/tracing-roots-candidate-evidence-card.md](../product-bridge/tracing-roots-candidate-evidence-card.md). |
7575
| Dataset-inference `CDI` official release | `hold-semantic-shift` | The official `sprintml/copyrighted_data_identification` repo is code-public and scientifically relevant because it explicitly pivots from weak pointwise MIAs to dataset inference. It is not a current automatic execution lane: the public tree has no ready small score packet, configs target local Google Drive model checkpoints plus ImageNet/COCO assets, default experiments are large (`25k`-style), and promotion would require a consumer-boundary decision separating dataset-level evidence from per-sample membership rows. Do not download CDI model folders, ImageNet, COCO, text embeddings, or submodule payloads by default. See [cdi-official-artifact-gate-20260515.md](cdi-official-artifact-gate-20260515.md). |
7676
| Gray-box `tri-score` | candidate-only | CDI/TMIA-DM/PIA tri-score aggregation survives CPU truth-hardening as internal Research evidence, with all three frozen packets beating admitted PIA on AUC and both low-FPR fields. It remains internal-only because the packet contract forbids headline/external use and ASR is not stable enough for the support claim. See [gray-box-triscore-consolidation-review.md](gray-box-triscore-consolidation-review.md) and [gray-box-triscore-truth-hardening-review.md](gray-box-triscore-truth-hardening-review.md). |

docs/evidence/stable-diffusion-rediffuse-collaborator-artifact-20260517.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,36 @@ split, recomputed `AUC = 0.710319`, `ASR = 0.6846`,
108108
confirms detector JSON, merged score NPZ, validation JSON, and source files are
109109
present and internally consistent.
110110

111+
## 2026-05-25 Source-Label Boundary Audit
112+
113+
A CPU-only metadata audit checked whether the imported `result.csv` supports a
114+
strict same-distribution membership claim or whether the member label is
115+
confounded with the source domain. This did not rerun Stable Diffusion, download
116+
COCO, request new LAION payloads, or create a new tool. It inspected the
117+
existing `5,000` result rows only.
118+
119+
Observed boundary facts:
120+
121+
| Check | Result |
122+
| --- | ---: |
123+
| Row-level ReDiffuse score AUC | `0.71031888` |
124+
| Source-only AUC from the `source` column | `1.000000` |
125+
| Member source rows | `2,500 / 2,500` from `LAION-5B member subset` |
126+
| Nonmember source rows | `2,500 / 2,500` from `COCO2017-val non-member subset` |
127+
| Caption-unique groups | `4,637` |
128+
| Duplicate-caption groups | `271` |
129+
| Mixed-label caption groups | `0` |
130+
| Caption-deduplicated group AUC | `0.707006` |
131+
| File-name duplicate groups | `0` |
132+
133+
This closes the main interpretation gap. The packet is internally replayable
134+
and nontrivial, but it is not a clean same-distribution per-sample membership
135+
asset because the member/nonmember label is perfectly aligned with
136+
`LAION-5B member subset` versus `COCO2017-val non-member subset`. The
137+
ReDiffuse score remains useful as candidate evidence for a Stable Diffusion
138+
cross-source privacy stress test, not as a second asset for strict
139+
member/nonmember portability or Platform/Runtime admission.
140+
111141
## Usefulness
112142

113143
- This is a real imported Stable Diffusion candidate packet, not another empty
@@ -140,6 +170,9 @@ This does not satisfy the current Lane A reopen gate for a public replay asset:
140170
- it is a collaborator local transfer, not a public immutable packet;
141171
- the member side is a LAION-like repeatable subset rather than the exact paper
142172
LAION-5B member split;
173+
- the member and nonmember rows are also perfectly separated by source
174+
(`LAION-5B member subset` versus `COCO2017-val non-member subset`), so the
175+
packet cannot support a strict same-distribution membership claim;
143176
- the current boundary is local-model-query black-box, not strict external
144177
API-only black-box; and
145178
- the missing COCO payload is not needed for artifact audit, but it also means

0 commit comments

Comments
 (0)