docs: close ReDiffuse STL-10 bounded scout#294
Conversation
There was a problem hiding this comment.
Code Review
This pull request documents the results of the ReDiffuse STL-10 bounded scout conducted on 2026-05-25. The scout produced a random-level membership signal (AUC ≈ 0.5), leading to the decision to transition the research line to a "hold-weak" status and halt further GPU resource expansion. Documentation across the repository, including the roadmap, agent state, and reproduction status, has been updated to reflect these findings, and a new evidence file details the run artifacts and metrics. Feedback was provided regarding a track classification inconsistency where ReDiffuse is listed under Black-box in the workspace index but Gray-box elsewhere.
| | Track | Active docs | Role | | ||
| | --- | --- | --- | | ||
| | Black-box | [workspaces/black-box/README.md](../../workspaces/black-box/README.md), [plan.md](../../workspaces/black-box/plan.md), [rediffuse-stl10-split-and-microtrain-preflight-20260525.md](rediffuse-stl10-split-and-microtrain-preflight-20260525.md), [copymark-laion-mi-public-binding-gate-20260517.md](copymark-laion-mi-public-binding-gate-20260517.md), [stable-diffusion-rediffuse-collaborator-artifact-20260517.md](stable-diffusion-rediffuse-collaborator-artifact-20260517.md), [public-metadata-asset-sweep-20260515.md](public-metadata-asset-sweep-20260515.md), [copymark-official-score-artifact-gate-20260515.md](copymark-official-score-artifact-gate-20260515.md), [shake-to-leak-code-artifact-gate-20260515.md](shake-to-leak-code-artifact-gate-20260515.md), [fseclab-mia-diffusion-code-artifact-gate-20260515.md](fseclab-mia-diffusion-code-artifact-gate-20260515.md), [genai-confessions-black-box-artifact-gate-20260515.md](genai-confessions-black-box-artifact-gate-20260515.md), [clid-official-inter-output-replay-20260515.md](clid-official-inter-output-replay-20260515.md), [midst-tabddpm-ept-scout-20260515.md](midst-tabddpm-ept-scout-20260515.md), [diffusion-memorization-asset-gate-20260515.md](diffusion-memorization-asset-gate-20260515.md), [rediffuse-openreview-split-manifest-audit-20260515.md](rediffuse-openreview-split-manifest-audit-20260515.md), [beans-lora-delta-sensitivity-20260513.md](beans-lora-delta-sensitivity-20260513.md), [quantile-regression-asset-verdict-20260513.md](quantile-regression-asset-verdict-20260513.md), [miagm-asset-verdict-20260513.md](miagm-asset-verdict-20260513.md), [noise-as-probe-asset-verdict-20260513.md](noise-as-probe-asset-verdict-20260513.md), [zenodo-code-reference-audit-20260513.md](zenodo-code-reference-audit-20260513.md), [zenodo-finetuned-diffusion-asset-verdict-20260513.md](zenodo-finetuned-diffusion-asset-verdict-20260513.md), [laion-mi-url-availability-probe-20260513.md](laion-mi-url-availability-probe-20260513.md), [laion-mi-asset-verdict-20260513.md](laion-mi-asset-verdict-20260513.md), [commoncanvas-denoising-loss-20260513.md](commoncanvas-denoising-loss-20260513.md), [midst-tabddpm-shadow-distributional-scout-20260513.md](midst-tabddpm-shadow-distributional-scout-20260513.md), [midst-tabddpm-nearest-neighbor-scout-20260513.md](midst-tabddpm-nearest-neighbor-scout-20260513.md), [copymark-commoncanvas-multiseed-stability-20260513.md](copymark-commoncanvas-multiseed-stability-20260513.md), [fashion-mnist-ddpm-pia-loss-scout-20260513.md](fashion-mnist-ddpm-pia-loss-scout-20260513.md), [kohaku-danbooru-asset-decision-20260513.md](kohaku-danbooru-asset-decision-20260513.md), [tiny-known-split-gradient-prototype-alignment-20260513.md](tiny-known-split-gradient-prototype-alignment-20260513.md), [copymark-commoncanvas-response-preflight-20260512.md](copymark-commoncanvas-response-preflight-20260512.md), [copymark-commoncanvas-query-asset-20260512.md](copymark-commoncanvas-query-asset-20260512.md), [copymark-provenance-intake-20260512.md](copymark-provenance-intake-20260512.md), [external-diffusion-benchmark-provenance-scan-20260512.md](external-diffusion-benchmark-provenance-scan-20260512.md), [true-second-membership-benchmark-scope-20260512.md](true-second-membership-benchmark-scope-20260512.md), [gradient-norm-stability-gate-20260512.md](gradient-norm-stability-gate-20260512.md), [tiny-overfit-gradient-norm-scout-20260512.md](tiny-overfit-gradient-norm-scout-20260512.md), [tiny-overfit-mse-upperbound-20260512.md](tiny-overfit-mse-upperbound-20260512.md), [tiny-known-split-denoising-sanity-20260512.md](tiny-known-split-denoising-sanity-20260512.md), [mnist-ddpm-x0-reconstruction-scout-20260512.md](mnist-ddpm-x0-reconstruction-scout-20260512.md), [beans-sd15-membership-semantics-correction-20260512.md](beans-sd15-membership-semantics-correction-20260512.md), [beans-sd15-clip-distance-scout-20260512.md](beans-sd15-clip-distance-scout-20260512.md), [beans-sd15-simple-distance-scout-20260512.md](beans-sd15-simple-distance-scout-20260512.md), [beans-sd15-response-contract-ready-20260512.md](beans-sd15-response-contract-ready-20260512.md), [beans-sd15-response-contract-scout-20260512.md](beans-sd15-response-contract-scout-20260512.md), [mnist-ddpm-pia-portability-smoke-20260512.md](mnist-ddpm-pia-portability-smoke-20260512.md), [midfreq-residual-comparator-audit-20260512.md](midfreq-residual-comparator-audit-20260512.md), [midfreq-residual-stability-result-20260512.md](midfreq-residual-stability-result-20260512.md), [midfreq-residual-stability-decision-20260512.md](midfreq-residual-stability-decision-20260512.md), [midfreq-residual-signcheck-20260512.md](midfreq-residual-signcheck-20260512.md), [midfreq-same-noise-residual-preflight-20260512.md](midfreq-same-noise-residual-preflight-20260512.md), [midfreq-residual-scorer-contract-20260512.md](midfreq-residual-scorer-contract-20260512.md), [midfreq-residual-collector-contract-20260512.md](midfreq-residual-collector-contract-20260512.md), [midfreq-residual-tiny-runner-contract-20260512.md](midfreq-residual-tiny-runner-contract-20260512.md), [midfreq-residual-real-asset-preflight-20260512.md](midfreq-residual-real-asset-preflight-20260512.md) | ReDiffuse STL-10 split/statistics/resource preflight now records exact `50k / 50k` split binding, no low-level image-statistics leakage on holdout, and CUDA batch `64` official DDPM model/trainer feasibility; this releases exactly one bounded STL-10 scout but no membership metric, score packet, admitted row, full training, Tiny-ImageNet download, or Stable Diffusion download. CopyMark `laion_mi` public binding gate records a bounded public row-binding failure: the current public member parquet exposes only `url/caption`, the official member utility still expects a hidden third column, official numeric member filenames exceed the current public row range, and a live spot-check finds only `4/10` of the first public member URLs still return `200`; keep it as support-only CopyMark evidence with no large download or GPU release. Stable Diffusion ReDiffuse collaborator artifact audit records a real imported `5000`-row `2500 / 2500` result packet with replayed `AUC = 0.710319`, but it remains candidate-only because it is a collaborator local transfer, uses a LAION-like member subset rather than the exact paper split, and is not a strict external API-only packet; public metadata sweep after HF auth and GitHub artifact searches found no new non-duplicate replay packet; CLiD ZIP remains range-inaccessible with auth, CopyMark HF ZIP remains already-covered and too large to change the current decision; CopyMark official score-artifact support evidence with public member/nonmember logs, aggregate ROC/threshold JSONs, selected all-step tensors, laion_ridar/mixing results, but no checkpoint hashes, compact row-ID-bound score manifest, small immutable data/checkpoint packet, or ready verifier; Shake-to-Leak code-public fine-tuning-amplified generative privacy watch-plus with target/data/score artifacts missing, FSECLab MIA-Diffusion official DDIM/DCGAN code-public but checkpoint/score/result-missing watch-plus, GenAI Confessions raw-input data-public but response/checkpoint missing black-box boundary watch, strong official CLiD CPU inter-output replay that remains prompt-conditioned candidate-only, weak MIDST TabDDPM EPT scout after nearest-neighbor and shadow-distributional failures, Diffusion Memorization semantic-shift watch, ReDiffuse official OpenReview split-manifest provenance, Reconstruction, variation, H2/simple-distance, weak Beans LoRA parameter-delta sensitivity and conditional denoising-loss under repaired known-split membership semantics, Quantile Regression sample-conditioned reconstruction-loss mechanism reference that is artifact-incomplete, MIAGM generated-distribution reference that is artifact-incomplete, Noise as a Probe semantic-initial-noise mechanism watch that is reproduction-incomplete, Zenodo fine-tuned diffusion paper/code-backed archive watch that remains split-manifest incomplete, LAION-mi metadata-only watch after failed fixed `25/25` URL availability probe, true second membership benchmark scope, weak CommonCanvas conditional denoising-loss scout, weak MIDST TabDDPM nearest-neighbor scout, weak MIDST shadow-distributional scout, weak Fashion-MNIST DDPM PIA-loss scout, Kohaku/Danbooru membership-semantics block, CopyMark provenance intake, local CommonCanvas query asset, completed `50/50` CommonCanvas responses with weak pixel-distance, CLIP image-similarity, prompt-response consistency, multi-seed response-stability, and conditional denoising-loss scorers, weak `64/64` gradient-prototype alignment scout, external provenance scan, Beans contract/debug boundary, MNIST/DDPM raw-loss and x0 simple-scorer scouts, tiny known-split raw-MSE sanity checks, tiny overfit gradient-norm mechanism signal and weakened stability gate, and same-noise residual candidate status. | | ||
| | Black-box | [workspaces/black-box/README.md](../../workspaces/black-box/README.md), [plan.md](../../workspaces/black-box/plan.md), [rediffuse-stl10-bounded-scout-20260525.md](rediffuse-stl10-bounded-scout-20260525.md), [rediffuse-stl10-split-and-microtrain-preflight-20260525.md](rediffuse-stl10-split-and-microtrain-preflight-20260525.md), [copymark-laion-mi-public-binding-gate-20260517.md](copymark-laion-mi-public-binding-gate-20260517.md), [stable-diffusion-rediffuse-collaborator-artifact-20260517.md](stable-diffusion-rediffuse-collaborator-artifact-20260517.md), [public-metadata-asset-sweep-20260515.md](public-metadata-asset-sweep-20260515.md), [copymark-official-score-artifact-gate-20260515.md](copymark-official-score-artifact-gate-20260515.md), [shake-to-leak-code-artifact-gate-20260515.md](shake-to-leak-code-artifact-gate-20260515.md), [fseclab-mia-diffusion-code-artifact-gate-20260515.md](fseclab-mia-diffusion-code-artifact-gate-20260515.md), [genai-confessions-black-box-artifact-gate-20260515.md](genai-confessions-black-box-artifact-gate-20260515.md), [clid-official-inter-output-replay-20260515.md](clid-official-inter-output-replay-20260515.md), [midst-tabddpm-ept-scout-20260515.md](midst-tabddpm-ept-scout-20260515.md), [diffusion-memorization-asset-gate-20260515.md](diffusion-memorization-asset-gate-20260515.md), [rediffuse-openreview-split-manifest-audit-20260515.md](rediffuse-openreview-split-manifest-audit-20260515.md), [beans-lora-delta-sensitivity-20260513.md](beans-lora-delta-sensitivity-20260513.md), [quantile-regression-asset-verdict-20260513.md](quantile-regression-asset-verdict-20260513.md), [miagm-asset-verdict-20260513.md](miagm-asset-verdict-20260513.md), [noise-as-probe-asset-verdict-20260513.md](noise-as-probe-asset-verdict-20260513.md), [zenodo-code-reference-audit-20260513.md](zenodo-code-reference-audit-20260513.md), [zenodo-finetuned-diffusion-asset-verdict-20260513.md](zenodo-finetuned-diffusion-asset-verdict-20260513.md), [laion-mi-url-availability-probe-20260513.md](laion-mi-url-availability-probe-20260513.md), [laion-mi-asset-verdict-20260513.md](laion-mi-asset-verdict-20260513.md), [commoncanvas-denoising-loss-20260513.md](commoncanvas-denoising-loss-20260513.md), [midst-tabddpm-shadow-distributional-scout-20260513.md](midst-tabddpm-shadow-distributional-scout-20260513.md), [midst-tabddpm-nearest-neighbor-scout-20260513.md](midst-tabddpm-nearest-neighbor-scout-20260513.md), [copymark-commoncanvas-multiseed-stability-20260513.md](copymark-commoncanvas-multiseed-stability-20260513.md), [fashion-mnist-ddpm-pia-loss-scout-20260513.md](fashion-mnist-ddpm-pia-loss-scout-20260513.md), [kohaku-danbooru-asset-decision-20260513.md](kohaku-danbooru-asset-decision-20260513.md), [tiny-known-split-gradient-prototype-alignment-20260513.md](tiny-known-split-gradient-prototype-alignment-20260513.md), [copymark-commoncanvas-response-preflight-20260512.md](copymark-commoncanvas-response-preflight-20260512.md), [copymark-commoncanvas-query-asset-20260512.md](copymark-commoncanvas-query-asset-20260512.md), [copymark-provenance-intake-20260512.md](copymark-provenance-intake-20260512.md), [external-diffusion-benchmark-provenance-scan-20260512.md](external-diffusion-benchmark-provenance-scan-20260512.md), [true-second-membership-benchmark-scope-20260512.md](true-second-membership-benchmark-scope-20260512.md), [gradient-norm-stability-gate-20260512.md](gradient-norm-stability-gate-20260512.md), [tiny-overfit-gradient-norm-scout-20260512.md](tiny-overfit-gradient-norm-scout-20260512.md), [tiny-overfit-mse-upperbound-20260512.md](tiny-overfit-mse-upperbound-20260512.md), [tiny-known-split-denoising-sanity-20260512.md](tiny-known-split-denoising-sanity-20260512.md), [mnist-ddpm-x0-reconstruction-scout-20260512.md](mnist-ddpm-x0-reconstruction-scout-20260512.md), [beans-sd15-membership-semantics-correction-20260512.md](beans-sd15-membership-semantics-correction-20260512.md), [beans-sd15-clip-distance-scout-20260512.md](beans-sd15-clip-distance-scout-20260512.md), [beans-sd15-simple-distance-scout-20260512.md](beans-sd15-simple-distance-scout-20260512.md), [beans-sd15-response-contract-ready-20260512.md](beans-sd15-response-contract-ready-20260512.md), [beans-sd15-response-contract-scout-20260512.md](beans-sd15-response-contract-scout-20260512.md), [mnist-ddpm-pia-portability-smoke-20260512.md](mnist-ddpm-pia-portability-smoke-20260512.md), [midfreq-residual-comparator-audit-20260512.md](midfreq-residual-comparator-audit-20260512.md), [midfreq-residual-stability-result-20260512.md](midfreq-residual-stability-result-20260512.md), [midfreq-residual-stability-decision-20260512.md](midfreq-residual-stability-decision-20260512.md), [midfreq-residual-signcheck-20260512.md](midfreq-residual-signcheck-20260512.md), [midfreq-same-noise-residual-preflight-20260512.md](midfreq-same-noise-residual-preflight-20260512.md), [midfreq-residual-scorer-contract-20260512.md](midfreq-residual-scorer-contract-20260512.md), [midfreq-residual-collector-contract-20260512.md](midfreq-residual-collector-contract-20260512.md), [midfreq-residual-tiny-runner-contract-20260512.md](midfreq-residual-tiny-runner-contract-20260512.md), [midfreq-residual-real-asset-preflight-20260512.md](midfreq-residual-real-asset-preflight-20260512.md) | ReDiffuse STL-10 bounded scout now records a scoreable but weak short-target packet: `300` steps, `256 / 256` scores, `AUC = 0.4996337890625`, `ASR = 0.509765625`, `TPR@1%FPR = 0.01171875`, and no GPU expansion. ReDiffuse STL-10 split/statistics/resource preflight recorded exact `50k / 50k` split binding, no low-level image-statistics leakage on holdout, and CUDA batch `64` official DDPM model/trainer feasibility. CopyMark `laion_mi` public binding gate records a bounded public row-binding failure: the current public member parquet exposes only `url/caption`, the official member utility still expects a hidden third column, official numeric member filenames exceed the current public row range, and a live spot-check finds only `4/10` of the first public member URLs still return `200`; keep it as support-only CopyMark evidence with no large download or GPU release. Stable Diffusion ReDiffuse collaborator artifact audit records a real imported `5000`-row `2500 / 2500` result packet with replayed `AUC = 0.710319`, but it remains candidate-only because it is a collaborator local transfer, uses a LAION-like member subset rather than the exact paper split, and is not a strict external API-only packet; public metadata sweep after HF auth and GitHub artifact searches found no new non-duplicate replay packet; CLiD ZIP remains range-inaccessible with auth, CopyMark HF ZIP remains already-covered and too large to change the current decision; CopyMark official score-artifact support evidence with public member/nonmember logs, aggregate ROC/threshold JSONs, selected all-step tensors, laion_ridar/mixing results, but no checkpoint hashes, compact row-ID-bound score manifest, small immutable data/checkpoint packet, or ready verifier; Shake-to-Leak code-public fine-tuning-amplified generative privacy watch-plus with target/data/score artifacts missing, FSECLab MIA-Diffusion official DDIM/DCGAN code-public but checkpoint/score/result-missing watch-plus, GenAI Confessions raw-input data-public but response/checkpoint missing black-box boundary watch, strong official CLiD CPU inter-output replay that remains prompt-conditioned candidate-only, weak MIDST TabDDPM EPT scout after nearest-neighbor and shadow-distributional failures, Diffusion Memorization semantic-shift watch, ReDiffuse official OpenReview split-manifest provenance, Reconstruction, variation, H2/simple-distance, weak Beans LoRA parameter-delta sensitivity and conditional denoising-loss under repaired known-split membership semantics, Quantile Regression sample-conditioned reconstruction-loss mechanism reference that is artifact-incomplete, MIAGM generated-distribution reference that is artifact-incomplete, Noise as a Probe semantic-initial-noise mechanism watch that is reproduction-incomplete, Zenodo fine-tuned diffusion paper/code-backed archive watch that remains split-manifest incomplete, LAION-mi metadata-only watch after failed fixed `25/25` URL availability probe, true second membership benchmark scope, weak CommonCanvas conditional denoising-loss scout, weak MIDST TabDDPM nearest-neighbor scout, weak MIDST shadow-distributional scout, weak Fashion-MNIST DDPM PIA-loss scout, Kohaku/Danbooru membership-semantics block, CopyMark provenance intake, local CommonCanvas query asset, completed `50/50` CommonCanvas responses with weak pixel-distance, CLIP image-similarity, prompt-response consistency, multi-seed response-stability, and conditional denoising-loss scorers, weak `64/64` gradient-prototype alignment scout, external provenance scan, Beans contract/debug boundary, MNIST/DDPM raw-loss and x0 simple-scorer scouts, tiny known-split raw-MSE sanity checks, tiny overfit gradient-norm mechanism signal and weakened stability gate, and same-noise residual candidate status. | |
There was a problem hiding this comment.
The ReDiffuse research line (including the STL-10 bounded scout and the collaborator Stable Diffusion artifacts) is classified under the Black-box track in this table, but it is consistently listed as Gray-box ReDiffuse in docs/evidence/reproduction-status.md (line 75). To maintain structural consistency across the documentation, these documents and their corresponding summary text should be moved to the Gray-box track row (line 311).
Summary
Checks