Skip to content

fix(cow): return ENOENT when opening a file deleted in the branch#144

Open
dzerik wants to merge 2 commits into
multikernel:mainfrom
dzerik:fix/cow-whiteout-read-path
Open

fix(cow): return ENOENT when opening a file deleted in the branch#144
dzerik wants to merge 2 commits into
multikernel:mainfrom
dzerik:fix/cow-whiteout-read-path

Conversation

@dzerik

@dzerik dzerik commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Problem

A file deleted in a seccomp COW branch is recorded as a whiteout in the in-memory deleted set; the real lower file is intentionally left untouched until commit. The stat/access handlers honor that whiteout and return ENOENT, but the read/open path did not: for a deleted path opened without O_CREAT, prepare_open returned CowOpenPlan::Skip, which handle_cow_open maps to NotifAction::Continue — so the kernel opened the untouched lower file and the reader saw the file's pre-delete content.

The two paths disagreed, so a deletion was invisible to a reader.

Example

# workdir/secret.txt exists with content "PREDELETE", COW active
rm -f secret.txt
test -e secret.txt        # -> false   (stat path: ENOENT, correct)
dd if=secret.txt of=out   # -> succeeds, out == "PREDELETE"  (open path: leaks lower bytes)

cat secret.txt happens to hide the bug because coreutils cat stats first and short-circuits on the (correct) stat ENOENT; a bare open() with no preceding stat (dd, a raw openat, some readers) hits the leaking path.

Fix

Add CowOpenPlan::Deleted for a whiteout opened without O_CREAT; handle_cow_open maps it to NotifAction::Errno(libc::ENOENT), matching the stat/access handlers. Reads of a deleted file now fail closed instead of leaking lower content.

Tests

  • Unit: prepare_open reports Deleted (not Skip) for a whiteout opened O_RDONLY.
  • Integration: a child deletes a workdir file, then a bare open (dd) must be denied (ENOENT) and copy zero bytes — the test fails with leaked: "PREDELETE" on the old behavior.

Scope

This is a general COW whiteout fix on the read/open path — it also reproduces in a single (non-pipeline) sandbox. It additionally makes a stage's deletion visible to later stages under the read-committed model of the multi-sandbox transactions work (#65), which is why I split it out as its own small change.

A file deleted in a seccomp COW branch is recorded as a whiteout in the
in-memory `deleted` set; the real lower file is intentionally left untouched
until commit. The stat/access handlers honor the whiteout and return ENOENT,
but the read/open path returned `CowOpenPlan::Skip` -> `NotifAction::Continue`,
so the kernel opened the untouched lower file and the reader saw the file's
pre-delete content. Concretely: after `rm f`, `test -e f` reports the deletion
(stat path) while a bare `open(f, O_RDONLY)` — e.g. `dd if=f`, which does not
stat first — reads the original bytes. The two paths disagreed and a deletion
was invisible to a reader.

Add `CowOpenPlan::Deleted` for a whiteout opened without `O_CREAT`; dispatch
maps it to `Errno(ENOENT)`, matching the stat/access handlers. Reads of a
deleted file now fail closed instead of leaking lower content. This also makes
a stage's deletion visible to later stages under the transactional-pipeline
read-committed model.

Tests: a unit test asserting `prepare_open` reports `Deleted` for a whiteout,
and an integration test where a child deletes a workdir file and a bare open
(`dd`) must be denied rather than read the pre-delete bytes.

@congwang-mk congwang-mk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Two issues in the same bug class, one per inline comment.

// lower file (which still physically exists with its pre-delete
// content); report the deletion so the caller returns ENOENT,
// matching the stat/access path.
return Ok(CowOpenPlan::Deleted);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes the seccomp dispatcher, but chroot mode still leaks: the sync handle_open above (~line 385) returns Ok(None) for a whiteout without O_CREAT, and chroot/dispatch.rs maps Ok(None) to a fall-through that opens the real lower file via openat2_in_root, so a reader still gets the pre-delete bytes there. handle_open should surface the whiteout too, with the chroot call site mapping it to ENOENT.

@@ -447,7 +451,11 @@ impl SeccompCowBranch {
if flags & O_CREAT != 0 {
return self.prepare_cow_copy(&rel);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

O_CREAT on a whiteout bypasses the fail-closed guarantee this PR adds: prepare_copy removes the path from deleted and copies the lower bytes into upper, so rm f; touch f; cat f (or a bare open(f, O_CREAT|O_RDONLY)) resurrects the pre-delete content. After unlink, O_CREAT should create an empty upper file instead of copying lower. Fine as a tracked follow-up if out of scope here.

PR multikernel#144 fixed the async seccomp-notif open path (prepare_open ->
CowOpenPlan::Deleted -> Errno(ENOENT)), but the sync handle_open still
returned Ok(None) for a whiteout without O_CREAT, and chroot/dispatch.rs
maps Ok(None) to a fall-through that opens the real lower file via
openat2_in_root — so a reader under chroot mode still got the pre-delete
bytes (maintainer review, Inline #1).

Add BranchError::Deleted (sync mirror of CowOpenPlan::Deleted): handle_open
now returns it for a read-open of a whiteout, and the single chroot call
site maps it to ENOENT, matching the async path and the
stat/statx/access/readlink handlers. Covers both openat and legacy open
(legacy synthesizes into handle_chroot_open).

BranchError is not #[non_exhaustive] and every other match on it uses a
wildcard, so the added variant compiles cleanly; the only production caller
of handle_open is chroot/dispatch.rs.

Tests: a sync unit test mirrors the async prepare_open one, and a chroot-mode
integration test (test_chroot_cow_read_deleted_file_is_enoent) drives a bare
`cat` open of a whiteout through the real chroot dispatcher and asserts the
pre-delete lower bytes do not leak — the chroot sibling of the async
test_seccomp_cow_read_deleted_file_is_enoent. Verified it fails (leaks
"PREDELETE") without the handle_open fix.
@dzerik

dzerik commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

Fixed Inline #1: added BranchError::Deleted (the sync mirror of CowOpenPlan::Deleted) — handle_open now surfaces the whiteout instead of Ok(None), and the single chroot call site maps it to ENOENT, matching the async path and the stat/statx/access/readlink handlers. Covers both openat and legacy open (legacy synthesizes into handle_chroot_open).

Added a chroot-mode integration test (test_chroot_cow_read_deleted_file_is_enoent) that drives a bare cat open (no preceding stat) of a whiteout through the real chroot dispatcher and asserts the pre-delete lower bytes do not leak — the chroot sibling of your async test_seccomp_cow_read_deleted_file_is_enoent. Confirmed it fails (leaks the lower "PREDELETE" bytes) without the handle_open change, so it locks the actual no-leak contract rather than just the branch return value.

Inline #2 (O_CREAT-on-whiteout resurrection) I'll take as the tracked follow-up you suggested — it also needs test_quota_handle_open_create_denied rewritten (it currently asserts the old copy-into-upper quota behavior), so keeping it out of this ENOENT-signal change keeps both reviewable in isolation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants