Skip to content

[EN-1595] fix(orchestrator): unpack OCI layers in chroot like Docker daemon#3215

Merged
jakubno merged 8 commits into
mainfrom
fix/clamp-symlinks-to-root
Jul 7, 2026
Merged

[EN-1595] fix(orchestrator): unpack OCI layers in chroot like Docker daemon#3215
jakubno merged 8 commits into
mainfrom
fix/clamp-symlinks-to-root

Conversation

@jakubno

@jakubno jakubno commented Jul 7, 2026

Copy link
Copy Markdown
Member
  • Fixes an issue for symlinks as /../../a/b/c to be resolved to /a/b/c
  • Change in behavior ❗ : Nested whiteouts aren't supported anymore (matches Docker behavior)
  • IgnoreChownErrors shouldn't be needed as it's run as root

Layers containing relative symlinks that lexically escape the layer dir but resolve fine at runtime (e.g. /nix/store/.links entries created by nix store optimise) failed with 'invalid symlink'. Use moby/go-archive chrootarchive, which untars on an OS thread chrooted into the layer dir so the breakout check clamps at the layer root. Drops the containers/storage dependency.

Layers containing relative symlinks that lexically escape the layer dir
but resolve fine at runtime (e.g. /nix/store/.links entries created by
nix store optimise) failed with 'invalid symlink'. Use
moby/go-archive chrootarchive, which untars on an OS thread chrooted
into the layer dir so the breakout check clamps at the layer root.
Drops the containers/storage dependency.
@cla-bot cla-bot Bot added the cla-signed label Jul 7, 2026
@cursor

cursor Bot commented Jul 7, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes how every OCI image layer is unpacked during template builds; most images gain compatibility but some with nested whiteouts may start failing, and chroot-based unpack depends on correct privileges in the build environment.

Overview
Template builds were failing when OCI layers contained relative symlinks whose paths step above the layer root (for example Nix .links entries from nix store optimise), because the previous unpack path treated those as invalid. Layer extraction now uses the same chroot-based untar path as the Docker daemon, which keeps symlink targets verbatim while still applying overlay whiteouts. Behavior change: images with redundant nested whiteouts that stock Docker also rejects will now fail at unpack instead of being accepted, and IgnoreChownErrors is no longer passed because export runs as root.

Reviewed by Cursor Bugbot for commit 26f6535. Bugbot is set up for automated code reviews on this repo. Configure here.

@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
3179 1 3178 6
View the top 1 failed test(s) by shortest run time
github.com/e2b-dev/infra/tests/integration/internal/tests/api/sandboxes::TestEgressFirewallAllowDomainAndIP
Stack Traces | 4.54s run time
=== RUN   TestEgressFirewallAllowDomainAndIP
=== PAUSE TestEgressFirewallAllowDomainAndIP
=== CONT  TestEgressFirewallAllowDomainAndIP
Executing command curl in sandbox ikjt9lsv1fnuxexlqlb46
    sandbox_network_out_test.go:68: Command [curl] output: event:{start:{pid:1054}}
    sandbox_network_out_test.go:68: Command [curl] output: event:{data:{stdout:"HTTP/2 301 \r\nlocation: https://www.google.com/\r\ncontent-type: text/html; charset=UTF-8\r\ncontent-security-policy-report-only: object-src 'none';base-uri 'self';script-src 'nonce-ToZtrQxbOQBB2fApIM6evQ' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle..../csp/gws/other-hp\r\ndate: Tue, 07 Jul 2026 19:29:24 GMT\r\nexpires: Thu, 06 Aug 2026 19:29:24 GMT\r\ncache-control: public, max-age=2592000\r\nserver: gws\r\ncontent-length: 220\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_out_test.go:68: Command [curl] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_network_out_test.go:68: Command [curl] completed successfully in sandbox is5yb09wojcvk80oenzw1
Executing command curl in sandbox is5yb09wojcvk80oenzw1
    sandbox_network_out_test.go:68: Command [curl] output: event:{start:{pid:1056}}
    sandbox_network_out_test.go:68: Command [curl] output: event:{data:{stdout:"HTTP/2 301 \r\ndate: Tue, 07 Jul 2026 19:29:25 GMT\r\nlocation: https://one.one.one.one/\r\nserver: cloudflare\r\ncf-ray: a1794203dbb17112-IAD\r\n\r\n"}}
    sandbox_network_out_test.go:68: Command [curl] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_network_out_test.go:68: Command [curl] completed successfully in sandbox is5yb09wojcvk80oenzw1
Executing command curl in sandbox is5yb09wojcvk80oenzw1
    sandbox_network_out_test.go:678: Command [curl] output: event:{start:{pid:1057}}
    sandbox_network_out_test.go:678: Command [curl] output: event:{end:{exit_code:35  exited:true  status:"exit status 35"  error:"exit status 35"}}
Executing command curl in sandbox is5yb09wojcvk80oenzw1
    sandbox_network_out_test.go:679: Command [curl] output: event:{start:{pid:1059}}
    sandbox_network_out_test.go:679: 
        	Error Trace:	.../api/sandboxes/sandbox_network_out_test.go:78
        	            				.../api/sandboxes/sandbox_network_out_test.go:679
        	Error:      	"failed to execute command curl in sandbox is5yb09wojcvk80oenzw1: invalid_argument: protocol error: incomplete envelope: unexpected EOF" does not contain "failed with exit code"
        	Test:       	TestEgressFirewallAllowDomainAndIP
        	Messages:   	Expected connection failure message
--- FAIL: TestEgressFirewallAllowDomainAndIP (4.54s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d00ec07. Configure here.

Comment thread packages/orchestrator/pkg/template/build/core/oci/oci.go
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

@jakubno jakubno changed the title fix(orchestrator): unpack OCI layers in chroot like Docker daemon [EN-1595] fix(orchestrator): unpack OCI layers in chroot like Docker daemon Jul 7, 2026
@linear-code

linear-code Bot commented Jul 7, 2026

Copy link
Copy Markdown

EN-1595

@jakubno jakubno marked this pull request as ready for review July 7, 2026 13:46

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d00ec074f1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/orchestrator/pkg/template/build/core/oci/oci.go
Layers with .wh.foo + foo/.wh.bar (rm -rf-style deletes, emitted by
kaniko and others) fail with ENOTDIR on moby chrootarchive; the
previous containers/storage path tolerated the redundant child
whiteout.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Comment thread packages/orchestrator/pkg/template/build/core/oci/whiteout_filter.go Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3cd8331573

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/orchestrator/pkg/template/build/core/oci/whiteout_filter.go Outdated
Comment thread packages/orchestrator/pkg/template/build/core/oci/untar_test.go Outdated
@jakubno jakubno force-pushed the fix/clamp-symlinks-to-root branch from 3cd8331 to 9386304 Compare July 7, 2026 14:26
jakubno added 2 commits July 7, 2026 14:27
euid 0 alone is not enough: rootful containers without CAP_SYS_ADMIN
fail unshare(CLONE_NEWNS) with EPERM. Probe on a discarded locked OS
thread and skip when unavailable.
Layers with redundant nested whiteouts (.wh.foo + foo/.wh.bar) are
rejected by stock Docker too (overlay2 and containerd snapshotter both
fail with ENOTDIR); only containers/storage tolerates them. Not worth
carrying a custom tar filter for images Docker cannot pull. Test now
documents the limitation.
)

// untarLayer runs the same unpack call used by createExport in oci.go.
func untarLayer(t *testing.T, layer *bytes.Buffer) (string, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the tests here are testing if the library works as expected (which they shouldn't), not if our implementation works as expected

@jakubno jakubno merged commit 507993a into main Jul 7, 2026
44 checks passed
@jakubno jakubno deleted the fix/clamp-symlinks-to-root branch July 7, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants