-
Notifications
You must be signed in to change notification settings - Fork 1.4k
feat: validate actions for merged-usr violations #1912
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
617cd7b
feat: validate actions for merged-usr violations
thesayyn b620b53
Update private/util/validate_usr_symlinks.bzl
thesayyn 06b929d
Update validate_usr_symlinks.bzl
thesayyn ad4cccb
Update MODULE.bazel
thesayyn 833b9c2
Update MODULE.bazel.lock
thesayyn d130ebf
Update MODULE.bazel
thesayyn 8b9906c
Update MODULE.bazel.lock
thesayyn 022f61b
Update validate_usr_symlinks.bzl
thesayyn 10e936a
Update validate_usr_symlinks.bzl
thesayyn 3329f94
review comments
thesayyn ea6c116
comments
thesayyn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| exports_files(["validate_usr_symlinks.awk"]) | ||
|
|
||
| sh_test( | ||
| name = "validate_usr_symlinks_test", | ||
| srcs = ["validate_usr_symlinks_test.sh"], | ||
| data = [ | ||
| "validate_usr_symlinks.awk", | ||
| "@bazel_tools//tools/bash/runfiles", | ||
| "@gawk", | ||
| ], | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| BEGIN { | ||
| # Mapping from root-level path to expected symlink destination. | ||
| # https://github.com/floppym/merge-usr/blob/15dd02207bdee7ca6720d7024e8c0ffdc166ed23/merge-usr#L17-L25 | ||
| # Note: Debian does NOT merge /usr/sbin into /usr/bin, so /sbin -> usr/sbin. | ||
| expected["bin"] = "usr/bin" | ||
| expected["sbin"] = "usr/sbin" | ||
| expected["lib"] = "usr/lib" | ||
| expected["lib32"] = "usr/lib32" | ||
| expected["lib64"] = "usr/lib64" | ||
| expected["libx32"] = "usr/libx32" | ||
| prefixes = "bin|sbin|lib|lib32|lib64|libx32" | ||
| } | ||
| { | ||
| original_path = $1 | ||
| path = original_path | ||
| # Normalize: strip leading ./ or / | ||
| sub(/^\.\//, "", path) | ||
| sub(/^\//, "", path) | ||
|
|
||
| if (path in expected) { | ||
| if ($0 !~ /type=link/) { | ||
| VIOLATIONS[original_path] = original_path " is not a symlink (must link to " expected[path] ")" | ||
| } else if (match($0, / link=([^ \t]+)/, dest) && dest[1] != expected[path]) { | ||
| VIOLATIONS[original_path] = original_path " symlinks to '" dest[1] "' instead of '" expected[path] "'" | ||
| } | ||
| } else if (path ~ ("^(" prefixes ")/")) { | ||
| VIOLATIONS[original_path] = original_path " found under a merged-usr symlink path (should not exist)" | ||
| } | ||
| } | ||
|
thesayyn marked this conversation as resolved.
|
||
| END { | ||
| for (violation in VIOLATIONS) { | ||
| print "VIOLATION: " VIOLATIONS[violation] " violates usr-merge convention." | ||
| print violation | ||
| } | ||
| if (length(VIOLATIONS) > 0) { | ||
| exit 1 | ||
| } | ||
| print "" > validation_output_file | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| "Bazel aspect to validate merged-usr conventions in tar files." | ||
|
|
||
| load("@aspect_bazel_lib//lib:tar.bzl", "tar_lib") | ||
|
|
||
| # https://wiki.gentoo.org/wiki/Merge-usr | ||
| # https://salsa.debian.org/md/usrmerge/raw/master/debian/README.Debian | ||
| # https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/ | ||
| # Mapping taken from https://github.com/floppym/merge-usr/blob/15dd02207bdee7ca6720d7024e8c0ffdc166ed23/merge-usr#L17-L25 | ||
| # https://salsa.debian.org/md/usrmerge/-/tree/master/debian?ref_type=heads | ||
|
|
||
| def _validate_usr_symlink_impl(target, ctx): | ||
| if target.label.name.find("debian12") != -1: | ||
| return [] | ||
|
|
||
| if not hasattr(ctx.rule.files, "tars"): | ||
| return [] | ||
| bsdtar = ctx.toolchains[tar_lib.toolchain_type] | ||
|
|
||
| output = ctx.actions.declare_file(target.label.name + ".mtree") | ||
|
|
||
| args = ctx.actions.args() | ||
| args.add("--create") | ||
| args.add("--file", output) | ||
| args.add("--format=mtree") | ||
| args.add_all(ctx.rule.files.tars, format_each = "@%s") | ||
|
|
||
| ctx.actions.run( | ||
| executable = bsdtar.tarinfo.binary, | ||
| inputs = ctx.rule.files.tars, | ||
| outputs = [output], | ||
| tools = bsdtar.default.files, | ||
| arguments = [args], | ||
| mnemonic = "PackageListing", | ||
| ) | ||
|
|
||
| validation_output = ctx.actions.declare_file(target.label.name + ".validation") | ||
| ctx.actions.run( | ||
| executable = ctx.executable._awk, | ||
| inputs = [output, ctx.file._validate_symlinks], | ||
| outputs = [validation_output], | ||
| arguments = [ | ||
| "-v", | ||
| "validation_output_file=" + validation_output.path, | ||
| "-f", | ||
| ctx.file._validate_symlinks.path, | ||
| output.path, | ||
| ], | ||
|
thesayyn marked this conversation as resolved.
|
||
| mnemonic = "ValidateUsrSymlinks", | ||
| ) | ||
|
|
||
| return [ | ||
| OutputGroupInfo(_validation = depset([validation_output])), | ||
| ] | ||
|
|
||
| validate_usr_symlinks = aspect( | ||
| implementation = _validate_usr_symlink_impl, | ||
| attrs = { | ||
| "_awk": attr.label(default = "@gawk//:gawk", cfg = "exec", executable = True), | ||
| "_validate_symlinks": attr.label( | ||
| default = "//private/util:validate_usr_symlinks.awk", | ||
| allow_single_file = True, | ||
| ), | ||
| }, | ||
| attr_aspects = ["tars"], | ||
| toolchains = [tar_lib.toolchain_type], | ||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| # Bazel runfiles resolution | ||
| # shellcheck source=/dev/null | ||
| source "${RUNFILES_DIR:-/dev/null}/bazel_tools/tools/bash/runfiles/runfiles.bash" 2>/dev/null || | ||
| source "$(dirname "$0")/../bazel_tools/tools/bash/runfiles/runfiles.bash" 2>/dev/null || | ||
| { echo >&2 "ERROR: cannot find runfiles.bash"; exit 1; } | ||
|
|
||
| GAWK="$(rlocation gawk/gawk)" | ||
| AWK_SCRIPT="$(rlocation distroless/private/util/validate_usr_symlinks.awk)" | ||
|
|
||
| run() { | ||
| printf '%s\n' "$1" | "$GAWK" -v validation_output_file=/dev/null -f "$AWK_SCRIPT" | ||
| } | ||
|
|
||
| fail() { echo "FAIL: $*" >&2; exit 1; } | ||
|
|
||
| # --- passing cases --- | ||
|
thesayyn marked this conversation as resolved.
|
||
|
|
||
| run "./bin type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/bin" \ | ||
| || fail "./bin -> usr/bin should pass" | ||
|
|
||
| run "/bin type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/bin" \ | ||
| || fail "/bin -> usr/bin should pass" | ||
|
|
||
| run "bin type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/bin" \ | ||
| || fail "bin -> usr/bin should pass" | ||
|
|
||
| run "./sbin type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/sbin" \ | ||
| || fail "./sbin -> usr/sbin should pass" | ||
|
|
||
| run "./lib type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/lib" \ | ||
| || fail "./lib -> usr/lib should pass" | ||
|
|
||
| run "./lib32 type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/lib32" \ | ||
| || fail "./lib32 -> usr/lib32 should pass" | ||
|
|
||
| run "./lib64 type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/lib64" \ | ||
| || fail "./lib64 -> usr/lib64 should pass" | ||
|
|
||
| run "./lib64 type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/lib64 | ||
| ./usr/bin/ls type=file mode=0755 nlink=1 uid=0 gid=0 size=12345" \ | ||
| || fail "content under ./usr/ alongside valid symlinks should pass" | ||
|
|
||
| # --- failing cases --- | ||
|
|
||
|
thesayyn marked this conversation as resolved.
|
||
| run "./bin type=dir mode=0755 nlink=2 uid=0 gid=0" \ | ||
| && fail "./bin as a directory should fail" || true | ||
|
|
||
| run "./bin type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/sbin" \ | ||
| && fail "./bin -> usr/sbin should fail" || true | ||
|
|
||
| run "bin type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/fin" \ | ||
| && fail "bin -> usr/fin should fail" || true | ||
|
|
||
| run "/bin type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/fin" \ | ||
| && fail "/bin -> usr/fin should fail" || true | ||
|
|
||
| run "./sbin type=link mode=0777 nlink=1 uid=0 gid=0 link=usr/bin" \ | ||
| && fail "./sbin -> usr/bin should fail (Debian keeps sbin separate)" || true | ||
|
|
||
| run "./lib/libfoo.so.1 type=file mode=0644 nlink=1 uid=0 gid=0 size=4096" \ | ||
| && fail "content under ./lib/ should fail" || true | ||
|
|
||
|
thesayyn marked this conversation as resolved.
|
||
| run "lib/libfoo.so.1 type=file mode=0644 nlink=1 uid=0 gid=0 size=4096" \ | ||
| && fail "content under lib/ should fail" || true | ||
|
|
||
| run "/lib/libfoo.so.1 type=file mode=0644 nlink=1 uid=0 gid=0 size=4096" \ | ||
| && fail "content under /lib/ should fail" || true | ||
|
|
||
| run "./bin/ls type=file mode=0755 nlink=1 uid=0 gid=0 size=12345" \ | ||
| && fail "content under ./bin/ should fail" || true | ||
|
|
||
| echo "All tests passed." | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.