Skip to content

Commit 11505d0

Browse files
pcjpcj
authored
Implement //example:docker (#3)
1 parent 39a40df commit 11505d0

67 files changed

Lines changed: 1349 additions & 7543 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.bazelrc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
build --noexperimental_convenience_symlinks

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
1-
/bazel-*
1+
/bazel-*
2+
/.vscode/
3+
.pdm-python

‎BUILD.bazel‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,16 @@
33
# ----------------------------------------------------
44
# gazelle:prefix github.com/stackb/pycross_image
55

6-
load("//tools/example_test:defs.bzl", "example_test_filegroup")
6+
load("//bazel/tools/example_test:defs.bzl", "example_test_filegroup")
77

88
example_test_filegroup(
99
name = "example_test_files",
1010
srcs = [
1111
"BUILD.bazel",
1212
"WORKSPACE",
13-
"//bazel:example_test_files",
14-
"//platform:example_test_files",
15-
"//rules:example_test_files",
16-
"//tools/example_test:example_test_files",
13+
"//bazel/platforms:example_test_files",
14+
"//bazel/rules:example_test_files",
15+
"//bazel/tools/example_test:example_test_files",
16+
"//bazel/workspace:example_test_files",
1717
],
1818
)

‎Makefile‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
.PHONY: pdm_sync
2+
pdm_sync:
3+
bazel run //bazel/tools/pdm -- sync
4+
rm -rf .venv
5+
6+
.PHONY: pdm_lock
7+
pdm_lock:
8+
bazel run //bazel/tools/pdm -- lock --static-urls
9+
rm -rf .venv
10+
11+
.PHONY: example_oci_pdm_add
12+
example_oci_pdm_add:
13+
PDM_PROJECT=$(PWD)/example/oci bazel run //bazel/tools/pdm -- add grpclib
14+
rm -rf example/oci/.venv

‎README.md‎

Lines changed: 91 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,95 @@
1-
# pycross_image
1+
# @pycross_image
22

3-
`pycross_image` provides starlark rules for building container images for python
4-
applications with bazel.
3+
![build-status](https://github.com/stackb/pycross_image/actions/workflows/ci.yaml/badge.svg)
54

6-
You can use the rules directly from this repo, or simply as examples and
7-
copy/paste them into your own project.
5+
Bazel starlark rules for building container images from `py_binary` :sparkles:
6+
using [@rules_pycross](https://github.com/jvolkman/rules_pycross) :magic:.
87

8+
`@pycross_image` provides:
99

10+
- `load("@pycross_image//bazel/rules:oci.bzl", "py_image")`: image rule
11+
compatible with [@rules_oci](https://github.com/bazel-contrib/rules_oci)
12+
- `load("@pycross_image//bazel/rules:docker.bzl", "py_image")`: image rule
13+
compatible with [@rules_docker](https://github.com/bazelbuild/rules_docker)
14+
15+
## Installation & Usage
16+
17+
See [releases] page for an `http_archive` of the latest `@pycross_image`.
18+
19+
Examples:
20+
- [@rules_oci example](example/oci/WORKSPACE.in).
21+
- [@rules_docker example](example/docker/WORKSPACE.in).
22+
23+
A few notes about the workspace setup:
24+
25+
- It's divided into "steps" based on load statement dependencies. `step1.bzl`
26+
only depends on things declared in `repositories.bzl`, `step2.bzl` depends on
27+
things declared in `step3.bzl`, etc (this pattern is from
28+
[tensorflow](https://github.com/tensorflow/tensorflow/tree/master/tensorflow)).
29+
- Your workspace may already have many of the dependencies. Some of the
30+
external workspace names may be differ from yours. Use the example as a study
31+
guide rather than canonical reference.
32+
- The examples are only tested with the older `WORKSPACE`. The rules may not be
33+
compatible with bzlmod yet.
34+
35+
## How it Works
36+
37+
```mermaid
38+
graph TD;
39+
pypi[(pypi)]
40+
DefaultInfo[[DefaultInfo]]
41+
numpy{{numpy}}
42+
grpclib{{grpclib}}
43+
44+
subgraph linux_x86_64["linuxx86_64\n"]
45+
image.tar-->image;
46+
image-->app_layer;
47+
image-->site_packages_layer;
48+
image-->interpreter_layer;
49+
app_layer-->DefaultInfo;
50+
site_packages_layer-->DefaultInfo;
51+
interpreter_layer-->DefaultInfo;
52+
end
53+
54+
DefaultInfo--transition :linux_x86_64-->pycross_binary;
55+
pycross_binary-->py_binary;
56+
py_binary-->numpy;
57+
py_binary-->grpclib;
58+
59+
numpy-.->numpy-cp310-macosx_arm64.whl
60+
numpy-.->numpy-cp310-manylinux_x86_64.whl
61+
grpclib-.->grpclib.whl
62+
numpy-cp310-macosx_arm64.whl-->pypi
63+
numpy-cp310-manylinux_x86_64.whl-->pypi
64+
65+
subgraph zig_toolchain
66+
grpclib.whl-->grpclib-tar.gz
67+
end
68+
69+
grpclib-tar.gz-->pypi
70+
71+
style pypi fill:#3171b2,stroke:#333
72+
style numpy fill:#5d97d2,stroke:#3171b2,color:black
73+
style grpclib fill:#5d97d2,stroke:#3171b2,color:black
74+
style grpclib.whl stroke:#bc082b
75+
76+
style numpy-cp310-manylinux_x86_64.whl stroke:#bc082b
77+
style numpy-cp310-macosx_arm64.whl stroke:#bb22d8
78+
style linux_x86_64 stroke:#bc082b,fill:none
79+
```
80+
81+
In this example the `py_binary` rule has `deps` on two python wheels:
82+
- `numpy` has a binary wheel available from pypi for both the darwin and linux
83+
platforms.
84+
- `grpclib` only has a source distribution available.
85+
86+
The `pycross_binary` rule transitions from the host platform to `:linux_x86_64`.
87+
- the transition affects how `@rules_pycross` fetches wheels. If the binary
88+
distribution is available, take it.
89+
- if the binary distribution is not available, compile from source using (in
90+
this case, with `zig` and `@hermetic_cc_toolchains`).
91+
92+
The image is partitioned into three tar layers by matching against filename
93+
patterns (see rule implementation for details).
94+
95+
> `@rules_pycross` supports dependency fetching using PDM or poetry.

‎WORKSPACE‎

Lines changed: 22 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,68 +1,51 @@
11
workspace(name = "pycross_image")
22

3-
load("@pycross_image//bazel:repositories.bzl", "repositories")
3+
# -----------------------------------------
4+
5+
load("@pycross_image//bazel/workspace:repositories.bzl", "repositories")
46

57
repositories()
68

79
# -----------------------------------------
810

9-
load(
10-
"@pycross_image//bazel:workspace0.bzl",
11-
"setup_aspect_bazel_lib",
12-
"setup_bazel_gazelle",
13-
"setup_bazel_skylib",
14-
"setup_rules_docker",
15-
"setup_rules_go",
16-
"setup_rules_oci",
17-
"setup_rules_python",
18-
"setup_zig_toolchains",
19-
)
11+
load("@pycross_image//bazel/workspace:step1.bzl", "step1")
12+
13+
step1.setup_zig_toolchains()
2014

21-
setup_zig_toolchains()
15+
step1.setup_rules_go()
2216

23-
setup_rules_go()
17+
step1.setup_bazel_gazelle()
2418

25-
setup_bazel_gazelle()
19+
step1.setup_bazel_skylib()
2620

27-
setup_bazel_skylib()
21+
step1.setup_rules_python()
2822

29-
setup_rules_python()
23+
step1.setup_aspect_bazel_lib()
3024

31-
setup_aspect_bazel_lib()
25+
step1.setup_rules_oci()
3226

33-
setup_rules_oci()
27+
step1.setup_rules_docker()
3428

35-
setup_rules_docker()
29+
step1.setup_container_structure_test()
3630

3731
# -----------------------------------------
3832

39-
load(
40-
"@pycross_image//bazel:workspace1.bzl",
41-
"setup_docker_containers",
42-
"setup_oci_containers",
43-
"setup_rules_pycross",
44-
)
33+
load("@pycross_image//bazel/workspace:step2.bzl", "step2")
4534

46-
setup_oci_containers()
35+
step2.setup_pycross_image_base_oci()
4736

48-
setup_docker_containers()
37+
step2.setup_pycross_image_base_container()
4938

50-
setup_rules_pycross()
39+
step2.setup_rules_pycross()
5140

5241
# -----------------------------------------
5342

54-
load(
55-
"@pycross_image//bazel:workspace2.bzl",
56-
"setup_pdm_deps",
57-
)
43+
load("@pycross_image//bazel/workspace:step3.bzl", "step3")
5844

59-
setup_pdm_deps()
45+
step3.setup_pypi_deps_for_pdm()
6046

6147
# -----------------------------------------
6248

63-
load(
64-
"@pycross_image//bazel:workspace3.bzl",
65-
"install_pdm_deps",
66-
)
49+
load("@pycross_image//bazel/workspace:step4.bzl", "step4")
6750

68-
install_pdm_deps()
51+
step4.install_pypi_deps_for_pdm()

‎bazel/BUILD.bazel‎

Lines changed: 0 additions & 14 deletions
This file was deleted.

‎bazel/container.bzl‎

Lines changed: 0 additions & 3 deletions
This file was deleted.

‎bazel/platforms/BUILD.bazel‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
load("//bazel/tools/example_test:defs.bzl", "example_test_filegroup")
2+
3+
platform(
4+
name = "linux_x86_64",
5+
constraint_values = [
6+
"@platforms//os:linux",
7+
"@platforms//cpu:x86_64",
8+
],
9+
visibility = ["//visibility:public"],
10+
)
11+
12+
platform(
13+
name = "darwin_x86_64",
14+
constraint_values = [
15+
"@platforms//os:macos",
16+
"@platforms//cpu:x86_64",
17+
],
18+
visibility = ["//visibility:public"],
19+
)
20+
21+
platform(
22+
name = "darwin_arm64",
23+
constraint_values = [
24+
"@platforms//os:macos",
25+
"@platforms//cpu:arm64",
26+
],
27+
visibility = ["//visibility:public"],
28+
)
29+
30+
example_test_filegroup(
31+
name = "example_test_files",
32+
srcs = ["BUILD.bazel"],
33+
)

‎bazel/python/deps/BUILD.bazel‎

Lines changed: 0 additions & 5 deletions
This file was deleted.

0 commit comments

Comments
 (0)