[Distroless] Enable Distroless Java SDK based container images

[damondouglas]

Summary

Add distroless variants to existing Java SDK container images:

• beam-sdk/beam_java17_sdk
• beam-sdk/beam_java21_sdk

Description

The Publish Beam SDK Snapshots, and build_release_candidate GitHub workflows build and publish Beam container images using docker. The distroless base images by design lack tools assumed to exist in Java Dockerfiles such as file system tools. In order to reuse the existing Dockerfile to add a distroless variant, we need to employ a multi-stage build approach.

Blockers

• [Distroless] Enable BuildKit by default for container image builds #32819
• [Distroless] Enable optional multi-stage build support #32820

Measures of Done

• Add Java SDK Dockerfile distroless variants (PR: Enable Java SDK Distroless container image variants. #33173)
• [Distroless] GitHub workflow runs checks on Java SDK (non ML) distroless variants #33226
• [Distroless] Add ARM-based Distroless Java SDK variants #32899
• [Distroless] Publish Java SDK distroless variants #33201

[kennknowles]

How's the status here? I would love to cherry-pick changes if they are not risky, in order to offer users an option to get fewer vuln scanner results.

[kennknowles]

It seems that Java may not be as up to date as we want so this may not be a good choice.

[Abacn]

Current Java and Python distroless containers are built from existing debian containers and copy Beam artifacts from there

beam/sdks/java/container/distroless/Dockerfile

Line 36 in dbae047

COPY --from=base /opt /opt

beam/sdks/python/container/distroless/Dockerfile

Lines 29 to 45 in dbae047

	# Prevents internal errors found with distroless container images and Flex templates.
	COPY --from=base /usr/lib/locale /usr/lib/locale
	# Contains header files needed by the Python interpreter.
	COPY --from=base /usr/local/include /usr/local/include
	# Contains the Python interpreter executables.
	COPY --from=base /usr/local/bin /usr/local/bin
	# Contains the Python library dependencies.
	COPY --from=base /usr/local/lib /usr/local/lib
	# Python standard library modules.
	COPY --from=base /usr/lib/python* /usr/lib/.
	# Contains the boot entrypoint and related files such as licenses.
	COPY --from=base /opt /opt

This may not work if the corresponding debian container hasn't build. In terms of testing, it is found beam_PostCommit_Java_Examples_Dataflow_V2 running on PR would fail if sdks/java/container has incompatible change, due to that distroless containers are built from gcr.io/apache-beam-testing/beam-sdk/beam_java17_sdk:latest instead of Beam source.

[github-actions]

This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.