try building docker image w/ Github URL

[kevinjqliu]

Rationale for this change

Are these changes tested?

Are there any user-facing changes?

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request modifies the Docker configuration for the spark-iceberg service. It changes the build context to use a GitHub URL, specifying the branch and Dockerfile. Additionally, it corrects the path for copying the spark-defaults.conf file within the Docker image build process.

Highlights

• Docker Build Context: The Docker build context for the spark-iceberg service was updated to use a GitHub URL, allowing the Docker image to be built directly from a specified branch of a GitHub repository.
• Dockerfile Path: The Dockerfile path was explicitly defined within the build context to ensure the correct Dockerfile is used when building the image.
• Configuration File Path: The path for copying the spark-defaults.conf file into the Docker image was updated to reflect its location within the dev/spark/ directory.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request modifies the Docker build process to use a remote GitHub repository as the build context. This introduces a critical dependency on a personal fork, which compromises build stability and reproducibility. A related change in the Dockerfile adjusts a file path to accommodate the new build context, but this breaks local build workflows. I've recommended reverting both changes to maintain a robust and self-contained build process.

[gemini-code-assist]

Building the Docker image from a personal GitHub repository URL introduces a significant risk to the project's stability and reproducibility. This configuration makes the build process dependent on an external fork (kevinjqliu/iceberg-python) and a specific branch that is outside the control of this project's maintainers. If the repository is removed, made private, or the branch is altered, the build will fail. It's strongly recommended to use a local build context to ensure that the build relies only on the code within this repository.

    build: spark/

[gemini-code-assist]

This path change is coupled with the modification in docker-compose-integration.yml to build from a Git URL. While it makes that specific build work, it breaks the ability to build this Docker image locally using a command like docker build dev/spark, as the path dev/spark/spark-defaults.conf will not be found within the dev/spark build context. This change should be reverted along with the change in docker-compose-integration.yml to maintain support for local builds.

COPY --chown=spark:spark spark-defaults.conf ${SPARK_HOME}/conf/