Skip to content

feat(docker-compose): use Lakekeeper as the Iceberg Catalog when launching via docker compose#4274

Open
mengw15 wants to merge 16 commits intoapache:mainfrom
mengw15:Lakekeeper-Single-node-deployment
Open

feat(docker-compose): use Lakekeeper as the Iceberg Catalog when launching via docker compose#4274
mengw15 wants to merge 16 commits intoapache:mainfrom
mengw15:Lakekeeper-Single-node-deployment

Conversation

@mengw15
Copy link
Copy Markdown
Contributor

@mengw15 mengw15 commented Mar 9, 2026

What changes were proposed in this PR?

This PR updates the single-node Docker Compose deployment to integrate Lakekeeper as the Iceberg REST catalog.

Changes to bin/single-node/docker-compose.yml:

  • Added lakekeeper-migrate init container: runs database migration before Lakekeeper starts
  • Added lakekeeper service (vakamo/lakekeeper:v0.11.0): REST catalog server with health checks
  • Added lakekeeper-init container: bootstraps default project, creates MinIO bucket, and configures warehouse

Changes to bin/single-node/.env:

  • Added Lakekeeper config: database URLs, encryption key, base URI
  • Added REST catalog config: warehouse name, S3 bucket, region, catalog URI
  • Set STORAGE_ICEBERG_CATALOG_TYPE=rest
  • Added IMAGE_REGISTRY, IMAGE_TAG, TEXERA_PORT, MINIO_PORT, TEXERA_SERVICE_LOG_LEVEL variables

Any related issues, documentation, discussions?

issue: close #4370

How was this PR tested?

Manually tested

Was this PR authored or co-authored using generative AI tooling?

co-authored with Claude

@github-actions github-actions bot added the build label Mar 9, 2026
@mengw15 mengw15 self-assigned this Apr 6, 2026
Copy link
Copy Markdown
Contributor

@bobbai00 bobbai00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good! Left some comments

Comment thread bin/single-node/.env Outdated
Comment thread bin/single-node/.env Outdated
Comment thread bin/single-node/docker-compose.yml Outdated
@bobbai00
Copy link
Copy Markdown
Contributor

@xuang7 @Ma77Ball Can you help test the PR?

@bobbai00 bobbai00 changed the title feat: introduce Result Service using Lakekeeper as REST catalog for Iceberg - single-node deployment feat: use Lakekeeper as the Iceberg Catalog when launching via docker compose Apr 17, 2026
@bobbai00 bobbai00 changed the title feat: use Lakekeeper as the Iceberg Catalog when launching via docker compose feat(docker-compose): use Lakekeeper as the Iceberg Catalog when launching via docker compose Apr 17, 2026
@Ma77Ball
Copy link
Copy Markdown
Contributor

Ma77Ball commented Apr 18, 2026

@bobbai00 I am not sure if it is just a problem on my end, but the script to build the images appears not to work. I'm testing this PR, but when building the images, it fails and appears to be missing the License service that should be included. If needed, I can raise a Pr if this is a known issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integrate Lakekeeper REST catalog in single-node Docker Compose deployment

3 participants